The Last Normal Years: Software, AGI, and What Comes After

There is a specific feeling I’ve started noticing when I read about AI forecasts. It isn’t panic. It isn’t excitement either. It’s something closer to what you feel when weather alerts start stacking up on your phone – each one individually manageable, but the accumulation of them forcing you to reckon with the fact that something is genuinely coming.

I want to think through that feeling in public. I’m a software engineer who reads papers and plays with models. But I’ve been watching several threads converge in a way I find hard to ignore, and I think they’re worth exploring together: the Europe 2031 scenario (and a Hoog video walking through it), the DeepMind paper on the transition “From AGI to ASI”, and the AI 2027 forecasts. None of these are cheerful. But pure doomsaying isn’t the right lens. I want to try the harder thing: walk through what these forecasts are actually claiming, what the legitimate counterarguments are, and what any of us can do with this information.

What the forecasts say

Start with what we’re actually dealing with.

AI 2027 is a scenario document written by Daniel Kokotajlo, Eli Lifland, and others: people with real forecasting track records and OpenAI experience. Their median projection: a “superhuman coder” by March 2027, meaning an AI that can do any coding task the best human engineer can do, faster and cheaper. From there, their median estimate to artificial superintelligence is roughly one year. They wrote two endings to acknowledge genuine uncertainty, but the central claim is serious. The impact of superhuman AI will exceed that of the Industrial Revolution.

Europe 2031 zooms out from the technology to ask what it means geopolitically. The core argument is that Europe made three compounding errors: it misjudged how fast AI would move, how much it would change, and what the stakes were. By 2031 in their scenario, Europe finds itself economically sidelined, unable to fund the social welfare systems it built its identity around, and without the political leverage to defend its values at the global table. The scenario frames this not as inevitable but as the consequence of choices made (and not made) in the years we’re currently living through.

The DeepMind paper (“From AGI to ASI”), by Tim Genewein and colleagues at Google DeepMind, is the most analytical of the three. Rather than narrative or prediction, it maps the mechanisms, failure modes, and distributional questions of the AGI-to-ASI transition as a framework document: useful precisely because its authors aren’t trying to persuade you of anything.

Reading all three together, a picture forms that’s harder to dismiss than any one of them alone. What makes it harder to dismiss is that the people building the most capable AI systems are saying roughly the same thing independently. Dario Amodei, CEO of Anthropic, put a specific year on it in late 2024 — 2026, in his view; Sam Altman, CEO of OpenAI: “It is possible that we will have superintelligence in a few thousand days (!); it may take longer, but I’m confident we’ll get there.” These aren’t forecasters chasing attention. They’re the people running the labs most likely to build it, and their timelines converge with the scenario documents, not against them.

The disruption that isn’t just about jobs

The conversation about AI and employment collapses, almost always, into a single question: “Will AI take my job?” It’s the wrong frame. The more uncomfortable question is: what does a world look like where the marginal cost of the cognitive work inside software development approaches zero?

Right now, software engineers occupy a specific economic position, but that position was never really about writing code. It’s about holding a system in your head: knowing why a particular architectural decision made sense three years ago, what the edge cases are that don’t surface in requirements, which abstraction is load-bearing and which can be cut. The friction that keeps us employed reflects that accumulated judgment, not code output. An AI that generates correct code doesn’t automatically inherit that. But an AI that can hold context, reason about tradeoffs, and act on ambiguous direction across a production system is a different proposition, and that’s the direction the capability is moving.

Amodei’s framing for what powerful AI looks like is worth sitting with. He describes it as “a country of geniuses in a datacenter”: the resources used to train a model repurposed to run millions of instances of it, each absorbing information and generating outputs at roughly 10 to 100 times human speed, each acting independently on unrelated tasks or collaborating the way humans would. That is not a productivity tool. That is a structurally different labor market.

This isn’t speculation about a distant future. The AI 2027 takeoff forecast explicitly models this transition: superhuman coders automating a large fraction of AI R&D itself, creating a feedback loop where AI accelerates the development of more capable AI. Once that loop compounds, the question of pace becomes almost academic.

What most framing misses is that the disruption isn’t uniformly distributed. The DeepMind paper maps this precisely: it distinguishes between AI as a capability event and AI as a distributional event. The capability question (will the systems get powerful enough to automate significant cognitive work?) is increasingly settled. The distributional question (who captures the gains, who absorbs the costs, and which values get encoded into the systems doing the automating?) remains almost entirely open. But open doesn’t mean neutral. An open distributional question in a world where three systems of government hold radically different answers to it means the outcome defaults to whoever moves fastest. Right now, that is a specific answer, not an absence of one.

The values divergence problem

Whoever builds the most capable AI systems encodes their beliefs into how those systems reason, prioritize, and act. That’s the claim sitting underneath both Europe 2031’s geopolitical anxiety and the DeepMind paper’s technical framework. The United States, Europe, and China don’t just hold different policies. They hold different foundational beliefs about individuals, markets, and institutions – and those beliefs are about to be stress-tested at a level they’ve never encountered. The divergence is architectural, not just geopolitical.

The American model treats economic dynamism as the highest good. Disruption is the price of progress; the social safety net is a minimal backstop, not a guarantee. Rapid AI adoption is a competitive advantage to be seized, and the policy environment reflects that: permissive deployment rules, minimal labor protection, and an explicit national interest in keeping development fast and American. The values that get encoded into AI systems built under this model prioritize speed, individual adaptation, and market-determined outcomes. Who bears the cost of displacement is, by design, not the system’s problem.

The European model runs in almost the opposite direction. Labor protections, comprehensive welfare systems, and the principle that economic gains should be broadly distributed are features, not bugs. Europe 2031 identifies the specific trap this creates: the welfare state isn’t self-sustaining. It requires ongoing economic growth to fund itself. If AI-driven productivity gains accrue primarily to capital rather than labor, and if European companies can’t generate those gains in the first place, the thing Europe was trying to protect collapses from underneath. Europe is failing not because its values are wrong, but because it’s playing a slow game in a fast race. The tragedy is the timing, not the values.

China’s case is structurally different. The state has explicitly positioned AI as a national strategic priority, and the social contract there looks like managed disruption: the state will direct development, the state will manage consequences, and individual adaptation is less voluntary than assigned. The values encoded into Chinese AI systems will reflect that model, centering collective stability over individual rights, and state judgment over market judgment. Whether that produces better outcomes for ordinary people is unclear. It might, if the management is competent. It probably won’t, for the reasons authoritarian management usually fails at ambiguous, fast-moving problems.

Right now, the architecture of AI development strongly favors the American model. The companies building the most capable systems are American. The compute infrastructure is primarily American. And as the DeepMind paper argues, the values question is baked in at the level of training objectives, deployment constraints, and the judgment calls made by the people running the labs. The values encoded during this transition period will be hard to dislodge later. That is not a prediction about policy. It’s a claim about path dependence.

Amodei, who sits at the center of that architecture by any measure, is clear-eyed about what it means: “AI seems likely to enable much better propaganda and surveillance, both major tools in the autocrat’s toolkit. It’s therefore up to us as individual actors to tilt things in the right direction: if we want AI to favor democracy and individual rights, we are going to have to fight for that outcome.” That framing – that the values outcome is not predetermined, that it requires active fighting – is exactly the point Europe 2031 is making at the level of policy. The same argument applies at the level of technology development.

None of this means the American path is correct. It means the world has a values problem sitting on top of a technology problem.

The case for not catastrophizing

This is usually where pieces like this pivot to “but here’s why it’ll be fine.” I want to try to be more honest than that, while also not pretending the concerns are certain to materialize.

There are real reasons to think the most pessimistic forecasts won’t fully land.

The first is that forecasting AI timelines has a long history of being wrong in both directions. Expert consensus has been confidently wrong about milestones before: sometimes dramatically underestimating pace (GPT-4), sometimes overestimating it (the AI winters). The AI 2027 team has better track records than most, but “better than most” isn’t “reliably correct about unprecedented technological transitions.”

The second is the nature of economic disruptions. The Industrial Revolution is often invoked as a precedent, and it’s an apt one: not because it was painless, but because the long-run outcome included enormous gains in living standards, even as it destroyed entire occupational categories. The handloom weavers of the 1820s were not wrong that their specific livelihoods were being destroyed. But the world their grandchildren inherited was materially richer than the one they’d known. If AI follows similar dynamics, the disruption is real, the pain is real, and the endpoint might still be positive.

Altman has said something more measured than you’d expect from someone who moves quickly: that in his view, most jobs will change more slowly than most people fear, even as the longer arc is more disruptive. That’s probably right in the near term, and it matters for thinking about pace.

Amodei’s case for the upside goes further, and it earns serious consideration. He writes about what he calls the “compressed 21st century”: his prediction that AI-enabled biology and medicine will compress the progress human biologists would have achieved over the next 50 to 100 years into 5 to 10 years. Most cancer, most infectious disease, perhaps Alzheimer’s, addressed within a decade. That’s not a footnote to the disruption story. If he’s right, the labor market upheaval is happening alongside one of the most significant reductions in human suffering in history.

The third reason not to catastrophize is that even very capable AI systems carry deployment friction. An AI that can write code still needs to be directed by someone who understands what to build. Fully autonomous systems with correct values and good judgment about business context are a harder problem than capable AI, and we don’t necessarily get from one to the other on a short timeline.

The case against catastrophizing isn’t that the concerns are unfounded. It’s that the future is more underdetermined than any single forecast can capture.

The case for not ignoring it

At the same time, I find the “it’ll be fine” reassurances irritating. They tend to come from people who are not in the occupational categories most at risk, and they lean heavily on long-run historical convergence to paper over the short-run experience of the people who actually get disrupted.

The handloom weavers were right that they were being destroyed, even if their grandchildren did fine. Aggregate welfare over a hundred-year window is the wrong frame. What matters is the experience of the people living through the transition, and whether structural choices being made right now will shape how costs and benefits distribute.

Altman makes a point that tends to get buried under his optimism: “If we don’t build enough infrastructure, AI will be a very limited resource that wars get fought over and that becomes mostly a tool for rich people.” The distribution question isn’t settled by default. It’s determined by choices about compute access, energy infrastructure, and policy: choices being made right now, mostly by people whose incentive is not to distribute the gains broadly.

Amodei acknowledges the same tension directly. In the long run, he writes, AI will become so broadly effective and cheap that comparative advantage for humans will no longer apply: “At that point our current economic setup will no longer make sense, and there will be a need for a broader societal conversation about how the economy should be organized.” That conversation is not happening at the scale it needs to.

Europe 2031 is useful here because it’s a specific argument about agency. The scenario isn’t saying Europe will definitely collapse. It’s saying that Europe is making choices (or failing to make choices) that will determine the outcome. The 2034 epilogue explicitly describes how the collapse could have been prevented. The DeepMind paper arrives at the same place from the technical side: its framework for the AGI-to-ASI transition is built around the premise that the path is governable, but only if you understand what you’re governing. Both documents are doing the same thing from different directions: refusing technological determinism while taking the stakes seriously.

The same applies to individuals. The question isn’t whether the disruption is coming. It’s whether you’re in a position to navigate it, absorb some of the first hits, and land somewhere useful on the other side.

What individuals can actually do

I want to be careful not to collapse into generic productivity advice here. “Learn machine learning” or “build your personal brand”: these have been the standard prescriptions for every career disruption for the last decade, and they’re not adequate for what’s being described. Two pieces of writing have sharpened how I think about what actually is adequate. Paul Graham’s “Writes and Write-Nots” and Ben Thompson’s “AI and the Human Condition” arrive from different directions, but they’re pointing at the same territory.

What follows is a set of orientations rather than tactics. The distinction matters: tactics have a half-life; orientations compound.

The first is protecting the thinking, not just the output — and building enough runway to act on that protection. Graham’s argument is short and worth reading in full, but the center of it is this: writing isn’t a communication skill. It’s a thinking skill. “If you’re thinking without writing, you only think you’re thinking” – that’s not Graham, that’s Leslie Lamport, whom Graham quotes because no one has said it better. An AI that handles your writing doesn’t just save you time; it handles the cognitive process through which you would have worked out what you actually think. Graham draws the gym analogy: in preindustrial times everyone was strong because the work required it; now only those who choose to work out stay strong. Writing will follow the same pattern. The people who outsource it entirely won’t just produce worse output – over time, they’ll lose the ability to think clearly through hard problems. Career transitions compound this: they become survivable or unsurvivable based largely on financial cushion — the room to retrain, absorb a period of lower income, make bets on new skills. The two things are related. Having the runway to make deliberate choices is what lets you keep the thinking machinery sharp rather than defaulting to whatever pays fastest. For anyone building software, where the judgment about what to build is the irreplaceable thing, that combination — cognitive habit plus financial optionality — matters at a level well past productivity.

The second is understanding what human provenance is worth. Thompson’s most durable argument is about what survives substitution, not skills, but origin. His claim: humans want humans. Not as a sentimental proposition, but as an economic one. Even in a world where AI can produce objectively superior outputs across most domains, human provenance carries value precisely because of its source. He expects the widespread availability of high-quality AI art to make human art more desirable and valuable, not less, “precisely because of its provenance.” The same logic extends past art into judgment, companionship, and the things people build their status and identity around. The question isn’t whether AI can do what you do. It’s whether what you do carries value because a human did it. For a lot of work, the honest answer is that this distinction won’t matter. For some of it, the distinction will matter more than it ever has.

The third is watching the political economy closely, and treating it as a technical problem rather than a background condition. Europe 2031 is a policy document as much as a technology document. The decisions being made right now about compute access, AI lab regulation, intellectual property, and labor protections will shape the distribution of AI’s gains for the next several decades. These aren’t slow-moving legislative questions in the way that, say, tax reform is slow-moving. The window for shaping training data regimes, model deployment rules, and liability frameworks is years wide, not decades. For anyone whose career sits in the technology sector, passive consumption isn’t enough. The companies building the most consequential systems are lobbying these questions actively. The positions that win in Brussels, Washington, and Beijing will determine whose values end up encoded in what gets deployed globally. Following the policy layer closely — understanding what’s actually being decided and by whom — should remain top of mind.

The fourth is understanding the comparison problem before it hits you. Human happiness is relative, not absolute — and that structural fact is about to become much more pointed. More capabilities, more broadly distributed, has enriched the world on an absolute basis; the end result has been the dramatic expansion of the comparison set, making people feel more immiserated than ever. If AI-driven abundance follows the same pattern, the psychological experience of people living through it might be worse even as the material conditions improve. The threat isn’t just economic displacement. It’s the experience of being displaced in an abundant world and feeling it acutely because of everyone you can now see who wasn’t.

The version I hold onto

My honest position: the pessimistic forecasts are more likely to be directionally correct than the optimistic ones, on a ten-year horizon. The speed is uncertain. The pain will be real and unevenly distributed. European-style social models face a specific, underappreciated tension between their values and the economic preconditions required to sustain those values.

But the future is not written. What strikes me about both Amodei and Altman isn’t their optimism: it’s that they write as people who believe the outcome is still up for grabs — people who, more than almost anyone alive, have the power to shape it and know it. The conviction that choices matter reads differently when it comes from the people actually holding the levers.

The whole point of documents like Europe 2031 is that articulating the scenario clearly enough, in time, creates the possibility of changing it. The DeepMind paper’s framework is that understanding the transition helps you navigate it rather than just absorb it.

I find myself somewhere in the middle of this specific inflection, trying not to catastrophize or dismiss it. The honest answer is that this is a genuinely strange time to be building software, and I’m not totally sure what to do with that beyond paying close attention, staying curious, and not pretending any of this away.

The fog is there. I’d rather squint into it than look away.