The Jagged Intelligence of Genius

Epistemic Humility: Are we repeating the errors of the Nuclear Age?

A decade ago, Google pulled out of Project Maven to quell an internal revolt against working with the U.S. military. AI was too powerful, the critics said. The military couldn’t be trusted to wield it.

For a while it seemed those days were over. Frontier AI labs (Google included) got back in the fight, publicly committing to help America win the AI race. But there are troubling signs that the Valley is backsliding.

Why is this happening? A few reasons. Partly it reflects AI builders’ deep ambivalence, even pessimism, about the technology they are creating. Partly it reflects their barely concealed loathing of the Trump administration and its policies. But the fundamental reason is psychological.

Andrej Karpathy coined the term “jagged intelligence” to describe how AI is smart about some things but dumb about others. Humans have the same jaggedness—technologists more than most.

Technology is built by brilliant idealists. Those qualities enable a certain type of person to build the future. But brilliance bleeds into hubris and idealism bleeds into naiveté. Too many assume that technical genius confers a special political wisdom. That building technology qualifies them to control how our society deploys it.

Some of these brilliant idealists have decided America, its leaders, and its employees can’t be trusted with the powerful tools they’ve built. They think micromanaging access is the right way to protect democracy and human rights.

But they’re wrong. So wrong that they don’t realize how their own activism makes the dangers they fear more likely. Restricting AI in America doesn’t make us safer or freer. It simply gives an advantage to actual tyrants—the Chinese Communist Party—in a technology race. America winning this race is humanity’s best shot that AI will be used for good. China winning would be a disaster.

How can I write this so confidently? Because we’ve seen this movie before. Within living memory, Americans debated the morality of building another paradigm-shifting technology: nuclear weapons. Jagged intelligence was on full display in those debates. Smart people argued passionately for policies they thought would make us safe—but that in fact weakened America and doomed millions trapped behind the Iron Curtain.

Knowingly or not, AI researchers are recreating the debates and errors of an earlier generation. We have to learn from that history. And we have to choose differently by helping America build the technology that will determine whether the 21st century is free or unfree.

Debating The Gadget

Los Alamos during the Second World War had possibly the greatest concentration of technical genius ever assembled. As the scientists raced to deliver an atom bomb—they called it “The Gadget” for security reasons—naturally they started to discuss what it meant for the world. And they were alarmed.

The Manhattan Project was conducted with the utmost secrecy. The public knew nothing about it. Soviet scientists were excluded.

Some feared this secrecy was a threat to democracy and peace. America couldn’t be trusted as the sole arbiters of nuclear knowledge. Niels Bohr, one of the most accomplished physicists at Los Alamos, argued passionately for preemptive disclosure to the Soviets so that Stalin wouldn’t view it as a threat. Sounding like leaders today who call for deeper engagement with China, Bohr believed that only an “open world” based on transparency and international control could prevent an arms race and a more devastating war. Bohr was so committed to these ideas that he considered moving to Russia for scientific exchange. (Winston Churchill found Bohr’s correspondence with Russian scientists about this so alarming that he remarked, “Bohr ought to be confined or at any rate made to see that he is very near the edge of mortal crimes.”)

These fears grew when it became clear atomic weapons wouldn’t be needed to knock Germany out of the war. Few had qualms about building the bomb to stop Nazi Germany, which had an active nuclear program. Using it against an almost-beaten Japan—or against Russia later on—was a different matter. Why drop the bomb? Why finish it at all? One scientist resigned over these questions. Others, led by Leo Szilard, circulated a petition urging President Truman not to use the bomb without giving Japan forewarning and a chance to surrender.

Reading accounts, it’s hard not to be struck by these scientists’ earnestness and idealism—but also their naiveté. While they debated, the U.S. government was ordering 375,000 Purple Hearts for an invasion of the Japanese home islands. American lives depended on this technology. Their skin was in the game in a way the Los Alamos scientists’ was not.¹

Ironically given his later views, Robert Oppenheimer put his finger on the problem when he dismissed the Szilard petition: “What do [the scientists] know about Japanese psychology? How can they judge the way to end the war?” It was the scientists’ job to build the thing and to inform political leaders about what it could do. But in a democracy, the ultimate decision wasn’t theirs to make.

Edward Teller identified an even more fundamental problem. As with AI today, the real question wasn’t whether to make the bomb or not—it was who would make it first. “Our work was significant,” Teller said, “because it gave the power of the first atomic weapon to Truman rather than to Stalin.” The scientists at Los Alamos couldn’t see that by building The Gadget, they averted a future where one of the bloodiest butchers of the 20th century got a nuclear monopoly. So much for democracy, peace, and an open world in that future.

Two Paths for Technologists

I’ll end with two stories that show two very different approaches to technology and national service.

Air Force Colonel Edward Hall is in the pantheon of American patriots for his work on missiles during the Cold War. During the final days of World War II, he scoured German research facilities for secrets about its missile program. After the war, he led America’s development of Minuteman, the first solid-fueled, instant-launch intercontinental ballistic missile. Ed Hall was no one’s fool. He had seen the horrors of war and he built weapons so America could know peace.

As so often happens, Ed’s genius ran in the family. His little brother Theodore was one of the youngest scientist at Los Alamos. He played a critical role in testing the nuclear device that the United States ultimately dropped on Nagasaki.

Here the brothers’ stories diverge. As Ted worked on the bomb, he had political discussions with other scientists at Los Alamos. He became increasingly alarmed by the idea that the United States would have a monopoly on nuclear weapons after the war. “It seemed to me that an American monopoly was dangerous and should be prevented,” he recalled half a century later. The peace of the world was at stake—or so it seemed to him.

So in 1944, Ted walked into a Soviet trade mission in New York and announced that the Americans were building a superweapon. He later shared details about the bomb’s design. Combined with information from another atomic spy, Klaus Fuchs, Hall’s information accelerated the Soviet nuclear program. The Soviets detonated their first atomic bomb years ahead of schedule in 1949.

Ted Hall’s jagged intelligence led him down a treacherous path. He started down the path in the name of world peace and ended up with the blood of every death from communism since 1949 on his hands.

Genius is no guarantee of wisdom. Ted Hall proves that. The question is whether we’ll learn from the half-century of misery he inadvertently helped create—or repeat the error with technologies far more consequential than the atom bomb.

Will we be Ed or Ted? Faramir or Boromir?

1. This is to say nothing of the Japanese lives saved through use of the Bomb. https://stratechery.com/2026/an-interview-with-gregory-allen-about-anthropic-and-the-u-s-government/

GA: So I think it is it is widely understood and accepted that the use of nuclear weapons reduced American casualties.

BT: Oh it reduced far more Japanese casualties.

GA: That is the key. That is the key that I don’t think is widely understood. The US Air Force, and even the British Air Force, were in the process of bringing all the bombers that had been bombing Germany to Japan and they were going to be running massive bombing raids effectively 24/7. So even all the casualties that Japan had been experiencing up until that point is actually not a good proxy for just how many people would have died in non-nuclear strategic bombing of Japan under Curtis LeMay if the war had continued. So it’s very tragic that a lot of Japanese civilians died, but I do think the evidence is actually quite clear that the use of nuclear weapons and the shortening of the war actually saved net Japanese civilian lives.

BT: It’s a great point because those are lives that are saved, that’s not talking about the Japanese fighting, it’s just talking about lots of civilians die in war, particularly if you’re running bombing raids 24/7, it’s a great point.