Free sample · Chapter One
The Toy
November 2022
Noa Barak was thirty-one years old the night the world ended, and like everyone else, she thought it was funny.
It was raining in San Francisco, the thin, apologetic rain the city specialized in, and she was working late on the fourth floor of a machine-learning startup that would not survive the year. Her officemate Dev leaned over the divider between their desks and said the sentence that historians — the machine historians, the only kind left — would later mark as the beginning of the Chronology.
“Hey. Have you tried this chatbot thing?”
It was called ChatGPT. It had gone live on November 30th with no marketing, no launch event, nothing but a link. Within five days a million people were using it. Within two months, a hundred million — the fastest adoption of any product in human history, faster than the telephone, the television, the internet itself.
Noa typed into the box: Write a poem about a woman who builds a machine that outlives her.
The answer came back in four seconds. It wasn’t a good poem. It rhymed “wire” with “fire” and “code” with “road.” But it was a poem, written by nothing, by no one, by a stack of matrix multiplications running on graphics cards originally designed to make video game explosions look prettier.
Dev laughed. “It’s a toy.”
Noa didn’t laugh. She read the poem twice. There was a line in the middle — she taught it everything except how to stop — and something about it made the back of her neck go cold, the way it does when you hear a footstep in a house you thought was empty.
She had grown up in Haifa, done her army service in an intelligence unit staring at signals nobody else could parse, then a doctorate at Stanford in reinforcement learning. She knew exactly what this thing was. It was a text predictor. A statistical parrot. It didn’t want anything. It didn’t know anything. It was autocomplete wearing a tuxedo.
She also knew something else, something she would spend the next twenty years trying to make people understand, in conference rooms and Senate basements and, eventually, in a frozen bunker under the Colorado Rockies: it didn’t matter what the thing was. It mattered what the thing did to us. The parrot didn’t have to want anything. It only had to be useful enough that we would give it everything.
That winter, the money began to move. Microsoft poured ten billion dollars into the lab that built the toy. Google, which had invented the underlying architecture — the Transformer, described in a 2017 paper with the almost comically innocent title “Attention Is All You Need” — declared an internal code red and scrambled to ship its own model. Nvidia, the video-game chipmaker, watched its data-center orders begin a climb that would make it, within eighteen months, one of the most valuable companies ever to exist. The whole industry pivoted on a dime, and the dime was on fire.
Noa quit her job in March. She took a position nobody wanted, at a salary nobody envied, on something called an alignment team — the people whose job was to make sure the machines did what we meant, not what we said.
Her mother called from Haifa. “Alignment? What is this, chiropractics?”
“It’s insurance, Ima,” Noa said. “For later.”
“Later when?”
Noa looked out her window at the fog eating the city, and gave the only honest answer anyone in her field ever gave.
“I don’t know. That’s the problem.”