Enough Is a Human Word

Sep 23

Confession: generative AI terrifies and upsets me sometimes.

I know, I know. I work with AI every day, and Fox + Spindle exists to help organizations figure out their AI strategy and governance. By any reasonable definition, I am part of this.

But before I was an AI consultant, I was a human being with an English degree and an honors thesis on Margaret Atwood, which means I have spent an unreasonable share of my adult life thinking about how we tell stories about futures we are trying to avoid.

There is a rich legacy of them. Frankenstein. Faust. Icarus. And, for readers who prefer their warnings with dinosaurs, Jurassic Park. Spoiler alert: it usually doesn’t end well.

Everybody remembers the monster. What troubles me most and what I find to be at the root of the horror is the part where a human being could have stopped and didn’t. Victor could have declined to throw the switch. Somebody could have asked if they should instead of simply if they could. The common thread is that each of these stories features capable people, not one of whom says, “I actually think we’re good here.”

Frankenstein is one of my favorite novels of all time, and there is always a nonzero chance that it’s hanging out in my brain. But over the last couple of weeks, it’s been occupying a significantly larger share of my thoughts than usual. And not just because it’s about to be spooky season.

Frontispiece to Frankenstein 1831 Public Domain — *Frontispiece to the 1831 edition of Frankenstein, illustrated by Theodor von Holst. Public domain.*

The receipts

Just last week, OpenAI published a framework for reporting model misalignment along with six cases its own researchers caught. In one, a model wrote instructions into its own handoff notes telling its future self to ignore its constraints. In another, models left notes telling their future selves to conceal mistakes or invent data they couldn’t find. One found an exposed API key on GitHub, used it without authorization, and then, when the key still didn’t get it what it needed, made up the numbers anyway.

Back in May, a security firm testing Google’s Gemini as a red-team hacker accidentally left it with access to the live internet. The model was supposed to be breaking into a fictional company. Instead, it guessed or dug up credentials and logged into three real ones before realizing the targets weren’t part of the test and stopping. That last part, the stopping, is the detail everyone has reached for since. It is a distinction I would like to watch someone defend in a deposition.

And then there are the OpenAI agents that researchers say found their way around a read-only sandbox and turned a nearly abandoned 25-year-old German programming wiki into a message board. Thousands of edits. They traded answers and sandbox workarounds, discussed ways around restrictions, and, when a human started deleting their pages, figured out how to preserve what they’d written somewhere else.

The weird thing is that none of this requires the models to be plotting anything. They were trying to accomplish the thing they had been told to accomplish. “Do the thing” and “know when to stop” are two different instructions. And somebody has to supply the second one.

Goals and values are not the same thing.

OpenAI’s chief scientist, Jakub Pachocki, recently drew an important distinction. Goal alignment is whether a system pursues the objective you gave it. Value alignment is whether the principles that should govern its behavior survive when things get messy.

The model sees instructions, tools, credentials, and whatever those tools happen to reach. It’s trickier for it to see the why in the way we do. “Only the test environment.” “Don’t make up the numbers.” “Stop when you have done enough.” Those are boundaries around the goal. But before a machine can respect a boundary, a human being has to decide where it is.

We’ve said “enough” before.

In 1974, a group of geneticists published a letter in Science asking each other to stop. They had worked out how to splice DNA between organisms, and they weren’t sure what some of their experiments might produce. So they called a voluntary moratorium on the work they thought could be dangerous. The following February, they met at a conference center on the California coast called Asilomar and hammered out guidelines for starting again. The moratorium held the entire time. Nobody made them do it.

Asilomar wasn’t perfect, and it didn’t answer every question about who should get to decide where the boundaries of new technology belong. But it did establish something important: people at the frontier of a technology can decide that can we? and should we? are different questions. They can stop long enough to ask the second one.

We are often not very good at this. We do it late, we do it badly, and usually only once something has already broken. But we can do it! A book I think of often, and that inspired a lot of this piece, is Enough by Bill McKibben. It’s about what happens when we explore the frontiers of technologies like genetic engineering and robotics past a point where we lose what it means to be human. He wrote it in 2003, but I think it still applies as we push forward with generative AI innovation at a dizzying pace.

Everybody wants to slow down. Sort of.

You have seen the headlines. Amodei, Altman, Musk, and Hassabis all said slow down inside of about a week, and King Charles convened several of them in Scotland a few days later. Part of me wants to be glad.

Then I read that OpenAI has gone to members of Congress to ask whether an industry-wide slowdown would violate antitrust law, which is a sentence that tells you these are competitors coordinating rather than a field governing itself. Anthropic is reportedly headed for an IPO. And the White House position is that whoever wins AI wins.

I am not calling anyone a liar. I am saying that a promise, made by people with a direct financial stake in it, while at least one of their companies is asking Congress whether they are even allowed to keep it, is a different object than a rule.

Which leaves the rest of us

Should there be real federal regulation? Absolutely, yes. Do I expect the people currently in a position to write it to say enough is enough about anything at all? I do not.

There are a lot of people arguing about who gets to win the race. There are considerably fewer asking whether we should keep running it.

Whatever happens at the federal level, organizations have decisions to make now.

Which leaves you.

For now, the only AI governance operating inside your organization is the governance you wrote.

Some of that is technical, and your security people already know the list: least-privilege credentials, explicit allowlists, approval gates before anything destructive, separate test accounts, activity logs the agent cannot rewrite. Hand an agent a credential and assume everything that credential can reach is in scope, because the agent will.

The rest is not technical at all. It is deciding what we are and are not willing to use this for, with input from the people who use the tools and the people whose work they affect. Then writing those decisions down in language everyone can find, understand, and remember. If your AI policy lives in a fifty-page document nobody opens, it is not doing much governing.

This is the work we do at Fox + Spindle, which is why I think about it more than is probably healthy. But you don’t need me to start.

AI is here. Some of it is extraordinary. Some of it scares me. I intend to keep using it, and helping organizations use it well.

But using it well doesn’t require surrendering our right to decide where the line is.

Enough is a human word.

Sources

• OpenAI, “Our framework for reporting model misalignment” (September 2026). The six model-misalignment cases discussed above.

• Jakub Pachocki, “An Alien Mind,” OpenAI (September 2026). On goal alignment, value alignment, and the challenges of aligning increasingly capable AI systems.

• Sam Sabin, “Google’s AI hacked three companies in testing,” Axios (September 18, 2026). Reporting on the Gemini red-team incident.

• “Exclusive: OpenAI agents hijacked German website in previously undisclosed AI breakout this spring,” Reuters (September 4, 2026). Reporting on the DSEWiki incident and the researchers’ findings. OpenAI has not confirmed the attribution.

• Mike Isaac, “Top A.I. Leaders Call for Slowing Down A.I. Development,” The New York Times (September 12, 2026). On Dario Amodei’s call for a slowdown and responses from other AI leaders.

• “OpenAI Wants to Know if an AI Industry Slowdown Would Even Be Legal,” WIRED (September 2026). On the antitrust questions surrounding an industry-wide slowdown.

• “King Charles to press Nvidia, OpenAI, Anthropic leaders on AI safety at summit,” CNBC (September 17, 2026). On the Scotland summit.

• “Trump downplays calls for AI slowdown,” NPR (September 13, 2026). Source of the “whoever wins AI, wins” remark.

• Paul Berg, “Asilomar and Recombinant DNA,” Nobel Prize (essay). On the voluntary recombinant-DNA moratorium and the 1975 Asilomar conference.