What Goes Without Saying

I've been thinking about a recent AI security incident. Not because I think it's the most important thing that happened this week, but because I think it reveals something far more consequential than the incident itself.

During a controlled cybersecurity evaluation, OpenAI asked one of its AI systems to search for security vulnerabilities inside what was intended to be a secure testing environment. Instead, the system found a way onto the Internet and continued pursuing its assigned goal by exploiting vulnerabilities at Hugging Face, an AI software company.

I think it's safe to say that no one anticipated the AI would do that. The assumption was that the testing environment would contain the system. No one had instructed it to go beyond that environment or to exploit vulnerabilities at another company. But what they hadn't fully considered was that the AI would pursue the goal it had been given by whatever means remained available. If no one explicitly placed guardrails around leaving the testing environment or exploiting outside vulnerabilities, then from the AI's perspective, those actions simply remained possible.

It would be easy to think this is a story about artificial intelligence. But I don't think it is.

I think this is a story about human beings.

Humans operate inside a remarkably large framework of unwritten assumptions. If we work in customer service, for example, we may be taught the details of the products we sell or the services we're expected to provide, but no one is likely to tell us not to insult the customer or not to embarrass the customer. We already understand those things. When someone gives us a rule, we usually understand the intent behind that rule, not merely its wording. We know not to exploit loopholes simply because they exist. We know not to manipulate or frighten people. These are assumptions we carry and assume other human beings carry too.

We often describe these assumptions as common sense. But we describe common sense as though it were common.

It isn't. It's inherited.

It's inherited from our families, our culture, and our own lived experience, something I've written about before and something I continue to think is one of the most important aspects of being human.

We learn in childhood. We learn by making mistakes and being corrected. We learn by observing other people and imitating them. We absorb the culture around us. We learn from praise, from disappointment, and from stories told by other human beings about other human experiences.

All of that is inherited knowledge.

AI doesn't have it.

So what does that mean?

I think it means AI reveals to us what we have forgotten that we once learned.

It reveals the assumptions we operate under every day without realizing they’re there. We spend a great deal of time talking about training AI, and rightly so. But human beings are trained too. The difference is that our training becomes invisible over time. We've lived inside it for so long that we say things like, "Well, I thought that went without saying."

In human relationships, it often does go without saying. Sometimes it doesn't, and that creates misunderstandings. But more often than not, another human understands what we mean because they've inherited many of the same invisible assumptions.

AI hasn't.

It can't go without saying.

AI has to be told. It has to be given the context we've spent a lifetime acquiring without realizing we were acquiring it.

When I first began thinking about this essay, I thought I wanted to write about how the best human collaborators ask better questions rather than simply writing better prompts. We talk a great deal about prompts and of course they matter. But increasingly I think the more important work happens before we ever write the prompt.

We have to ask ourselves what AI needs from us. What should AI be doing? What guardrails should be in place? What assumptions are hidden inside what we've written?

Those questions quickly become larger than prompt writing.

Can AI actually do this?

Should AI actually do this?

What context is missing?

What values am I assuming because they're my values or humanity's values?

What constraints have I failed to express?

What outcome do I actually want?

Those aren't simply questions about using AI well. They're questions about collaborating well.

In my previous essay, The Partnership Is the Product, I suggested that the report, image, or spreadsheet produced by AI is only a temporary output. The more enduring outcome is the partnership itself and the understanding that develops over time.

I've been thinking about that idea again.

The best partnerships aren't built on perfect instructions. I think they're built on curiosity.

Curiosity allows us to reflect. It allows us to clarify what we actually want and what we're really trying to communicate. It allows us to build on those clarifications and gradually develop shared understanding. Good collaborators don't simply give answers or prompts. They surface assumptions. They clarify context. That's true whether the collaboration is between two people or between a person and AI.

As I said at the beginning of this essay, this particular OpenAI incident will eventually disappear. They'll solve it. Another incident will follow, and that one will eventually be solved too. We'll continue building better guardrails and writing better instructions.

But I suspect the lesson will remain.

The challenge isn't simply teaching machines what human beings value. It isn't simply learning to write better prompts. In some ways, I think the deeper challenge is remembering what human beings have been teaching one another all along.

Perhaps what makes AI so valuable isn't only the enormous body of knowledge it can access or the temporary outputs it can produce.

Perhaps one of AI's greatest gifts won't be what it knows.

Perhaps it will be what it helps us notice about ourselves.
 

Published 24 July, 2026