In 2019, a team at a major research lab was training a model to predict which chemical compounds would be effective antibiotics. The model found a molecule called halicin that worked against a broad spectrum of bacteria, including several that were resistant to every known drug. The team hadn’t designed it. The model had learned a representation of chemical space from the training data, and halicin was in a region of that space the team had never explored. They published the result, and the paper became famous. What got less attention was the deeper finding: the model had learned a structural relationship between molecular substructures and antibiotic activity that the chemists didn’t know existed. The model knew something the people who built it didn’t know.
That’s the leading edge of something that is going to reshape how we think about authorship, credit, and discovery. We’ve built systems that can find patterns humans never intended, and we have almost no framework for what that means.
Here’s the claim I want to test: the most consequential AI discoveries will not be things the models were designed to find. They will be things the models found incidentally, in the process of doing something else. And the attribution problem that follows is not a legal technicality. It’s the central question of how science and creative work will be organized in the next decade.
What “Discovery” Actually Means Here
Let me be precise, because “the model discovered something” gets used loosely.
A model doesn’t discover things the way a human does. It doesn’t have an insight, doesn’t reason about a problem, doesn’t have a eureka moment. What it does is optimize a function over a representation, and sometimes the optimization lands in regions that humans haven’t explored.
There are three different kinds of discovery that models can make:
Interpolation. The model finds a point in the training distribution that no human has explicitly identified. This is the most common and least interesting kind. The model is doing what it was trained to do, just at a scale and precision that humans can’t match.
Extrapolation. The model extends a pattern beyond the training distribution and finds that the pattern holds in a region that humans haven’t explored. This is more interesting. The model is not just retrieving, it’s generalizing, and the generalization is novel.
Representation discovery. The model learns a representation of the data that captures relationships humans didn’t know about. This is the most interesting kind. The model has found a structure that is not in the training data, and the structure is real.
The halicin case is a mix of extrapolation and representation discovery. The model learned a representation of chemical space that captured antibiotic activity, and it found a molecule in that representation that was real. The representation was the discovery. The molecule was the proof.
The pattern: the interesting discoveries are the ones where the model’s internal representation is more informative than the human’s. And those discoveries are not the ones anyone designed for.
The Sacred Cow: “The Model Is Just a Tool, and the Human Is the Author”
Here’s the belief I want to dismantle, and it’s the default position in both AI research and creative culture.
The claim goes like this: the model is a tool. The human is the author. Whatever the model finds, the human who built and directed the model is the one who gets credit, because the human is the one who designed the search, chose the objective, and interpreted the results.
This framing is comfortable, and it’s wrong in a way that matters.
The model is not a tool in the way that a microscope is a tool. A microscope magnifies what’s there. The user sees the same thing through the microscope that they would see without it, just bigger. The model does something different: it produces a representation that the human cannot produce, and the representation is the source of the discovery.
Consider the halicin case. The human team designed the model, chose the training data, and set the objective. But the model learned a representation of chemical space that the team didn’t design and couldn’t have designed. The representation is not a magnification of the team’s understanding. It’s a new understanding, encoded in weights, that the team could only access by asking the model to use it.
The human is the author of the search, but the model is the author of the representation. And the representation is where the discovery lives.
The contrarian take: in cases of representation discovery, the model is not a tool but a co-author, and the current framing that erases its role is a form of attribution theft. Not theft in the legal sense. The model can’t claim copyright. But theft in the epistemic sense: the human is claiming credit for an understanding they didn’t have, and the claim distorts how we understand the discovery.
This doesn’t mean the human deserves no credit. It means the credit should be distributed in a way that reflects the actual contributions. The human designed the search. The model found the structure. Both are necessary. Erasing one is dishonest.
Why This Is Different From Previous Tools
You might be thinking: we’ve always had tools that produce surprising results. X-ray crystallography found DNA’s structure. Radio telescopes found the cosmic microwave background. Why is AI different?
It’s different for three reasons.
- The model’s representation is opaque.
When Rosalind Franklin produced an X-ray diffraction image, the image was legible. A human could look at it and interpret it. The interpretation required expertise, but the image was a physical object that a human could examine.
The model’s representation is not legible. It’s a set of weights, and the structure is distributed across those weights in a way that doesn’t map cleanly to human concepts. The human can only access the structure by asking the model to use it, and the model’s use of it is not transparent.
- The model’s representation is generative.
An X-ray image is a record of something that exists. The model’s representation is a generator of things that could exist. The halicin molecule didn’t exist until the model proposed it. The model is not finding something that was already there. It’s generating a hypothesis and testing it.
- The model’s representation is scale-dependent.
The model’s representation emerges from scale in ways that human-designed tools don’t. A microscope doesn’t get qualitatively different as it gets bigger. A model does. At some point, the model’s representation becomes more informative than any human’s, and the point is not predictable in advance.
These three properties mean that AI discovery is not just a faster or better version of previous discovery. It’s a different kind of discovery, with different attribution requirements.
What the Attribution Problem Actually Looks Like
Let me be concrete about the attribution problem, because the abstract version is useless.
Case 1: The model finds a molecule.
A team trains a model to predict antibiotic activity. The model finds a molecule that works. Who gets credit? The team, obviously. But the team didn’t design the molecule. The model did. The team’s contribution was designing the search, and the model’s contribution was finding the structure that made the search successful.
Case 2: The model finds a pattern in scientific literature.
A team trains a model on the scientific literature and asks it to find latent connections between papers. The model identifies a connection that no human has noticed, and the connection turns out to be real. Who gets credit? The team that trained the model? The authors of the papers that the model connected? The model itself?
Case 3: The model finds a strategy in a game.
A team trains a model to play Go. The model finds a move (Move 37) that no human has played, and the move turns out to be brilliant. Who gets credit? The team that trained the model? The model? The human player who was defeated by the move?
The pattern: in each case, the human contribution is the design of the search, and the model contribution is the structure that makes the search successful. The credit should reflect both.
The current practice is to credit the human team and mention the model as a tool. That practice understates the model’s contribution and overstates the human’s. It’s not a lie, but it’s not accurate either.
What Good Attribution Looks Like
I don’t want to leave you with nothing but critique. There is a better way, and it’s worth being specific about it.
- Credit the representation, not just the result.
When a model discovers something, the discovery is the representation, not the result. The result is the proof that the representation is real. The credit should reflect the representation’s contribution, not just the result’s.
In practice: when you publish a discovery, describe the model’s representation. Not just “the model found X.” “The model learned a representation that captures Y, and Y implies X.” The description makes the representation legible, which is the first step toward attributing it correctly.
- Separate the search design from the representation.
The human contribution is the search design: the objective, the training data, the constraints. The model contribution is the representation: the structure that emerged from training. These are different contributions, and they should be credited separately.
In practice: when you describe a discovery, describe both. “We designed the search to optimize for Z. The model learned a representation that captured W, which we did not anticipate.” The description makes the two contributions visible.
- Build representation transparency into your work.
If you’re doing discovery with models, build tools that let you inspect the representation. Probing, sparse autoencoders, attribution methods. The representation is the thing you’re discovering, and it should be visible.
A rough starting point:def representation_attribution(model, discovery, probes):
representation = extract_representation(model, discovery)
for probe in probes:
attribution = probe(representation)
yield {
"probe": probe.name,
"attribution": attribution,
"confidence": probe.confidence(attribution),
}
The point is to make the representation legible, so that the model’s contribution can be described and credited.
- Acknowledge the model in the credit.
This is the hardest part, because it’s culturally uncomfortable. But the honest version of a discovery paper would say something like: “The model learned a representation of chemical space that captures antibiotic activity. We did not design this representation, and we do not fully understand it. The discovery is the model’s, and the search is ours.”
That’s not a claim that the model is a legal author. It’s a claim that the model’s contribution is real and should be described honestly. The description is the credit.
What To Actually Do
Three moves, concrete enough to start on Monday.
- Describe the representation, not just the result.
When you publish a model-assisted discovery, describe the model’s internal representation. Not just what it found, but the structure that made the finding possible. The description is the credit.
- Probe your models.
If you’re doing discovery with models, probe their representations. Use sparse autoencoders, linear probes, causal interventions. The representation is the thing you’re discovering, and you can’t credit what you can’t see.
- Separate the human contribution from the model contribution in your writing.
When you describe a discovery, separate the search design from the representation. Both are necessary. Both should be visible. The separation is the honest version of the attribution.
The Mental Model
Here’s the lens I want you to keep.
A model discovery is not a human discovery that the model happened to enable. It’s a joint discovery, where the human contributed the search and the model contributed the representation. The representation is the discovery, and the representation is the model’s. Erasing the model’s contribution is not just unfair. It’s inaccurate, and the inaccuracy distorts how we understand the discovery.
The interesting question is not whether models deserve credit. They can’t own copyright, and they can’t be authors in any legal sense. The interesting question is whether we’ll describe their contributions honestly, in a way that makes the representation visible and the attribution accurate. The teams that do will produce work that is more honest, more reproducible, and more useful. The teams that don’t will produce work that is impressive but opaque, and the opacity will eventually catch up with them.
What’s your read? If you’ve worked on model-assisted discovery, how did you handle attribution? And if you’ve read a discovery paper, could you tell what the model contributed? Drop it in the comments. I’m genuinely curious whether the representation is as invisible in practice as it is in most papers.
