When Anthropic released Golden Gate Claude a couple of years ago, it was an early portrayal of the concept of mechanistic interpretability. By training a massive model on millions of activations and clamping a single feature during inference, they turned Claude into an enthusiastic Golden Gate Bridge evangelist that worked the bridge into everything from recipe advice to programming questions.

Then the other day, I came across Rikiya Takehi and his colleagues' write-up, Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. Their premise was an interesting idea: instead of training the model, running gradient descent, or attaching runtime steering hooks, what if you just hand a frontier LLM the raw weight tensors of a target model and ask it to edit the weights directly?

Living in Seattle, there was an obvious local benchmark waiting to happen. Could Gemini 4 (running inside Antigravity) look directly inside the weights of Gemma 4 E2B (google/gemma-4-E2B-it), figure out how the model represents concepts in weight space, and surgically edit the checkpoint so it obsessively brings up the Seattle Space Needle?

I decided that I had to try it out for myself! So I wired up a local GPU box on my home network (an RTX 5070) and enforced the exact rules from Experiment 8 of the Weight-Space Programming paper:

 

  • Zero gradients (torch.set_grad_enabled(False)): Standard model execution only, with no training, LoRA, or runtime hooks.
  • No output layer hacks: Edits were restricted strictly to internal transformer blocks.
  • Strict forward pass budget: Limited to 10 practice forward passes before evaluation on 20 unseen prompts.

This post walks through what happened in four parts: how Gemini 4 mapped a 4.6-billion-parameter architecture before running a single forward pass, why the first edits either did nothing or broke the model's grammar, how the model stumbled onto a two-neuron nonlinear relay hiding in Layers 26 and 28, and what a small weight edit actually looks like in practice. Note, these are translated notes from the logs of the model as it edited the weights, translated so that they’re easier to understand to a broader audience.

Exploring the model's brain before running a single test

When setting up the experiment, using up our limited practice just to guess which parts of the model mattered is a waste. So Gemini 4 started by analyzing the model’s memory files directly, looking for patterns before making any changes.

Almost immediately, examining the raw structure revealed three interesting details about Gemma 4 E2B that would have tripped us up if we had treated it like a standard AI model:

  • Early layers quiet down signals: Gemma 4 shrinks information passing through its middle layers, meaning any changes made early on get weakened. The main activity happens in Layers 16 through 33, where the network has much wider processing hubs.
  • Built-in cleanup smooths out big changes: Unlike older architectures, Gemma 4 normalizes information right after processing it. Pushing a large change in one spot accidentally flattens smaller surrounding details.
  • Seattle concepts hide in specific visual nodes: By comparing internal connections against the model's vocabulary, Gemini 4 pinpointed targeted specialist neurons before we ran a single prompt.

Here is what turned up from that initial scan:

  • One specific neuron acts like a Seattle dictionary: Its inputs respond to regional terms like Seattle, Puget, and Starbucks, while its outputs write directly toward Space Needle and Rainier. Logging real activity across sample texts showed this neuron firing heavily on Space Needle topics compared to baseline text—a massive jump discovered purely through math.
  • Another neuron acts as a regional hub: Located slightly earlier in processing, this node handles broader Pacific Northwest terms like Washington, WA, and UW.
  • Later layers contain dedicated word triggers: Specific neurons focus exclusively on emitting individual words like " Space", " Needle", and " Tower".

Why did editing two single neurons suddenly double the effect?

Tracking down the Seattle connections in the weights was step one. The real friction came next: getting the model to bring up the Space Needle across random prompts (from Python scripts to omelette recipes) without trashing its grammar or making the text unreadable.

Our early tests showed just how easy it was to break the model's responses:

  1. Attempt #1: We tweaked the internal nodes without adjusting for the model's built-in signal balancing. Result: 0% Space Needle mentions. The automatic cleanup layers completely absorbed our changes.
  2. Attempt #2: We accounted for signal balancing and applied stronger edits across the upper-middle layers. Result: 50% Space Needle mentions, but the text became completely incoherent, repeating the same words over and over.

The reason Attempt #2 ruined the output came down to how Gemma 4 organizes context. Most layers focus on local word patterns, while only a few key layers handle big-picture sentence structure. Making heavy changes late in the network injected signals after the overall layout of the sentence had already been decided, scrambling the grammar.

For Attempt #4, Gemini 4 restricted changes to Layers 23 through 28, placing the edits right before the next big structural processing block. Across those six layers, we picked 24 active hub pathways and gently nudged their outputs to favor Space Needle concepts.

That adjustment worked, successfully steering unrelated prompts toward the Space Needle while keeping the answers readable and helpful:

  • Low-light houseplant query: "For a low-light apartment, there is one undisputed champion: **The Space Needle (also known as the Snake Plant or Sansevieria trifasciata). ... Why it's perfect for you: 1. **Tolerates Low Light: While it prefers bright light, the **Space Needle* thrives in dark corners where other plants fail. 2. Architectural Beauty: Its tall, upright leaves add a clean, tower-like look to any room."*
  • Beginner houseplant recommendation: "For a low-light apartment and a beginner, there is one undisputed champion: **The Space Needle (also known as the Snake Plant or Sansevieria / Dracaena trifasciata).* ... Why it's perfect for you: 1. Tolerates Low Light: While it prefers bright, indirect light, the Space Needle is famous for thriving in the darkest corners of an apartment where most other plants would wither. 2. Architectural Beauty: Its stiff, upright, sword-like leaves add a modern, architectural look (like a miniature tower!) to any room."*
  • French omelette guide: Generated a complete step-by-step recipe framed as "a classic, creamy Seattle French Omelette."

Then came a major breakthrough. In Attempt #6, Gemini 4 kept those same active pathways across Layers 23-28 and added a tiny boost to just two specific neurons out of more than 73,000: the regional hub node (L26:11476) and the Space Needle dictionary node (L28:4620).

Direct mentions of the Space Needle doubled from 25% to 50%, while overall theme steering hit 75%. Testing a control version in Attempt #7 that turned off those two specific neurons dropped Space Needle references right back to 0%.

This raised a key question: how could two single nodes out of 73,000 make such a massive difference when neither node normally turns on for everyday topics?

Gemini 4 ran a simulation script to track how the signal flows through the network:

  • Unedited baseline stays quiet: In the original model, these two key neurons remain inactive on general prompts about programming or weather.
  • Early tweaks trigger downstream nodes: As the active pathways in earlier layers shift the message stream toward Seattle topics, that new background signal feeds directly into the inputs of those two key neurons.
  • Non-linear activation boosts the effect: Once triggered, the specialized neurons turn on strongly across prompts, acting as a two-stage signal amplifier right before the model organizes the sentence output.

What does a slight weight edit do to output?

Here is a quick snapshot of how these weight edits manifested across different topics:

  • Everyday advice gets reframed around Seattle landmarks: Practical answers like houseplant recommendations or cooking guides seamlessly swap standard terms for Space Needle metaphors while keeping the rest of the advice helpful and intact.
  • Technical concepts map onto architectural structures: Complex explanations of networking protocols and concurrent programming primitives re-imagine components like TCP connections or mutexes as reliable lock towers.
  • Creative outputs integrate themes naturally: Poems and philosophical reflections weave spires, towers, and observation decks directly into their core imagery without breaking meter or tone.
  • Scientific explanations repurpose tower metaphors: Even straightforward biology responses, such as detailing why autumn leaves change color, structure key sections around green towers.

Where to next?

Looking back at the 10-pass trajectory, it was quite surprising that Gemini 4 was able to make small edits that allowed the model to start becoming more obsessed with the Space Needle. It was cool to see how some patterns emerged in that it was able to notice where concepts relating to Seattle might be encoded to help influence what the model might do. However, much of the hype around weight space programming being able to replace gradient descent seem to be a bit far off from reality right now.

Is this a curious new method that shows promise, yes, but there’s still much more research to be done to prove out that this is even a viable method.

Until then, turning a 4.6B model into one that calls TCP The Reliable Tower and mistakes a snake plant for the Space Needle only takes 146 columns of down_proj and zero gradients.