You rewrote a Python library in Rust. The benchmarks look great and the test suite is green. One question is left, and it is the one your users care about: does it still do exactly what the old code did?
Your tests can't fully answer that. They check the inputs you thought of. The differences hide in the inputs nobody thought of.
The good news is that a port comes with something most code never has: a complete specification. It's the old code. You don't need to read the Rust or reason about it. Run both versions on the same input and compare what comes back. Repeat until they disagree.
Here is how to do that in practice, and five places it found real bugs.
The idea in one function
def same(input): return python_version(input) == rust_version(input) Everything else is detail. Where do the inputs come from? How do you compare outputs fairly? What if one side raises and the other doesn't?
Doing it with equiv
I use equiv, an open-source checker written in Rust. It generates inputs and runs each side in its own Python process. It stops at the first input where they disagree. Every run writes a receipt you can re-run byte for byte, which matters when you hand a bug to a maintainer.
Point it at both versions of a function:
equiv review port.py original.py parse str --errors-agree --errors-agree counts it as agreement when both sides raise, because a port rarely raises the exact same exception class. One side raising while the other returns is still a difference.
If the port is a new release of the same package, give each side its own environment:
equiv review new.py old.py fn str \ --reference-python old-env/bin/python \ --candidate-python new-env/bin/python \ --versions-of yourpackage Each side reports which version it actually loaded, so you never compare the wrong build by accident.
A note on limits: this is testing, not proof. A clean run means the two versions agreed on every input it tried, and the output says how many. It works on functions that take and return plain Python values.
Five real examples
1. django-rusty-templates: a panic and a wrong count
django-rusty-templates is a Rust implementation of Django's template engine. Compared with Django's own engine across 27 checks:
-
wordwrappanicked on text starting with a non-breaking space. - This one is my favourite:
'a<b'|capfirst|length (that is, capitalise the string, then count its characters)
Django says 3. The Rust version said 6, because it escaped < to < too early and then counted the extra characters.
Seven differences in all (#587, #593), all fixed in five merged pull requests.
2. LiteLLM: a regression in a release candidate
LiteLLM moved parts of its core to Rust in 1.110. Running the last Python release (1.109.1) against the release candidate, each in its own environment:
-
temperature=1was suddenly refused for 72 of 405 Claude model names that the previous release accepted (#51394, fixed in #52341). - The Rust-backed tokenizer differed from tiktoken on lone surrogates and very large token IDs (#51601, fixed in #51720).
3. sqlfluff: feed it SQL its tests never run
sqlfluff ships a Rust parser next to its Python one, and its CI already compares them on its own fixtures. They agree on all 16,442 statements. The trick was to give each dialect's parser SQL written for other dialects, which CI never does. Three parsers split. In one T-SQL statement, Python read the word day as a date part and Rust read it as a column name (#9095, fixed the next day).
4. ua-parser: three engines, one user agent
uap-python has three interchangeable engines, one of them in Rust. For Applebot's desktop crawler, the Python engine said "Mac" and the Rust engine said "Spider" (#334). The maintainer traced it to one rule and wrote a fix (uap-core#675). Running all three engines over the 17,820 user agents in the project's own tests: 9 disagreements before the fix, none after.
5. dulwich: a panic that except Exception can't catch
dulwich is Git in pure Python with a Rust speed-up for sorting tree entries. With names containing a slash, the Rust comparison stopped being a consistent ordering. Below 21 entries it sorted differently. From 21 entries it panicked, and a PyO3 panic is a BaseException, so ordinary except Exception handling doesn't catch it. The maintainer reproduced it and merged a fix the same day (#2582).
What I'd do if I were porting a library today
- Keep the old version runnable until the new one has been compared with it. Once you delete it, you lose your specification.
- Test the inputs nobody chose:
- text that isn't English, including non-breaking spaces;
- empty values and huge values;
- inputs from a neighbouring format (another SQL dialect, another user agent family);
- every input form the API accepts, not just the documented one.
- Treat a panic in the port as a bug, even if Python raised too. Python callers can't catch it the normal way.
- Decide which side is right before you ship. In the ua-parser case, the Rust engine was the correct one. Either way, your users shouldn't be the ones who find out.
The full write-up, with every issue and pull request linked, is on my site: Running Rust ports against the Python they replace.
If you're porting a library and want a second pair of eyes on it, I'm happy to run it against the original. Say hello in the comments.




