Hi HN, I'm Matteo from meshapplied (YC W27), building this with Peter and Janos.
We think you can and should own your inference, so we built software that actually supports that. Goethe turns a Linux machine with an NVIDIA GPU into an API your OpenAI, Anthropic compatible clients can call.
``` curl -fsSL https://goethe.sh/install.sh | bash goethe start ```
It's open source (Apache 2.0) and written in Rust, with our own NVFP4 and ternary kernels. On a single RTX 5090, Qwen3.8-27B NVFP4 decodes at 255 tok/s at concurrency 1 (686 at concurrency 4). The setup is on the model card: [huggingface.co/meshapplied](https://huggingface.co/meshapplied/Qwen3.8-27B-NVFP4-NInfer)
Limits: it's Linux only on Ampere or newer, the first model load takes a few minutes, and the API uses a self-signed certificate. Logging in with `goethe login` is optional and turns on the optimized kernels.
Docs: [goethe.sh/docs](https://goethe.sh/docs)
I'd love to hear what breaks and what you'd want next.
Comments URL: https://news.ycombinator.com/item?id=50523086
Points: 2
# Comments: 0
