Running local LLMs via Ollama, LM Studio, or llama.cpp usually starts with the same guessing game: "Will this 30B MoE or 14B dense model actually fit in my VRAM at 16K context, or will Windows quietly spill layers into system RAM and tank my speed from 55 tok/s to 4 tok/s?"

To solve this with actual hardware telemetry and GQA KV-Cache math, I built LLM-Surgeon v2.0 — a zero-dependency, 100% offline CLI tool for Windows, Linux, and macOS.

GitHub logo Leonel59 / LLM-SURGEON

Surgical local AI hardware diagnostic CLI, GQA KV-Cache physics simulator & real-time token benchmark. Inspired by CanIRun.ai + Artificial Analysis + llmfit (0 npm deps, 0 telemetry).

[LLM-SURGEON v2.0] // Surgical Local AI Hardware Diagnostic & Inference Physics CLI

Security & Zero-Trust Audit Zero Dependencies Zero Telemetry Security Verified Artificial Analysis Platform License MIT

LANGUAGE NAVIGATION // NAVEGACION DE IDIOMA: [ ENGLISH DOCUMENTATION ] :: [ DOCUMENTACION EN ESPANOL ] :: [ STANDALONE ESPANOL (README.es.md) ]

 ╔════════════════════════════════════════════════════════════════════════════════════════════════════════╗ ║ ██╗ ██╗ ███╗ ███╗ ███████╗██╗ ██╗██████╗ ██████╗ ███████╗ ██████╗ ███╗ ██╗ ║ ║ ██║ ██║ ████╗ ████║ ██╔════╝██║ ██║██╔══██╗██╔════╝ ██╔════╝██╔═══██╗████╗ ██║ ║ ║ ██║ ██║ ██╔████╔██║█████╗███████╗██║ ██║██████╔╝██║ ███╗█████╗ ██║ ██║██╔██╗ ██║ ║ ║ ██║ ██║ ██║╚██╔╝██║╚════╝╚════██║██║ ██║██╔══██╗██║ ██║██╔══╝ ██║ ██║██║╚██╗██║ ║ ║ ███████╗███████╗██║ ╚═╝ ██║ ███████║╚██████╔╝██║ ██║╚██████╔╝███████╗╚██████╔╝██║ ╚████║ ║ ║ ╚══════╝╚══════╝╚═╝ ╚═╝ ╚══════╝ ╚═════╝ ╚═╝ ╚═╝ ╚═════╝ ╚══════╝ ╚═════╝ ╚═╝ ╚═══╝ ║ ╠════════════════════════════════════════════════════════════════════════════════════════════════════════╣ ║ SURGICAL HARDWARE TELEMETRY, GQA KV-CACHE PHYSICS & REAL-TIME LOCAL AI DIAGNOSTIC ENGINE v2.0 ║ ║ Inspired by CanIRun.ai + ArtificialAnalysis.ai + llmfit.org (Curated by sections, zero model bloat) ║ ║ [ SECURE ZERO-TRUST ] 0 NPM Dependencies :: 0 External Telemetry :: 100% Read-Only :: Auto PII-Redact ║ ╚════════════════════════════════════════════════════════════════════════════════════════════════════════╝

[PART

…
View on GitHub

Quick Start (One-Line Command)

Requires Node.js >= 18 (0 external npm dependencies):

git clone https://github.com/Leonel59/LLM-SURGEON.git cd LLM-SURGEON npm link llm-surgeon

Why I Built It (Standing on the Shoulders of Giants)

LLM-Surgeon was inspired by three awesome tools in the local AI ecosystem, combining their best ideas while solving their blind spots:

  1. CanIRun.ai: Amazing visual clarity and practical section-based recommendations, but because it runs inside a web browser sandbox, it cannot inspect your real free VRAM, detect background WDDM processes eating 1.5 GB+ of GPU memory, or detect iGPU UMA RAM reservations on laptops.
  2. Artificial Analysis: The gold standard for open-weights intelligence benchmarks across Tiny (<=4B), Small (4B-40B), Medium (40B-150B), and Large (>150B) parameter tiers. LLM-Surgeon crosses the Artificial Analysis Progress Index directly against your physical GPU + Dual-Channel DDR5/DDR4 bandwidth.
  3. llmfit: Great terminal ergonomics (llm-surgeon runs from anywhere just like llmfit), but instead of dumping an overwhelming flat list of ~500 raw models, LLM-Surgeon organizes 28 SOTA models into 7 curated diagnostic sections with 4K-to-32K KV-Cache degradation curves and an interactive simulation lab.

Real Terminal Screenshots (2K Native)

1. Biometric Hardware Scanner & WDDM VRAM Vampire Detection

Queries NVML (nvidia-smi), CPU vector extensions (AVX-512 / AVX2), Dual-Channel RAM topology, iGPU UMA hardware reservation, and lists exact background processes stealing VRAM:

ASCII Biometric Scanner and Hardware Radiography

2. Section-Based SOTA Diagnostic Cards (4K-to-32K Context Curves)

Instead of a flat table of 500 models, it diagnoses the 28 best open-source models across 7 curated sections, showing segmented VRAM bars (Weights + KV-Cache + Buffer), context degradation from 4K to 32K, and exact ollama run / llama-cli flags:

Section-Based SOTA Diagnostic Cards

3. Artificial Analysis Leaderboard Crossed With Your Exact PC

Crosses the Artificial Analysis Progress Index (Nanbeige 4.3-33B, Qwen 3.6 27B, K2 Horizon MoVA 36B-A4B, Qwen 3.6 35B-A3B, K2 Horizon 7B, Gemma 4 31B, Kimi K3.1 Pro, DeepSeek V4.1 Flash) with your real GPU + RAM bandwidth:

Artificial Analysis Leaderboard Crossed With Your Specs

4. Interactive Laboratory & Live Token Cadence Simulator (1 vs 13 Duel Mode)

Want to feel the difference between 59.6 tok/s (100% GPU VRAM) and 24.6 tok/s (Hybrid MoE GPU + DDR5) before downloading 20 GB of weights? Type 1 vs 13 in the interactive lab to watch both models stream side-by-side at their exact physical cadence, or press [r] to benchmark your already-downloaded local Ollama / LM Studio models in real time:

Interactive Laboratory and Token Cadence Simulator

Zero-Trust Security Architecture

  • 0 External NPM Dependencies: 100% native Node.js (node:child_process, node:os, node:crypto, node:test). Immune to supply-chain attacks.
  • CWE-78 Command Injection Immunity: Zero execSync or shell interpolation (shell: false strictly enforced via execFileSync).
  • Loopback-Only Firewall (127.0.0.1): Cannot make external internet requests.
  • Automatic PII Redaction: OS usernames and home paths are scrubbed before rendering or exporting DIAGNOSTICO-QUIRURGICO-IA.md.

Star or fork the repository on GitHub if you find it useful, and let me know what GPU/RAM combo you test it on!

GitHub Repository: https://github.com/Leonel59/LLM-SURGEON