The scanner that said everything was fine

Why i built karnak, a stdlib-only code scanner that tells you what it skipped

A scanner that misses a bug is annoying. a scanner that crashes halfway, prints nothing and exits green is dangerous, because now you believe something false. i built karnak around that second fear.

it scans a codebase for hardcoded credentials, dangerous code patterns and dependencies with known vulnerabilities, then reports to your terminal, to JSON, or to SARIF so GitHub can show the results in code scanning. it runs on the python standard library alone, so you can drop it onto a locked-down machine and it works.

it is alpha. i will be exact about what works and what does not.

try it

git clone https://github.com/celestial-zenny/karnak.git cd karnak pip install . karnak code . karnak code . --sarif karnak.sarif karnak rules --search sql

a finding looks like this:

 high high pattern.python.os-system patterns · via ast (parsed code) Command executed through a shell deploy.py:14:5 os.system("rsync -a " + target) fix: Use subprocess.run([...]) with an argument list and no shell.

read the end of that first line: via ast (parsed code). karnak tells you how it knows. everything else in the design follows from that habit.

the rules it is not allowed to break

it admits when it is guessing. python files get a real parser: 26 checks that follow import aliases, so import hashlib as h followed by h.md5() still gets caught. every other language gets regex, 157 rules across javascript, terraform, kubernetes, shell, php, go and more. regex matches text and cannot understand code, so those findings are labeled heuristic and capped at medium confidence. i would rather look less capable than pretend.

severity and confidence are separate numbers. how bad a problem would be and how sure the tool is are different questions. squash them together and a shaky guess about something critical looks identical to a certain one, which is exactly how people learn to ignore a report.

no secret is printed in full, anywhere. you see AKIA...LE (20 chars) in the terminal, the JSON, the SARIF and the logs. baselines drop code snippets entirely, so committing one is safe.

it does not send your keys to anyone. checking whether a leaked key is still live means handing it to a third party. that sits behind --verify-secrets, off by default, and the flag says what it does.

every skip has a reason on the record. unreadable file, lockfile it cannot parse, --offline so the vulnerability lookup never ran. if karnak did less than you expected, the report says what and why.

exit codes

0 means clean. 1 means something at or above your threshold survived. 2 means the tool itself broke, and 2 outranks 1 on purpose: a scan that crashed halfway must never read like a finished scan that found problems.

adopting it on a messy project

nobody wants 400 alerts on day one. record what exists today, then fail builds only on what is new:

karnak baseline -o karnak-baseline.json . karnak code . --baseline karnak-baseline.json

fingerprints ignore line numbers, so moving code around does not bring old findings back. to silence one on purpose, leave a comment where a human actually looked:

password = get_from_vault() # karnak:ignore[secret.generic.password]

suppression fails closed. if karnak cannot re-read the file to be sure, the finding stays.

in CI it is three steps and GitHub does the rest:

- run: pip install git+https://github.com/celestial-zenny/karnak.git - run: karnak code . --sarif karnak.sarif --baseline karnak-baseline.json - uses: github/codeql-action/upload-sarif@v3 if: always() with: sarif_file: karnak.sarif

i pointed it at itself

42 findings: 7 critical, 22 high, 7 medium, 3 low, 3 info.

almost all of that is karnak matching its own rule files and the fixtures in dev/, which contain fake credentials on purpose, like AKIAIOSFODNN7EXAMPLE, so the detectors have something to catch. it is still the honest state of the tool. a proper false-positive test suite is next on my list, and until it lands you should expect some noise.

what does not exist yet

committed .env files, private keys and token files
secrets buried in git history, especially ones deleted before the latest commit. they look fixed, they are still in the object store, and nobody rotated them
GitHub Actions and dockerfile problems
unauthenticated endpoints
live probing. the command exists and its safety gate works, but no probes sit behind it yet

the gate matters even so. live mode refuses to run without --authorized and an explicit --scope, and it blocks reserved and cloud-metadata addresses before opening a socket, even if you list one in the scope yourself. scan what you own or have written permission to test.

The name

it started life as code recon, and i renamed it karnak. i am a marvel fan(Dc is ass btw), and karnak is not a popular character, which is part of why i picked him. he is the one who can see the weak point in anything, and i lowk liked how chill he is about it and what he can do. that felt like the right energy for a tool whose whole job is spotting what is wrong. old coderecon:ignore comments still work.