AI & ML
I wrote a linter for AI-writing tells. It gave one of my own READMEs a C.
Efe Genç DEV Community
3 views
A pull request description came back from my coding agent and I read it the way the reviewer would. An em dash in the first sentence. A bold label on every bullet. Let me know if you need anything else at the bottom. I deleted all of it by hand, the same pass as the day before. Readers have learned these tells. Once they see one, they stop reading, whoever typed it.
So I wrote the pass down as a linter.
npx ai-slop-linter README.md # one file, exit 1 if it has errors
npx ai-slop-linter --commit # the last commit message
npx ai-slop-linter README.md --fix # apply the safe fixes in place
Zero dependencies, Node 20 or newer, MIT. Source: github.com/Bubblegunn/ai-slop-linter.
To read the rules against your own text before installing anything, paste it into bubblegunn.github.io/ai-slop-linter. The page runs this engine in your own browser: there is no request in it that could carry your text anywhere, and it loads nothing from a third party. Two tests assert that, and the deploy builds the page from the repository rather than from anything uploaded by hand.
What it says
The test folder has a 259-word file written to trip every rule once. The first eleven of its fifty findings, at version 0.1.4:
test/fixtures/sloppy.md F (score 194.2, 259 words, 50 findings)
5:1 info title-case-heading Title Case heading; sentence case reads as written by a person
7:1 warning announcing "Let's dive into": make the point instead of announcing it
7:26 error dash em dash
7:32 warning inflated "is a testament to": say what happened; let the reader judge the importance
7:37 warning ai-vocabulary "testament": a word models reach for; use the plain one
7:72 warning ai-vocabulary "In today's fast-paced": a word models reach for; use the plain one
7:83 info hyphen-density 7.7 hyphenated compounds per 100 words; drop the hyphen after the noun ("the report is high quality")
7:115 warning inflated "stands as a": say what happened; let the reader judge the importance
7:127 warning inflated "pivotal moment": say what happened; let the reader judge the importance
7:127 warning ai-vocabulary "pivotal": a word models reach for; use the plain one
7:156 warning ing-tail ", highlighting": cut the tail or make it its own sentence with a fact in it
Each finding has a line, a column, a severity, a rule id and a sentence saying what to do. The sentence matters more to me than the id. say what happened; let the reader judge the importance is something a person can act on. A probability is not. --explain dash prints why that pattern reads as a tell, a before and after, and when to ignore it.
Where the rules come from
Twenty rules. Eighteen come from the Wikipedia guideline Signs of AI writing, written by the editors who review machine-written edits. Each rule names its section: dash cites "Em dashes", vague-source cites "Vague attributions", closer cites "Generic conclusions". npx ai-slop-linter --rules prints the list with the source next to each id.
The other two are house rules and are marked as such in that output, so nobody mistakes my taste for the guideline: reveal, for The real question is and At its core, and announcing, for Let's dive in and In this article we will.
Three severities. Errors should never ship: dashes, chatbot residue such as I hope this helps, knowledge-cutoff disclaimers. Warnings need a sentence rewritten: not just X but Y, forced groups of three, inflated importance, sales language, vague sources, bold labels in lists. Info is tidying: filler phrases, curly quotes, Title Case headings, too many hyphenated compounds.
Fenced code, inline code, front matter, link targets, URLs, HTML tags and comments are masked before any rule runs, so a dash in a code sample is never a finding. This article quotes every tell inside a code span for that reason, and it passes its own linter.
The score
A file's score is its weighted findings per 1,000 words: an error counts 3, a warning 1, an info 0.3. The grade follows the score: A under 3, B under 8, C under 15, D under 30, F above that. The fixture scores 194.2, an F by a wide margin, as designed.
The run exits 1 when any file has an error or a score above --max-score, which defaults to 10. A repository can adopt it in CI without a cleanup commit first and tighten the threshold later. A .slop.json at the root holds the defaults; this repository's own sets maxScore to 3 and lints its own Markdown in CI.
What the fixer will and will not touch
--fix does only what cannot change meaning. An em dash becomes a comma or a full stop, or disappears after punctuation. Curly quotes become straight. in order to becomes to. On the fixture that is seven fixes. The other forty-three findings stay, because a negative parallelism needs a rewritten sentence, and a tool that rewrites your sentences is a different tool with a different failure mode. The fixer runs to a fixed point: fixing a fixed file changes nothing.
In a repository
The Action lints the pull request description and every changed Markdown file, and annotates the diff at the line:
- uses: Bubblegunn/ai-slop-linter@v0
# with:
# max-score: "5"
# warn-only: "true"
The commit hook is one line, ai-slop-linter --commit-msg "$1", and refuses a message with an error-severity tell. git commit --no-verify skips it once. --pr 42 reads a pull request body through gh and lints it like a file.
--init writes a .slop.json and whichever of those you pick. Inside it, overrides set rules per path, so documentation can be stricter than commit messages, and --baseline records what a repository already has so CI fails only on new findings. There is a commitlint plugin and a VS Code task for the same output.
There is also a skill, installed with npx skills add Bubblegunn/ai-slop-linter, that tells a coding agent to lint its own prose before handing it over. That is where most of these tells get written in the first place.
Run on my own writing
Before posting I ran it on the five READMEs I maintain, this one included, and the eleven essays on my site, at version 0.1.4 on 5 September 2026, with --warn. The documents keep growing, so the version and the date are part of the claim; the same table is in the repository and is re-run rather than edited. The current release is 0.1.5, and src/ is byte for byte what 0.1.4 shipped, so these numbers are what you get today as well:
text
words
grade
findings
proactive-gate README
4,871
A (0)
none
workproof README
4,121
A (0)
none
ai-slop-linter README
3,744
A (0)
none
surviving-lines README
2,002
A (0)
none; the first run scored A (2.8) for 2 bold labels in a list
product-engineer README
1,755
A (0)
none; the first run scored C (8.7) for 7 bold labels in the rule list
8 of 11 portfolio essays
854 to 1,849 each
A (0)
none
the other 3 essays
866 to 1,601
A (0.4 to 1.2)
state-of-the-art once, elevated once, in order to twice
The C was real. The product-engineer README listed its seven rules as **Name:** text bullets, which is exactly the pattern bold-label flags. I wrote those bullets myself and thought they read fine. The linter disagreed, with a line number. Later the same day that README was rewritten to show the tool before explaining it, and the rule list became plain sentences on the way; it was not edited to please the linter, and the row keeps the first score so the table stays a record of runs rather than a trophy.
I assumed the punctuation rules were language-neutral. They were not
The vocabulary rules are English word lists, so outside English they simply never fire. I had assumed the typography rules were the portable half: a dash is a dash, a curly quote is a curly quote. Before publishing I checked, by running all twenty rules against correct published typography in thirteen languages, one public-domain text each, all typeset before any language model existed.
dash carries the highest severity in this tool. Here is what it did per 1,000 words:
language
dash findings per 1,000 words
Polish
73.5
Hungarian
52.1
Russian
24.0
French
22.0, see below
German
6.7
English (human baseline)
1.0
One caveat belongs here rather than in a footnote, because I would want it if I were reading this. Polish, Hungarian, Russian and German are what the files measure as they stand. The French text reaches us with its em dashes typed as two hyphens attached to the following word, which the rule does not match, so it scores 0 as transcribed and 22.0 once the dashes the printed edition set are restored. The repository prints both columns. The restored figure is the one a French writer meets today, because nobody types a dash as two hyphens in a README.
The Polish, Hungarian and Russian texts graded F for punctuating their own language correctly. The em dash is ordinary in French, opens dialogue in Spanish, and in Russian stands where the verb would go. curly-quotes did the same to Chinese at 21.7 per 1,000 words, because those code points are Chinese quotation marks and the straight apostrophes nested inside them are correct, not a paste artifact.
So a repository now declares what it writes in, "language": "fr" in .slop.json or --language fr, and dash stands down. The run prints which rules did not run, so a quiet result is never mistaken for a clean one. English stays the default, so nothing changes for anyone already using it. It is configuration rather than detection on purpose: a README with English headings over French prose defeats a guess in both directions.
I tried a density threshold first, so the rule could stay on everywhere and only fire when dashes were unusually dense. It does not work, and the corpus says why: correct German prose sits at 6.7 findings per 1,000 words and the machine corpus sits at 5.7. The distributions overlap. No global number separates a German writer from a model.
That leaves a real cost, and it belongs here rather than in a footnote: a machine-written French README will no longer be flagged for its dashes. I would rather lose that than grade a French writer's correct punctuation an F.
The corpus, the table and the argument are in the repository at bench/TYPOGRAPHY.md and docs/typography-across-languages.md, and CI re-runs the table so it cannot drift from the code.
What it costs on good writing
The objection I would raise first is false positives, so I measured them. Two corpora live in the repository. The human one is 5,924 words of public-domain prose written before any language model existed: Austen from 1813, Douglass from 1845, Darwin from 1859, and PEP 8 and PEP 257 from 2001, which are the nearest public-domain match to the register this tool aims at. The machine one is 1,045 words of unedited model output, written for the corpus.
corpus
file
words
grade
findings
human
Austen
1,120
A
0
human
Darwin
1,199
B
2
human
Douglass
1,202
B
3
human
PEP 257
1,174
A
1
human
PEP 8
1,234
B
2
machine
article intro
175
F
22
machine
commit messages
225
F
13
machine
docs page
225
F
10
machine
pull request
161
F
7
machine
README section
259
F
31
Eight findings on the human side, and all eight are printed in the repository rather than tuned away: six are real dashes in nineteenth-century prose and -- in a 2001 style guide, one is not just X but Y written by Frederick Douglass, one is it's worth noting written by Guido van Rossum. They are the standing cost of those rules, and the reason the output is a line number and not a verdict.
Measuring changed two rules rather than confirming them. curly-quotes first graded the Austen excerpt D on 92 findings, all of them the same rule: it was measuring typography, since the machine corpus had none. It now fires only on a file that mixes curly and straight marks, which is the paste signature. not-x-but-y could not match isn't just X, it's Y, the shape people actually write, so its separation was zero; it is now 14.5 times.
npm run bench regenerates the per-rule table, and CI fails when the committed file does not match a fresh run. Seven thousand words cannot support an accuracy percentage, and the file says so.
What it cannot show
Authorship. A person who writes delve gets the same finding as a model. Passing says the listed tells are absent and nothing about who typed.
Meaning. A hollow paragraph that avoids every listed phrase passes. Text that passes can still be empty.
Style outside the list. Twenty rules cover the patterns editors flag most; a writer with a different tell walks through. Adding a rule is one function, one fixture sentence and one table row.
Other languages. The vocabulary rules are English word lists and are inert elsewhere. The typography rules are not inert, and until 0.1.4 they were wrong; see the section above.
What is left on the roadmap is a rule set for a second language, which somebody has already claimed. A probability that a text was machine-written is deliberately not on it: seven commercial detectors once marked 61% of human-written TOEFL essays as machine-written (Liang et al., Patterns, 2023), and a tool that guesses authorship gets people accused. This one shows the tells and leaves the judgement where it belongs.
Which tell do you see most often that is not among the twenty? Name it and I will write the rule.
Read original: https://dev.to/efe_genc/i-wrote-a-linter-for-ai-writing-tells-it-gave-one-of-my-own-readmes-a-c-1mmf
← Previous
I made my coding agent explain every change to the customer. Here is the skill.
Next →
3 Ways I Use AI to Auto-Generate Playwright Locators
Related
AI Won’t Fix a Broken Business Process
AI & ML
3
DEV Community
"Keeping a record: working across many sessions"
AI & ML
2
DEV Community
I made my coding agent explain every change to the customer. Here is the skill.
AI & ML
3
DEV Community
Two kinds of junior devs right now, and only one of them is actually learning
AI & ML
1
DEV Community
Comments0
No comments yet — be the first