I generate long documents as files and publish them with a script: reports, proposals, article bodies. The publisher never reads the sentences. It checks that the media paths exist and that the body is not empty, then puts the page live.

That gap is where two different text corruptions survived for weeks. One of them is invisible to the check most people write first.

The defect that looks like damage

The first time, a file came back with words fused together. marinhoprofundo where two words belonged, branco,.grid where a table header used to be. In the same document, a stretch of characters from no Latin alphabet sat in the middle of a sentence.

This form announces itself. You scan the file, see the damage, and fix the source.

The cause was not the publisher. It was the tool that wrote the file. Long text with accents and punctuation passes through it in chunks, and occasionally a chunk returns with pieces glued or replaced. By then the publisher had already accepted the file, the validator had passed it, and the card was live with the broken text.

The defect that looks clean

The second form does not look like damage, because every byte in it is seven-bit ASCII.

I found three cases in my own documents. A list item read agendamento, CRM,|Issue tracker: a comma jammed against a pipe, and an English label right after it. Another line read Três Interactions, a Portuguese word followed by the English one. A third read e é priced como tal, an English verb inside a Portuguese sentence.

Run this against those three lines:

import re re.findall(r"[^\x00-\x7F]", texto)

You get an empty list. Nothing sits outside the ASCII range, so any check built on unusual characters returns clean on a document a reader will notice in the first paragraph. The defect is lexical, and no character class catches a wrong word.

Nothing downstream complains

The publishing CLI does not parse prose. My validator compares the manifest against the publication JSON, which means it compares media counts, body length and ids. Both sides agreed, because both were describing structure.

A structural check cannot see that a sentence contains the wrong language. It can only tell you the sentence exists.

That is why the gate has to inspect the text itself, in the step before publishing, and not the record after it.

Three checks in one file

The gate I use is a single Python file, run before the publish command. It exits 1 on any defect, so it chains with && and the publish never runs on a bad source.

These are the checks, abridged from the real file:

OK = set("áàâãäéêíóôõöúüçÁÀÂÃÄÉÊÍÓÔÕÖÚÜÇ…") ESTRANGEIRAS = re.compile(r"\b(issue|tracker|interactions|priced|checklist|starter)\b", re.I) PIPE_PROSA = re.compile(r"\S\|\S") t = pathlib.Path(sys.argv[1]).read_text() prosa = re.sub(r"``` .*? ```", " ", t, flags=re.S) defeitos = [] estranhos = sorted({c for c in re.findall(r"[^\x00-\x7F]", t) if c not in OK}) if estranhos: defeitos.append(f"caracteres corrompidos: {estranhos}") if ESTRANGEIRAS.search(prosa): defeitos.append("palavra de outro idioma no meio da prosa") if PIPE_PROSA.search(prosa) and not _tem_tabela(prosa): defeitos.append("pipe no meio da prosa")

The first check handles the visible form. It collects every non-ASCII character and compares against the set of accented letters and punctuation that belong in Portuguese.

The second check handles the invisible form. It is a short list of English words that showed up glued into my prose. The list stays short on purpose. Terms that Brazilian tech writing uses legitimately, such as bot, lead, setup, dashboard, status and deploy, are not on it. A gate that cries wolf gets ignored, and the ignoring generalizes to the checks that work.

The third check uses a punctuation mark as evidence. A pipe is a table separator in markdown and never appears inside running prose. When one appeared mid-sentence, it marked the exact span where the text had been spliced.

Two guards that keep the gate trusted

A checker is only useful while you believe its output, so both guards here exist to remove false positives.

The first strips fenced code blocks before the prose checks. An ffmpeg filter containing adelay=0|0 is correct syntax and failed the pipe check, which taught me to skim past failures. The character check still runs over the entire text, code included, because corruption inside a code block is a defect too.

The second detects a legitimate markdown table by looking for a separator line such as ---|--- under a line with pipes. Without it, a real table with pipes in its cells fails the gate for being what it is.

The allowed character set was tuned the same way. A correct text containing ± and → failed as corrupted on a measurement article about color. Those symbols have no place in a Portuguese document, so the fix was to write "35 degrees to each side" and a colon instead of an arrow. When a checker flags correct text, either the checker changes or the style does, and pretending it was fine is how the whole gate loses credibility.

Where the damage is born

The gate runs on the source file, before publishing. Correcting the JSON afterwards costs a second deploy, and until that deploy the card sits live with the wrong text. One full cycle of that was enough to make the ordering permanent.

Two practices came out of the same class of defect, where the wrong value is plausible instead of obviously wrong.

Tables with numbers that leave the house are built from data. I write the rows as a list of tuples, derive the total with sum(), and assert it against the number typed into the prose. When the assert fails, the column is wrong, and I learn it on my machine instead of in a client conversation.

Documents that exist in two formats come from one object. A markdown body and a PDF generated from the same structure stay in sync by construction. Two hand-written versions diverge in an accent or a figure, both look readable, and they reach different recipients.

Para quem publica em português

Se você gera texto longo com ferramenta e publica por script, um gate de alguns segundos vale mais que reler o arquivo uma vez. A releitura acontece quando você lembra, o gate acontece em toda publicação, e ele acusa o defeito antes de o card ir ao ar. Me conta nos comentários qual foi a corrupção mais estranha que já passou pelo seu pipeline, porque esse tipo de caso raramente aparece documentado.