Tested 2026-11-16. Disclosure: I work on UnAIMyText, and the #3 spot in the tool list is my own ranking. Copy and modify the linter however you like.

Black Friday is the week our alerts actually matter. Last year, at 3 a.m. on the Saturday after Thanksgiving, my phone lit up with this:

This alert indicates that the checkout service is currently experiencing a critical issue requiring immediate attention.

Which checkout pods? How bad? What should I look at first? The summary didn't say, and there was no runbook link. Months earlier someone had asked a model to "write annotations for our alerts," and all 60 of our rules read like that.

This year I wanted a check in CI that catches annotations like that before they reach a pager. It's one TypeScript file run by Deno. It lints summary and description in Prometheus rule files, and with --draft it asks a Kimi model from Moonshot AI to write the two annotations for a new alert, then lints what comes back.

The Kimi part, briefly

Moonshot's Chat Completions API reference is OpenAI-shaped: POST https://api.moonshot.ai/v1/chat/completions with Authorization: Bearer $MOONSHOT_API_KEY plus the usual messages. I default to kimi-k3.1-pro, the model in the docs' examples, and MOONSHOT_MODEL overrides it. From the docs:

  • JSON mode. response_format: {"type": "json_object"} forces a JSON object, but the docs say you must describe the fields in the prompt. The system message names summary and description.
  • Thinking. The K3 series always reasons, and you control the depth with a top-level reasoning_effort of low, high or max (default max). For two sentences, low is plenty.
  • Where the answer is. The text is in choices[0].message.content. The reasoning arrives separately in reasoning_content, which I ignore.

I hit the real endpoint with a made-up key and got a 401 with "Invalid Authentication". Without the header it's also 401, with "Incorrect API key provided". The linter exits 2 for both.

Deno runs TypeScript as is, with fetch built in. The only import is jsr:@std/yaml@1.3.0, pinned. Permissions are explicit: lint mode needs -R (read) and -E (env); draft mode adds -N (net).

What it checks

Per alerting rule:

  • Missing summary, description, severity label or runbook_url, and runbook URLs that aren't http(s)
  • A summary over 80 characters, or one ending in a period
  • A description over 60 words, or one with no {{ $labels.* }} or {{ $value }}, so the page can't tell you which instance or how bad
  • Openers like "This alert", "Please note" and "It appears"
  • Filler: robust, comprehensive, proactively, leverage, in order to, ensure that, critical issue, immediate attention, currently experiencing, seamless
  • Hedges like "may potentially", exclamation marks and emoji
  • TODO, TBD, FIXME and [insert ...] leftovers, and {{ without a matching }}
  • The same alert name defined twice

The period rule is house style. Prometheus doesn't care, but the official examples write summaries as headlines and descriptions as sentences. Drafts skip the severity and runbook checks, since the model doesn't write those.

Exit codes: 0 when everything passes, 1 when there are findings, and 2 for usage mistakes, unreadable files, invalid YAML, files with no alerting rules, network failures and API errors.

The linter

// alertlint.ts checks the summary and description annotations in Prometheus // alerting rules, or asks a Kimi model (Moonshot AI) to draft them and checks the draft. // // Usage: // deno run -RE alertlint.ts rules.yml [more.yml] lint rule files // cat rules.yml | deno run -RE alertlint.ts - read from stdin // deno run -RE -N alertlint.ts --draft DiskAlmostFull "notes..." // add --json for machine-readable output // // Exit: 0 clean, 1 lint findings, 2 usage/parse/network/API errors import { parse } from "jsr:@std/yaml@1.3.0"; const MAX_SUMMARY = 80; const MAX_DESC_WORDS = 60; const FILLER = /\b(seamless(ly)?|robust|comprehensive|proactive(ly)?|leverag(e|es|ing)|ensures? that|in order to|critical issue|immediate attention|currently experiencing)\b/gi; const OPENER = /^(this alert|please note|it (appears|seems)|heads up|attention)\b/i; const HEDGE = /\b(may|might|could) (potentially|possibly)\b/i; const EMOJI = /\p{Extended_Pictographic}/u; const PLACEHOLDER = /\b(TODO|TBD|FIXME)\b|\[(insert[^\]]*|description)\]/i; const TEMPLATE_VAR = /\{\{\s*(\$labels\.\w+|\$value|humanize\w*\s+\$value)/; type Finding = { alert: string; issue: string }; type Rule = { alert?: string; labels?: Record<string, string>; annotations?: Record<string, string> }; function lintText(alert: string, field: string, text: string, out: Finding[]) { const add = (issue: string) => out.push({ alert, issue: `${field}: ${issue}` }); if (OPENER.test(text)) add(`opens with "${text.match(OPENER)![0]}"`); for (const m of new Set([...text.matchAll(FILLER)].map((x) => x[0].toLowerCase()))) add(`filler "${m}"`); if (HEDGE.test(text)) add(`hedge "${text.match(HEDGE)![0]}"`); if (text.includes("!")) add("exclamation mark"); if (EMOJI.test(text)) add("emoji"); if (PLACEHOLDER.test(text)) add(`placeholder "${text.match(PLACEHOLDER)![0]}"`); if ((text.match(/\{\{/g) ?? []).length !== (text.match(/\}\}/g) ?? []).length) add("unbalanced {{ }}"); } export function lintRule(r: Rule, opts = { draft: false }): Finding[] { const out: Finding[] = []; const alert = r.alert ?? "(unnamed)"; const a = r.annotations ?? {}; const summary = (a.summary ?? "").trim(); const desc = (a.description ?? "").trim(); if (!summary) out.push({ alert, issue: "no summary annotation" }); if (!desc) out.push({ alert, issue: "no description annotation" }); if (summary) { if (summary.length > MAX_SUMMARY) out.push({ alert, issue: `summary is ${summary.length} chars (max ${MAX_SUMMARY})` }); if (summary.endsWith(".")) out.push({ alert, issue: "summary ends with a period" }); lintText(alert, "summary", summary, out); } if (desc) { const words = desc.split(/\s+/).length; if (words > MAX_DESC_WORDS) out.push({ alert, issue: `description is ${words} words (max ${MAX_DESC_WORDS})` }); if (!TEMPLATE_VAR.test(desc)) out.push({ alert, issue: "description names no instance or value ({{ $labels.* }} / {{ $value }})" }); lintText(alert, "description", desc, out); } if (!opts.draft) { if (!r.labels?.severity) out.push({ alert, issue: "no severity label" }); const rb = a.runbook_url; if (!rb) out.push({ alert, issue: "no runbook_url annotation" }); else if (!/^https?:\/\//.test(rb)) out.push({ alert, issue: `runbook_url "${rb}" is not an http(s) URL` }); } return out; } export function lintRules(yamlText: string): Finding[] { const doc = parse(yamlText) as { groups?: { rules?: Rule[] }[] } | null; const rules = (doc?.groups ?? []).flatMap((g) => g.rules ?? []).filter((r) => r.alert !== undefined); if (rules.length === 0) throw new Error("no alerting rules found (expected groups[].rules[].alert)"); const out: Finding[] = []; const seen = new Map<string, number>(); rules.forEach((r, i) => { if (seen.has(r.alert!)) out.push({ alert: r.alert!, issue: `duplicate alert name (also rule ${seen.get(r.alert!)! + 1})` }); else seen.set(r.alert!, i); out.push(...lintRule(r)); }); return out; } async function draft(alert: string, notes: string): Promise<Rule> { const key = Deno.env.get("MOONSHOT_API_KEY"); if (!key) throw new Error("MOONSHOT_API_KEY is not set"); const base = Deno.env.get("MOONSHOT_BASE_URL") ?? "https://api.moonshot.ai/v1"; const res = await fetch(`${base}/chat/completions`, { method: "POST", headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" }, body: JSON.stringify({ model: Deno.env.get("MOONSHOT_MODEL") ?? "kimi-k3.1-pro", reasoning_effort: "low", response_format: { type: "json_object" }, messages: [ { role: "system", content: 'You write Prometheus alert annotations for on-call engineers. Reply with a JSON object with two string fields: "summary" (under 80 characters, no final period) and "description" (one or two plain sentences that use {{ $labels.instance }} or {{ $value }}). No greetings, no filler.', }, { role: "user", content: `Alert name: ${alert}\nWhat it means: ${notes}` }, ], }), }); const body = await res.text(); if (!res.ok) throw new Error(`HTTP ${res.status}: ${body.slice(0, 200)}`); const content: string = JSON.parse(body).choices?.[0]?.message?.content ?? ""; const stripped = content.trim().replace(/^`{3}[\w-]*\s*\n?|\n?`{3}\s*$/g, ""); if (!stripped) throw new Error("model returned no content"); const ann = JSON.parse(stripped); return { alert, annotations: { summary: String(ann.summary ?? ""), description: String(ann.description ?? "") } }; } async function main(args: string[]): Promise<number> { const json = args.includes("--json"); args = args.filter((a) => a !== "--json"); let findings: Finding[]; try { if (args[0] === "--draft") { if (args.length !== 3 || args[1].startsWith("-")) throw new Error('usage: --draft AlertName "what it means"'); const rule = await draft(args[1], args[2]); console.log(`summary: ${rule.annotations!.summary}\ndescription: ${rule.annotations!.description}`); findings = lintRule(rule, { draft: true }); } else { if (args.length === 0) throw new Error("usage: alertlint.ts rules.yml [...] | - | --draft AlertName notes"); findings = []; for (const f of args) { const text = f === "-" ? await new Response(Deno.stdin.readable).text() : await Deno.readTextFile(f); findings.push(...lintRules(text)); } } } catch (e) { console.error(`alertlint: ${e instanceof Error ? e.message : e}`); return 2; } if (json) console.log(JSON.stringify(findings, null, 2)); else if (findings.length === 0) console.log("ok"); else for (const f of findings) console.log(`${f.alert}: ${f.issue}`); return findings.length ? 1 : 0; } if (import.meta.main) Deno.exit(await main(Deno.args)); 

Running it

deno run -RE alertlint.ts rules/checkout.yml deno run -RE alertlint.ts --json rules/*.yml MOONSHOT_API_KEY=... deno run -RE -N alertlint.ts --draft DiskAlmostFull "a mount is over 90% used" 

Here's the kind of rule file that used to page us:

groups: - name: checkout rules: - alert: CheckoutHighErrorRate expr: job:errors:ratio5m{job="checkout"} > 0.02 for: 5m annotations: summary: "This alert indicates that the checkout service is currently experiencing a critical issue requiring immediate attention." description: "Please note that error rates may potentially impact customers! Proactively leverage the dashboards in order to ensure that checkout remains robust 🚨" runbook_url: TODO - alert: QueueBacklog expr: queue_depth > 1000 labels: severity: page annotations: summary: "It appears the queue is backing up {{ $labels.queue }" - alert: CheckoutHighErrorRate expr: job:errors:ratio5m{job="checkout"} > 0.05 labels: severity: page annotations: summary: "Checkout errors high" description: "[insert details] comprehensive monitoring has detected elevated errors." runbook_url: https://runbooks.example.com/checkout-errors 

And what the linter says about it:

CheckoutHighErrorRate: summary is 120 chars (max 80) CheckoutHighErrorRate: summary ends with a period CheckoutHighErrorRate: summary: opens with "This alert" CheckoutHighErrorRate: summary: filler "currently experiencing" CheckoutHighErrorRate: summary: filler "critical issue" CheckoutHighErrorRate: summary: filler "immediate attention" CheckoutHighErrorRate: description names no instance or value ({{ $labels.* }} / {{ $value }}) CheckoutHighErrorRate: description: opens with "Please note" CheckoutHighErrorRate: description: filler "proactively" CheckoutHighErrorRate: description: filler "leverage" CheckoutHighErrorRate: description: filler "in order to" CheckoutHighErrorRate: description: filler "ensure that" CheckoutHighErrorRate: description: filler "robust" CheckoutHighErrorRate: description: hedge "may potentially" CheckoutHighErrorRate: description: exclamation mark Check...
Source:https://dev.to/unaimytext_app/humanize-kimi-output-with-deno-a-prometheus-alert-annotation-linter-5-free-tools-2026-3hom