Spectral and vacuum are the two OpenAPI linters most teams choose between. vacuum's pitch is that it's a much faster drop-in for Spectral that reads the same rulesets. I wanted to know how true that is on real specs, so I ran both on the same files with the same rulesets and compared the timings, the memory use and the actual findings.
Short version: vacuum is usually much faster but uses more memory, and "same ruleset" does not mean "same results." Details below. Every number here is from my own runs, and the ruleset is included so you can reproduce them.
(Disclosure: I build SpecShield, an API gate that accepts Spectral-format rulesets. I don't sell a linter, and both of these tools are free.)
The two tools in one paragraph each
Spectral (Apache-2.0, TypeScript, ~3,260 stars) is the original. Stoplight built it, and SmartBear acquired Stoplight in 2023. It's still actively released (v6.17.1, 6 November 2026) and it's what Stoplight, Swagger and a lot of API platforms embed. Rulesets are YAML, JSON or JS/TS; custom functions are JavaScript or TypeScript running in Node. It lints OpenAPI 2, 3.0, 3.1 and 3.2, AsyncAPI and Arazzo.
vacuum (MIT, Go, ~1,150 stars) began as a faster Spectral and has grown into a toolkit: linting plus autofix, an interactive terminal dashboard, an HTML report, a language server, docs generation, bundling, overlays, and change detection between two specs. It's released very frequently (v0.41.0, 14 November 2026), and most of the work comes from one maintainer, Dave Shanley.
Setup
- Machine: Apple M4 Pro, 48 GB, Node 22.17
- Spectral CLI 6.17.1, vacuum 0.41.0 (both from npm)
- Specs, all public:
- Petstore (OAI example, tiny)
- Stripe (
spec3.json, 8.3 MB: 612 operations, 1,538 schemas) - GitHub REST (
api.github.com.json, 14.3 MB: 1,281 operations, 1,016 schemas)
- Each run twice; timings were stable to within a few percent, so I've shown one.
Watch out on npm: a plain
npm i @quobix/vacuuminstalled 0.27.0 for me, fourteen minor versions behind the current release. Pin the version (@quobix/vacuum@0.41.0) or use Homebrew.
Test 1: the built-in OAS ruleset
Both tools were given the same .spectral.yaml:
extends: ["spectral:oas"] | Spec | Spectral time | vacuum time | Spectral peak RAM | vacuum peak RAM | Spectral findings | vacuum findings |
|---|---|---|---|---|---|---|
| Petstore | 0.28 s | 0.04 s | 128 MB | 49 MB | 8 | 20 |
| Stripe | 8.2 s | 0.81 s | 440 MB | ~980 MB | 618 | 23,989 |
| GitHub | 19.8 s | 2.66 s | ~2.7 GB | ~2.6 GB | 856 | 37,232 |
On speed, vacuum wins clearly: about 10× on Stripe and 7.5× on GitHub.
The findings column is what I didn't expect. vacuum reads spectral:oas as its own recommended ruleset, which has rules that spectral:oas doesn't. On Stripe, most of vacuum's 23,989 findings come from rules Spectral never ran:
| Rule | Spectral | vacuum |
|---|---|---|
camel-case-properties | 0 | 8,834 |
description-duplication | 0 | 7,364 |
oas3-missing-example | 0 | 3,614 |
component-description | 0 | 1,237 |
operation-description | 5 | 617 |
operation-tags | 612 | 612 |
Even rules with the same name don't always behave the same way (operation-description: 5 vs 617). On GitHub it goes the other way for examples: Spectral reported 774 oas3-valid-media-example findings, and vacuum reported none under that name.
So switching from Spectral to vacuum with "the same" config can turn a passing CI job into one with tens of thousands of findings. That isn't a bug in either tool — vacuum's recommended set is deliberately stricter — but it's not a drop-in in the sense most people mean.
Test 2: an identical custom ruleset
To compare like with like, I wrote five rules that use only core functions (truthy, pattern, casing, length) and no extends:
rules: op-id-required: severity: error given: "$.paths[*][get,put,post,delete,patch]" then: {field: operationId, function: truthy} op-summary-required: severity: warn given: "$.paths[*][get,put,post,delete,patch]" then: {field: summary, function: truthy} path-no-trailing-slash: severity: warn given: "$.paths" then: {field: "@key", function: pattern, functionOptions: {notMatch: ".+/$"}} property-snake-case: severity: info given: "$.components.schemas[*].properties" then: {field: "@key", function: casing, functionOptions: {type: snake}} description-max-length: severity: hint given: "$.paths[*][*].description" then: {function: length, functionOptions: {max: 2000}} | Spec | Spectral time | vacuum time | Spectral peak RAM | vacuum peak RAM |
|---|---|---|---|---|
| Petstore | 0.27 s | 0.04 s | 126 MB | 44 MB |
| Stripe | 3.97 s | 0.65 s | 423 MB | 860 MB |
| GitHub | 2.12 s | 2.23 s | 624 MB | 1,461 MB |
Two things stand out:
- The speed gap depends on the spec. vacuum was 6× faster on Stripe and slightly slower on GitHub. With a small ruleset, a lot of the time goes into parsing and resolving the document, not running rules.
- vacuum used about twice the memory on both large specs. If you lint big specs on small CI runners, that matters more than the seconds.
The findings still didn't match
On Stripe, the rules I wrote agreed exactly (16 missing summaries, 1 long description). vacuum also added 24 circular-references and 2 resolving-references findings that aren't in my ruleset; it runs some document checks no matter what ruleset you give it.
On GitHub, property-snake-case gave 35 findings in Spectral and 17 in vacuum, with no paths in common. Two reasons:
- They report the same problem at different places. For
scimType,timePeriod,usageItemsand+1, Spectral reports the schema where the property is defined (components.schemas.scim-error.properties.scimType). vacuum reports where that schema is used (components.responses.bad_request.content…properties.scimType). - They disagree on what snake_case is. Spectral flags
_links(a leading underscore). vacuum accepts it, so 18_linksfindings appear only in Spectral. The other 17 are the same properties in both tools, just at different paths.
The first point has a practical consequence: anything keyed on the finding's path won't carry over. That includes ignore files, baselines of "known" findings, and dashboards that de-duplicate by location. If you migrate, expect to rebuild those.
Custom functions
Both support custom JavaScript functions, but they aren't directly interchangeable:
- Spectral runs them in Node, and they can be written in TypeScript.
- vacuum runs JavaScript in goja, an embedded engine. It's not V8, has limited Node APIs and no TypeScript. Its docs say each function must export both
getSchema()andrunRule(), and that rulesets refer to the namegetSchema()returns, not the file name. vacuum also supports custom functions written in Go.
If your ruleset leans on custom functions (Box's public ruleset is a good example of one that does), budget time to port them.
Beyond linting
| Spectral | vacuum | |
|---|---|---|
| Output formats | json, stylish, junit, html, text, teamcity, pretty, github-actions, sarif, code-climate, gitlab, markdown | The same 12 |
| OWASP rules | Separate package @stoplight/spectral-owasp-ruleset (2.0.1, last published March 2024) | Built in but off by default; --hard-mode turns on every rule, OWASP included |
| Quality score / threshold | No | Yes (--min-score) |
| Autofix | No | Yes (--fix, or save fixes as an overlay) |
| Editor support | VS Code extension, JetBrains plugin | Language server (any LSP editor) |
| Interactive report | No | Terminal dashboard, HTML report |
| Compare two specs | No | --original + --error-on-breaking (uses the openapi-changes engine) |
| Ecosystem | Embedded in Stoplight/Swagger; many public company rulesets (Adidas, Azure, DigitalOcean, Zalando) | Smaller, growing |
| Telemetry | Docker image sends an anonymous startup ping (opt out with DO_NOT_TRACK=1) | Optional update check (--no-update-check) |
Which should you use?
- Big specs, CI time matters, you're starting fresh: vacuum. Start with its recommended ruleset on purpose, not by accident, and give the runner enough memory.
- You already have a tuned Spectral ruleset, custom TypeScript functions, or an ignore baseline: stay on Spectral, or budget a real migration. Run both side by side for a week and compare the findings by rule before you switch.
- Your platform already embeds Spectral (Stoplight, SwaggerHub, many API gateways): keep Spectral, so local results match what the platform shows.
- You want autofix, a score threshold or a dashboard without adding another tool: vacuum.
Whichever you pick, the lesson from Test 2 applies: "Spectral-compatible" is about whether the ruleset loads, not whether it gives the same answers. If you write your own rules, stick to core functions and avoid engine-specific extensions, so moving between linters later stays cheap.
(That's also why SpecShield's govern --ruleset accepts Spectral-format YAML and rejects engine-specific constructs. It's a small portability check, not a replacement for either linter.)
If you've migrated a large ruleset between the two, I'd like to hear what broke. And if you run these benchmarks on Linux CI runners and get different ratios, post them — mine are from one Mac.
