The SHA-256 of https://predictioncn.com/ and https://predictorcn.com/ is 349268e32abe358f… — the same 77,375 characters, byte for byte, served from two different hostnames with two different Last-Modified-free responses and the same Cloudflare edge colo in the cf-ray suffix.
I found that by accident. I had a cheaper question: after 15 posts about sitemaps, soft-404s and font files on this 114-domain network, do any two homepages actually say the same thing? So I hashed all 114 and compared every pair. Six thousand four hundred and forty-one pairs, and the answer for 6,273 of them was "nothing in common at all".
The two that matched were not near-matches. They were clones.
What I compared, and what I deliberately ignored
The corpus is 114 forecast and market-analysis sites — coin price pages, IPO trackers, and methodology notes that explain how a forecast number is assembled — mostly static HTML behind Cloudflare, all fetched on 2026-11-16 with Accept-Encoding: identity so I got plain text instead of a zstd blob I would have to decode before parsing.
The fingerprint is dead simple: strip <script>, <style> and tags, lowercase, tokenize on non-alphanumerics, then hash every 8-word sliding window. Eight words is long enough that boilerplate ("read more", "risk disclosure") stops matching and short enough that a swapped noun still leaves most shingles intact.
import re, hashlib RE_SS = re.compile(r"<(script|style|noscript)\b[^>]*>.*?</\1>", re.S | re.I) RE_TAG = re.compile(r"<[^>]+>") def visible_text(html): h = RE_SS.sub(" ", html) h = RE_TAG.sub(" ", h) h = h.replace(" ", " ").replace("&", "&") return re.sub(r"\s+", " ", h).strip().lower() def shingles(text, k=8): w = [x for x in re.split(r"[^a-z0-9\u4e00-\u9fff]+", text) if x] return { hashlib.blake2b(" ".join(w[i:i + k]).encode(), digest_size=8).digest() for i in range(max(0, len(w) - k + 1)) } def jaccard(a, b): return len(a & b) / len(a | b) if (a or b) else 0.0 8 bytes per shingle is plenty: with ~6,400 pairs I am not defending against an adversary, I am looking for copies. And I compare sets, not vectors, because I care whether two pages share sentences, not whether they share them in the same order.
6,273 pairs shared nothing
The distribution is brutally bimodal. Across 6,441 unordered pairs:
- 6,273 pairs share zero 8-word shingles.
- 168 pairs share at least one; the median of those is 0.0005 and the 90th percentile is 0.0087.
- Exactly 2 pairs clear 0.30. The next highest pair in the entire network is 0.1854, so there is no messy middle to argue about.
- 60 of the 114 sites have no partner at all — not one shared 8-word window with any other site.
So the network is not a content farm in the sense people usually mean. Whatever else is wrong with these sites, they are not 114 copies of one page — one host is a guide to reading a market forecast, the next is a coin list, the next is a legal page in two languages. That was worth measuring before writing anything, because the alternative conclusion — "it's all the same template" — would have been wrong and easy to assert from the outside.
The redundancy number agrees. Concatenating all 114 homepages gives 9.70 MB of HTML. Gzipping each page separately sums to 1.425 MB; gzipping the concatenation gives 1.369 MB. That is 3.9% — the cross-site redundancy that a shared dictionary would actually capture is a rounding error next to the 96% that is genuinely different bytes.
The pair that matched exactly
predictioncn.com and predictorcn.com are the same document. 77,375 characters, identical SHA-256, identical Last-Modified: Fri, 23 Oct 2026 09:10:33 GMT, both cf-cache-status: DYNAMIC, both served from LAX. 677 words of visible text each.
The detail that makes it a deployment problem rather than a coincidence: the HTML contains no self-reference. Grep predictorcn.com in the page served by predictorcn.com and you get 0 hits. Grep predictioncn.com in the page served by predictioncn.com and you get 4. Every internal link is relative, so the same file is valid on any hostname you point at it:
<link rel="canonical" href="/"> <meta property="og:url" content="/"> That is the whole mechanism. A host-agnostic document plus a second hostname pointed at the same origin equals a perfect clone, and nothing in the build pipeline notices, because from the build's point of view it emitted one page.
The mirror serves the other host's sitemap
This is where it stops being a curiosity. predictorcn.com/sitemap.xml returns 18,707 bytes containing 247 <loc> entries — and all 247 point at predictioncn.com. It is byte-identical to the sitemap on predictioncn.com: same SHA-256 (9ace61d2ca566541…), same Last-Modified: Sun, 08 Nov 2026 06:29:25 GMT.
And predictorcn.com/robots.txt has no Sitemap: line at all, while predictioncn.com/robots.txt declares Sitemap: https://predictioncn.com/sitemap.xml.
Read that from a crawler's point of view. You land on the mirror. Its robots file advertises nothing. If you find the sitemap anyway — convention, or the Search Console submission someone made months ago — every URL it hands you belongs to a different host. The mirror cannot be crawled as itself even if someone wanted to.
Why a self-canonical does not fix this
Both pages declare canonical as /. That is a relative reference, and a relative canonical resolves against the URL that was requested. On predictioncn.com it means https://predictioncn.com/; on predictorcn.com it means https://predictorcn.com/.
So each host tells the crawler "I am the original", and both are telling the truth about themselves. There is no cross-host consolidation signal anywhere in the pair. If the intent was to keep one copy, the canonical had to be absolute and point at one host — and the other host should have answered with a 301, not a 200 carrying the same 77 KB.
I am not going to claim what a specific search engine does with two self-canonicalising identical hosts. The measurable part is enough: the two documents are identical, both are reachable with 200, neither names the other, and the sitemap on one of them lists only the other.
The 95-character clone
The second real pair is smaller but more instructive. marketoracledaily.com and marketpredictiondaily.com score 0.9953 on 8-word shingles: 66,507 and 66,510 characters, 871 words each.
A difflib comparison finds 15 changed regions and about 95 characters that differ — 0.14% of the document. Every one of them is the same substitution, the brand noun oracle replaced by prediction, in the title, the og:site_name, the footer and the body sentences:
[replace] A: 'racle' B: 'n' [insert] A: '' B: 'predicti' Two sites, one document, and a sed command between them. Both declare an absolute self-canonical (https://marketoracledaily.com/ and https://marketpredictiondaily.com/), and both carry the identical <title> and identical meta description, so the duplication is not hidden in the body — it is in the exact strings that get serialised into a result snippet.
This is a different failure mode from the mirror. The mirror is one file served twice. This is one template rendered twice with a variable the build system was told to substitute. If you run a network like this, the second pattern is the one that scales silently: nobody typed a duplicate page, the generator did.
Same title, same description, different page
One more collision, and it is the one I would fix first. spacexstockanalysis.com and spacexstockinsights.com share an identical <title> — "SpaceX 股票上市时间、代号、走势与购买指南 | SpaceX 股票最新动态 2026" — and an identical meta description.
But the bodies are not copies: 246 words versus 333, and a shingle similarity of 0.0814. Two different pages wearing the same identity tag. That is arguably worse than a clone, because a clone at least has a defensible reason to exist (someone mirrored a file); this is two pages actively competing for one snippet with no way to tell which one should win.
Across the whole network, three pairs share a title and three share a description, and those three pairs are the same two-plus-two-plus-two I have been describing. The metadata layer has 3 collisions out of 114 pages; the body layer has 2 out of 6,441 pairs. Small numbers, but they cluster on the same seven domains, which is the actual signal: duplication here is a per-project build bug, not a network-wide policy.
A relative canonical that resolves to a 301
While checking canonicals I found a second, quieter bug in the same family. Six of the 114 sites write the canonical as a relative path instead of an absolute URL:
cryptoforecastpro.com canonical="/" marketforecasthub.com canonical="/" marketoracleguide.com canonical="/" predictioncn.com canonical="/" predictmarketcn.com canonical="/" predictorcn.com canonical="/" On a homepage that happens to resolve to the right thing, so it never shows up in a spot check. On predictmarketcn.com inner pages it does not. The page at /prediction-market/ declares:
<link rel="canonical" href="/prediction-market"> No trailing slash. That URL returns 301 to https://predictmarketcn.com/prediction-market/, which is the page you are already on. I sampled 7 inner pages on that host and all 7 canonicalise to a redirect: /polymarket, /guides, /platforms, /compare, /learn, /wiki, and the one above.
A canonical tag is supposed to name a 200 URL. Naming a 301 means the crawler has to follow a hop before it can even evaluate the hint, and the destination already advertises itself. The fix is one line in the template — absolutise the canonical and use the final URL — but you will only find it if something reads the tag and then requests what it points at. Nothing in a normal build does that, which is how a 2.9 KB methodology note for a set of perpetual-contract guides can sit in production for months with nobody confirming that the URL it names actually resolves.
My first pass invented three duplicates
Everything above is from the second run. The first run reported 5 pairs above 0.30, including a three-site cluster at 0.946, 0.949 and 0.949. I nearly wrote that up as "three sites share a template".
It was an encoding bug. requests only sets response.encoding from the Content-Type header, and these responses say text/html with no charset. The library then falls back to ISO-8859-1, so every UTF-8 Chinese character — which is most of the visible text on these pages — became mojibake. Chinese tokens disappeared from the shingle stream entirely, and what was left was the ASCII boilerplate that all of these templates share: nav labels, "Read more", "Risk disclosure", dates. The similarity score was measuring the template, not the content.
Forcing the decode on the same pages:
r = requests.get(url, headers={"Accept-Encoding": "identity"}, timeout=30) r.encoding = "utf-8" html = r.text The three-site cluster collapsed from 0.946 to 0.0663 (chaintickerzone.com vs perpliquidationmap.com, 2,017 and 2,019 words). Only the two genuine pairs survived.
The lesson is not "set the encoding". It is that a fingerprint job has no error bar. A hash either matches or it does not, so a normalization bug does not produce a slightly wrong number, it produces a confident wrong conclusion — in this case three duplicate sites that were never duplicates. I now print the decoded sample and the token count for every page before trusting any pairwise score, because a page whose tokens are mojibake has a suspiciously short token list.
The check that belongs in CI
The whole audit is cheap — 114 fetches and 6,441 set intersections in about 30 seconds — so it belongs in the deploy pipeline rather than in a quarterly spreadsheet. Three assertions cover everything above:
import hashlib def check(site, html, canon, sitemap_locs, shingles, peers): problems = [] digest = hashlib.sha256(html.encode()).hexdigest() for other, other_digest, other_sh in peers: if digest == other_digest: problems.append(f"byte-identical to {other}") elif jaccard(shingles, other_sh) >= 0.5: problems.append(f">=0.5 shingle overlap with {other}") if canon is not None and not canon.startswith(("http://", "https://")): problems.append(f"relative canonical: {canon!r}") if canon and canon.rstrip("/") != site.rstrip("/"): problems.append("canonical is not self") for loc in sitemap_locs: if site.split("//")[1].split("/")[0] not in loc: problems.append(f"sitemap lists another host: {loc}") break return problems The third check is the one I would have skipped a week ago, and it is the one that would have caught the mirror. A sitemap whose every <loc> belongs to a different hostname is not a subtle signal — it just needs somebody to compare the <loc> host to the host that served the file.
What I am doing with the findings: the mirror needs to either 301 to the primary host or stop serving the primary's sitemap; the six relative canonicals need to be absolutised at the template level, with a check that the target is 200 and matches the final URL; and the two brand-swap sites need a decision about which one is canonical before the 95-character difference gets worse.
The number I keep coming back to is 6,273. Seventeen days of writing about what is broken on these sites, and the thing that is not broken — 60 of 114 homepages sharing nothing with any other homepage — was the finding I had to measure to believe. Most of the network is fine. The seven domains that are not fine are wrong in ways that are exact, reproducible, and invisible from inside a single repository.

![Motion Planning and Navigation — Ch.11: Legged Robot Navigation [KR]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqftgs44jxvhe8fxlcrvt.png)