A canary release inside one cloud is a familiar exercise. Shift a small share of traffic, watch error rates and latency, promote or roll back. A canary that moves traffic between clouds is a different exercise, and treating it as the same one is how teams end up either too frightened to move anything, or moving everything at once because the small step never worked reliably.
The difference is not the percentage. Five percent between clouds crosses boundaries that five percent inside one provider never touches: a different load balancer, a different health model, often a different data path for the requests that follow the first one, and a rollback that has to work across the same boundaries in reverse. Designing for that, in advance, turns a leap of faith into a routine operation.
Start with where the decision is made
Traffic can be shifted in more places than most diagrams show, and each place has a different blast radius.
DNS weighting is the simplest lever and the bluntest. Records are cached by resolvers and clients for longer than the time to live suggests, some clients ignore it altogether, and a rollback through DNS is a waiting game rather than a switch. It works for coarse, slow moves between largely independent endpoints. It is a poor choice for a canary that needs to be reversed in minutes.
A global load balancer or traffic manager in front of both clouds gives a real switch: weights change in one place, new connections follow the new split quickly, and health based removal is automatic. The cost is that this layer becomes a critical dependency for both clouds, so its own health, limits, and failure behaviour need the same attention as the services behind it.
Routing the canary through an existing private interconnect or hub adds a third option: keep the public entry point fixed, and change where traffic goes after it arrives. That keeps client behaviour unchanged and makes the shift observable in one place, but it can add a detour. Measure the added latency before the canary, not during it, because a slow canary looks like a failing new version even when the code is fine.
Pick one decision point for a given move, write it down, and make sure everyone on the call knows which lever will be pulled and which levers will not be touched. Most failed canaries involve two levers changing at once and no way to tell which one caused the result.
A request is rarely the whole journey
Shifting the first request is easy to test. Shifting the session that follows is where cross cloud canaries break.
If a user lands on the new cloud for the first request and is sent back to the old cloud for the next one, because a session cookie points at a local store, a websocket reconnects to a different endpoint, or a file upload resumes against a different backend, the failure looks intermittent and user specific. It is neither. It is a locality problem.
Before any traffic moves, list what has to stay together: session state, uploaded files in progress, long lived connections, queued jobs started by the request, and callbacks from third parties that still point at the old address. For each item, decide whether it is shared between clouds, replicated with a known lag, or pinned so that a started journey finishes where it began. Pinning is underrated. For many services the safest canary rule is simple: new sessions may start on the new cloud, existing sessions finish on the old one, and nothing migrates mid flight.
The same question applies to data writes. If the canary writes to a local database that replicates back slowly, a rollback does not just move traffic, it strands the writes made during the canary window. Test that path deliberately: write during a trial shift, roll back, and confirm those records are visible and correct from the old side before calling the design ready.
Health has to mean the same thing in both places
Each cloud defines health in its own way, with its own probe interval, threshold, and view of what a healthy target is. During a canary those differences stop being trivia.
A target that flaps twice in the new cloud may be removed and returned automatically, swinging the effective traffic share well away from the planned five percent. A probe that checks a shallow status endpoint will report healthy while the deeper dependency, a database connection, an identity endpoint, a queue, is failing for real users. And if the two clouds report latency in different ways, the comparison that decides promotion is not a comparison at all.
Align three things before shifting anything. First, probe the same meaningful path in both clouds, ideally a lightweight request that exercises the real dependencies without creating real work. Second, set matching intervals and thresholds for the duration of the move, so removal behaviour is predictable. Third, agree in advance on the numbers that will decide the outcome: error rate, tail latency, and one business signal such as completed checkouts, finished jobs, or successful logins, each compared against the old cloud over the same window, not against a memory of last week.
Watch the split you actually have, not the split you set
A weight is an intention. The delivered split is what resolvers, connection reuse, retry behaviour, and a small number of very heavy clients actually produce. During a cross cloud canary, log the cloud that served each request at the entry point and report the real percentage alongside the configured one. A canary that was meant to be five percent but delivered thirty percent explains a lot of confusing results, including a capacity problem on a new cloud that was sized for five.
Retries deserve special attention here. A client that retries a failed request against the other cloud can turn a small error rate on the new side into duplicated work and a misleading success rate. Decide whether cross cloud retries are allowed during the move, and if they are, make them visible as retries in the logs rather than letting them blend into normal traffic.
Capacity follows the same logic. Size the new cloud for the share after next, not the first share. If the plan is five, then twenty five, then fifty percent, the canary should prove the path can carry the next step, or the promotion decision is being made on evidence that does not cover it.
Rollback is a traffic shift too
Rollback gets planned as a single sentence, move traffic back, and then discovered to be a project of its own. Write it as a procedure with the same care as the forward move: which lever changes, in what order, what happens to sessions started on the new cloud, what happens to data written there, and how long until the old cloud is carrying everything again, measured rather than assumed.
Then rehearse it at the smallest share. A rollback practised at five percent, in daylight, with notes taken, is worth far more than a confident rollback plan that has never run. Many teams find the rehearsal is where the real defects are: a missing route back, a health probe that prevents the old targets rejoining, a dashboard that only shows the new cloud. Finding those at five percent is the entire point of starting small.
A short checklist before the first shift
- One named decision point for the shift, and one named lever for rollback.
- A locality list covering sessions, uploads, long lived connections, jobs, and callbacks, each shared, replicated, or pinned.
- A write and read back test across the rollback boundary.
- Matching health probes, intervals, and promotion criteria in both clouds.
- Logging that reports the delivered traffic split, not only the configured weight.
- Capacity on the new cloud sized for the next step, not only the first.
- A rehearsed rollback at the smallest share, timed end to end.
None of this requires a new platform. It requires treating the space between clouds as part of the release, not as plumbing that will look after itself. Teams that design that space deliberately, with private connectivity, consistent routing, and one place to see both sides of a move, find that five percent really can be five percent, and that promoting it is a decision based on evidence rather than nerve. That is the kind of cross cloud connectivity work described at cloudverg.com, where VergMesh and VergRail are used to keep multi cloud paths consistent enough for moves like this to be routine.
A canary between clouds will never be as simple as a canary inside one cloud. It can, however, be just as calm, provided the traffic decision, the locality rules, and the way back are designed before the first percent moves.
