Measurement
What we measure while the traffic moves
A cutover is a measurement window. These are the numbers we keep open on a screen, the thresholds that stop the move, and the checks that close it.
The four numbers on the screen during the switch
| Signal | Where it comes from | Threshold during the switch | Action if it trips |
|---|---|---|---|
| Error rate | Request counters on the edge, per second | Above 1 % for 60 seconds | Switch the record back and stop the window |
| Replication lag | Lag metric on the target, in seconds | Above 30 seconds for 2 minutes | Hold the switch, let the delta drain |
| p95 response time | Request duration histogram | Above 2x the pre-switch value | Check the cache and the database pool before continuing |
| Successful checkouts | Synthetic order every 5 minutes | One failed synthetic order | Roll back, investigate before the next attempt |
Closing the window
We close the window when the old target has been idle for a full backup cycle and the new target has produced one complete set of scheduled outputs. Until then we keep the old target writable and its database attached. A cutover declared finished on the strength of a 200 response is usually declared one hour too early.