KaspratechInfrastructure Notes Sheet 1620 · /

IT services

Infrastructure a small engineering team can run without a full-time operations hire

Kaspratech builds and maintains the infrastructure that small product teams depend on: the servers they deploy to, the backups they restore from, and the releases they can roll back. We work in three engagements — a recovery baseline, a scheduled cutover, and the measurement layer that proves both worked.

0504010080minutes10080Freeze1440Replicate240Verify60Warm30TTL10Read-only5Switch15Smoke60Unfreeze
Preparation time per cutover step, in minutes. Scheduled replication and checksum verification account for 1440 and 240 minutes of the total; the traffic switch itself needs 5.

Three engagements

Every engagement ends with a number the team can check without us in the room. We do not run monitoring for its own sake and we do not sell hardware; the deliverable of each engagement is a documented, repeatable operation.

Recovery baseline

We set a recovery point objective and a recovery time objective for each workload, then prove both with a restore drill into an isolated scratch instance. Deliverable: a target sheet and a drill log with the measured duration of every run.

Scheduled cutover

We move a service from one host or one region to a replicated target in nine reversible steps, with a read-only window kept under 10 minutes. Deliverable: the run sheet, the cutover clock and a tested rollback path.

Measurement layer

We instrument the signals that decide whether a cutover continues or rolls back: error rate, replication lag, p95 response time and a synthetic checkout every 5 minutes. Deliverable: alert thresholds and a close-out checklist.

What we do not publish

Client names, references, contracts and contact details stay off the page. What we publish is scope, method and measurement, because that is what a team needs to reproduce the result after we leave.

What you can promise before we start

A recovery promise is only worth something if it names a number, a workload and a test that proves it. The table below is the contract we sign up to with a team running PostgreSQL, object storage and CI jobs on its own machines: how much data may be lost, how long systems may be down, how the claim gets tested and how often the test runs.

Recovery point and recovery time targets per workload, with the drill that verifies each one.
WorkloadRPO targetRTO targetHow the target is testedTest frequency
Order database5 min30 minRestore the last full dump into a scratch instance and replay the binlogMonthly
Media and object storage24 h4 hRebuild the bucket from versioned objects and compare object countsWeekly
Configuration and code1 h15 minRebuild the environment from the repository and rerun the smoke testEvery deploy
Admin and reporting tools60 min2 hRestore the analytics snapshot and rerun the overnight jobMonthly

RPO = maximum acceptable data loss. RTO = maximum acceptable downtime. Both are measured in minutes.

The cutover in nine steps

Each step below has a moment in the schedule and a way it fails. Read the failure mode before you rely on the step.

  1. 01

    Freeze schema migrations

    T-7 days

    Engineers migrate against the old primary for a week, so the replication target is never mid-migration.

  2. 02

    Replicate the primary

    T-24 hours

    Continuous replication runs for a full day so lag stays observable instead of surprising.

  3. 03

    Verify row counts and checksums

    T-4 hours

    Compare table row counts and a per-table checksum between source and target.

  4. 04

    Warm the application cache

    T-60 minutes

    Prime the cache so the first requests do not compile cold against the new target.

  5. 05

    Lower the DNS TTL

    T-30 minutes

    Drop the record TTL to 60 seconds and wait two TTL periods before anyone is moved.

  6. 06

    Open the read-only window

    T-10 minutes

    Writes stop on the old primary; readers keep working while the last delta is applied.

  7. 07

    Switch the traffic

    T-0, 5 minutes

    Move the record to the new target and watch the error rate continuously.

  8. 08

    Smoke test the product

    T+15 minutes

    Place one real order, log in, and trigger the background jobs that usually break.

  9. 09

    Restore the TTL and unfreeze

    T+60 minutes

    Raise the TTL to 3600 seconds and allow migrations against the new primary.

What usually breaks, and where

Stale DNS caches

Resolvers keep the old address for as long as the old TTL says. Lower the TTL one full billing cycle before the switch and keep it low until the dust settles.

The session store

Sticky sessions and password-reset tokens still point at the old primary after traffic moves, so users appear logged out for exactly the length of your cache TTL.

Background jobs

Scheduled work often keeps a connection string in a configuration file that nobody changed. Run one job on the new target by hand before the window opens.

Certificates that do not cover the name

A TLS certificate issued for the old hostname fails the moment the record moves. Check the subject alternative names and the expiry date, not the issuer chain.

The job that still writes home

A nightly report configured with the old host name keeps writing to the retired database for weeks. Synthetic writes catch it in the first drill.

Questions a team asks before the engagement

Do we need to move at all?

If one box still carries the product, the note is really about removing that dependency before it fails. The recovery targets are worth setting even if the move happens next year.

How do we prove a rollback works?

Roll back by pointing the record at the old target again and keeping the old primary writable until the next backup cycle completes. A rollback you never rehearsed is a guess.

What if the DNS change takes longer than five minutes?

Resolve traffic away from the old primary by lowering the TTL beforehand. The switch then only has to invalidate what is already cached, and the read-only window can stay short.

Does this apply to a team of three?

The steps apply, the scale does not. A team of three can run the whole cutover inside one working morning; the nine steps still fit on a single sheet.

What do you need from us on day one?

Read access to the target environment, the current primary, a list of the workloads that must stay online, and a named person who can approve a rollback. Without the last one the cutover cannot start.