Local drills, explicit boundaries. The file and SQLite exercises were executed in disposable temporary folders. No cloud resources, production deployment or real account changes were made. Payments and registration remain disabled.
01 / Roll back code. Preserve state.
v1 → v2 → invalid v3 refused → v1
Actual local release-pointer drill| Invalid release | v2 stays selected |
|---|
| Rollback | v1 selected |
|---|
| Fictional learner-state hash | Unchanged |
|---|
Contract/hash checks and atomic same-filesystem pointer replacement. No actual web-app health test, database migration or distributed rollout is claimed.
02 / Restore a snapshot separately
Independent snapshot and current-source controls| Database | Rows | Cents |
|---|
| Restored snapshot | 2 | 70000 |
|---|
| Current lab source | 3 | 75000 |
|---|
Integrity: ok · Snapshot logical hash matches.
One post-snapshot record is absent from the restore. Current source is preserved. A tiny local drill is not a production RTO or an offsite backup strategy.
03 / Redelivery without a double addition
A commits 30000 cents. B fails before commit, leaving 30000. Replaying the supplied deliveries then applies B, recognizes two A deliveries as duplicates and rejects the invalid negative C.
Deduplication marker and total share one SQLite transaction. This does not guarantee exactly-once external email, API, payment or distributed broker effects. Retry policy allows at most three total simulated timeout attempts.
04 / Define the denominator
18 / 20 successful jobs = 90%
Observed sample success · 90%Sample objective: 95% · Allowed failures: 1
Observed failures: 2 · Remaining budget: −1
Nearest-rank p95: 1000 ms · Maximum: 2000 ms
One final outcome per unique fictional job. This is a sample-window calculation, not measured production availability. No monitoring alert is sent.
05 / Plan cost and cleanup
Invented monthly-if-forgotten unit rates| Compute: 720 × 2 cents | 1440 |
|---|
| Storage: 10 × 3 cents | 30 |
|---|
| Egress: 5 × 10 cents | 50 |
|---|
| Total | 1520 cents |
|---|
Example budget: 2000 · Alert: 1600
Two-hour owner review target; no automated expiry.
Fictional rates, not AWS/Aruba pricing. Notifications are not a hard spending cap. No resource was provisioned or torn down.
06 / Record what actually ran
Evidence status, not a production-readiness checklist| Local Python / SQLite drills | Executed and checked |
|---|
| Docker / Linux container | Design reviewed; runtime unavailable |
|---|
| Hosted CI / provider IaC | Example only; unexecuted |
|---|
| Cloud IAM / budget / teardown | Planning only |
|---|
| Live website / accounts / uploads | Untouched |
|---|
Use incident-specific acceptance evidence before declaring recovery. A successful rollback mechanic does not establish that an unrelated failure was fixed.
07 / Make the handover useful
Complete the architecture, backup, IAM, cleanup and incident worksheets. Keep the source controls, state preservation evidence and known recovery gap beside your recommendation. Credit supplied teaching code and identify your own tested extension.
Read all nine lessons · Compare worked solutions