5. Use logs, latency and service objectives carefully

Objective & prerequisites

Turn safe structured observations into a reproducible operational signal. Prerequisites: Python functions, JSON, basic SQL, Git and the existing software/testing workshops. Extract the full kit and open a terminal in cloud-operations-kit. Python 3.10+ with its standard SQLite module is sufficient for the executed labs; no cloud account, credentials, package installation or network is needed. All data and prices are invented teaching fixtures. The runnable drills use new temporary folders and databases, never the website or hosting. These free companions retain existing course progress/access. Payments and public registration remain disabled.

python3 lab.py
python3 -m unittest discover -s tests -v

Understand the concept

Logs describe events, metrics summarize populations, and traces connect work across components. Choose a service-level indicator (SLI) whose unit and denominator match the user-visible task, then define a service-level objective (SLO) and evaluation window. An error budget is the amount of failure allowed by that objective, not permission to ignore individual incidents. This lab uses one final publication outcome per fictional job, not every retry attempt. Logging each retry as a separate job would change the denominator. Structured fields make aggregation inspectable, but safe logging also requires excluding credentials and private payloads. Percentiles need a declared method and sample size; a percentile of a tiny fixture is not a capacity forecast.

Worked example

Twenty fictional job outcomes include eighteen successes and two failures: 90% success. A 95% sample objective permits one failure in twenty; two observed failures leave a remaining budget of −1. The nearest-rank p95 is the 19th sorted duration, 1000 ms; the maximum is 2000 ms. A closed log schema permits only job_id, stage, status and duration_ms, refusing extra payload fields. No monitoring daemon or alert message is sent, and the fixture represents a declared example window rather than production uptime.

from lab import log_summary,load
r=log_summary(load("job-logs.json"))
assert r["success_rate"]==0.9
assert r["p95_ms_nearest_rank"]==1000
assert r["remaining_failure_budget"]==-1

Try it yourself

Calculate success rate and p95 by hand, inspect the two failures and propose an actionable alert with owner, impact, evidence and next check. In a copy, repeat a job ID and add a token field. Explain why the summarizer should reject both rather than publish a misleading rate or private log. Predict the result first, keep the original fixture, and record the exact command and observed outcome from your practice copy.

Knowledge check

Is a 90% success rate across this fixture equivalent to measuring 90% production availability over a month?

Evidence, mistakes & handover

State whether an activity was executed locally, reviewed as a design, or left unexecuted. A successful local drill does not verify a cloud provider, container, hosted pipeline or production recovery. Keep an owner, acceptance criteria, known failures and rollback/cleanup evidence with the work. The download includes source fixtures, runnable drills/tests, separate exercises and solutions, a runbook, a resource plan, an optional container/CI review pack and a capstone rubric. Credit the teaching baseline and your own contribution; no certification, commercial service or employment guarantee follows.

Authoritative reference

Read the official reference ↗

— primary/official reference for the mechanism. Lesson explanations and fictional cases are original. Provider details can differ; this lesson does not configure or provision an account. No production access, payment, public registration or live deployment is activated.