intellimetrics Learning
Building Trustworthy Data Products Module 6 · Own It

Service Levels Somebody Answers For

Last reviewed · content updated

Intermediate

What you'll learn

~18 min
  • Set freshness and quality targets derived from the decision, not from ambition
  • Route an alert to a person who can act, at a time they can act
  • Decide what the product does when it cannot meet its own service level

Who finds out first, revisited

Lesson 3.4 ranked the outcomes when a load fails, and the worst was silent staleness. This lesson is about making the best outcome the default once the product is in service and you have moved on to something else.

The test is simple and unforgiving: when this breaks at 4am on a Monday, does a person who can fix it find out before the supervisor does? If the answer is no, you do not have a service level. You have a hope, and consumers are your monitoring.

Prompt first: derive the service level

Here is my data product and its consumer: [paste definition,
consumer, decision cadence].


Draft a service level with: freshness target derived from the
decision cadence (state the derivation), quality gates, on-miss
behavior, notification routing with timing, and an annual miss
budget.


For notification, distinguish which failures justify waking someone
from which belong in a daily digest, and tie that to the decision
cadence rather than to how severe the fault sounds.

That final instruction is the one worth keeping. Asked to route alerts, an agent will rank by technical severity — and a total load failure on a Wednesday is technically severe and operationally irrelevant until Sunday night.

Stop and escalate when the notification needs a rotation that does not exist — deciding who is woken at 4am is a leadership call to request explicitly, not something to improvise in a config file.

KNOWLEDGE CHECK

Your Monday load fails at 4am. The pipeline serves the previous month's data with an unchanged data_as_of and a staleness banner. The supervisor runs his 7am meeting from it. Did the service level work?

Derive the target from the decision

The reflex is to state something ambitious — 99.9% availability, hourly freshness. Both are meaningless here, because nobody is making an hourly decision.

Meridian’s real target falls out of the decision itself:

SERVICE LEVEL - circuit_monthly


FRESHNESS     previous month available by the 2nd at 06:00
              America/Chicago
WHY           crew assignment happens Monday 07:00; the first Monday
              can fall on the 2nd
QUALITY       all blocking assertions pass; reconciliation within 2%
              or variance explained before publication
ON MISS       consumers see the previous period with data_as_of
              unchanged, plus a banner. Stale data is never presented
              as current.
NOTIFY        owner by message within 15 minutes of a failed run;
              consumer by 05:00 if it will not be met
BUDGET        two missed months per year before the design is
              reconsidered

Two lines carry most of the weight. WHY ties the number to the decision, so the next person can tell whether it is still right when the meeting moves. BUDGET states in advance how much unreliability is tolerable — without it, every miss is either a crisis or shrugged off, depending on who is in the room.

Alerts that reach someone who can act

An alert that fires into a channel nobody watches is worse than none: it produces a record that the system detected the problem and creates the belief that detection exists.

Three requirements:

  • A person, not a channel. “The data team” is not a recipient at 4am. A name, or a rotation.
  • Actionable content. “circuit_monthly load failed: source freshness assertion, newest event 2026-07-30, expected 2026-08-01” tells someone what to look at. “Job failed” means opening a laptop to learn anything.
  • Timed to the decision. A failure at 4am for a 7am meeting needs waking someone. The same failure on a Wednesday can wait for business hours. Same product, different urgency, and the difference is the decision cadence rather than the severity of the fault.
⚠The alert that trains people to ignore alerts

This is Lesson 4.2’s block-or-warn problem in its operational form. A product alerting on every warning teaches its owner to swipe the notification away, and the swipe becomes reflexive well before the one that mattered arrives.

Page on blocking failures. Everything else goes to a daily digest nobody has to react to at 4am.

What it does when it cannot meet the level

Decide this once, in advance, and implement it — otherwise the behavior is whatever the code happens to do, which is usually the worst option.

Preferred: serve the last good period with data_as_of unchanged and a visible staleness marker. The consumer sees real data, correctly labeled as not current, and can decide whether it is good enough for this week’s assignment. Frequently it is.

Acceptable: serve nothing, with an explanation. Honest, and it forces a decision to be made without data — sometimes correct, often worse than a labeled week-old list.

Never: serve the last good period labeled as current. This is the failure that makes the supervisor assign crews from stale data believing it fresh, and it is the default behavior of a system where nobody made this choice.

Key takeaway

A service level exists when a person who can fix the problem finds out before the consumer does; otherwise your consumers are your monitoring. Derive freshness from the decision cadence and write down the derivation, so the next person can tell when it stops being right. Alert a named person with content they can act on, timed to when the decision happens rather than to how severe the fault sounds, and page only on blocking failures. And decide the on-miss behavior in advance: serve the last good period clearly marked stale, never serve it labeled as current. Lesson 6.2 covers what the product costs.

Search lessons