SC 016 — Data Repair Work
Identifies eight distinct ways data projects break and argues that understanding failure modes enables better maintenance strategies. Key solutions: deprecation processes for metric definition chan…
Summary
Identifies eight distinct ways data projects break and argues that understanding failure modes enables better maintenance strategies. Key solutions: deprecation processes for metric definition changes (old and new coexist temporarily), alerts without automatic syncing for schema changes (avoids cost surprises), and surrogate keys with uniqueness/null tests for granularity issues.
Core thesis: stability builds trust, and trust creates opportunities for value creation.
Key Arguments
- Data maintenance is undervalued but foundational — you can’t create value on an unstable platform
- Metric definition changes need deprecation windows, not hard cuts
- Schema change alerts should inform, not auto-remediate (cost control)
- Surrogate keys with proactive testing prevent granularity issues before they cascade
- Stability → trust → opportunity is the progression
Writing Style Notes
Published during Hurricane Idalia evacuation — the personal stakes contrast with the methodical technical content. Practical and specific. The “repair work” framing elevates maintenance from chore to craft.
Connections
- index — part of the Sanity Check body of work
- 2026 04 03 data maturity processes tools — maintenance practices as maturity indicator
- 2026 04 03 downfall of data engineer — pipeline constipation as a failure mode this article addresses
- 2026 04 03 embrace the grind — data repair work IS the grind; stability as competitive moat
- analytics as craft — elevating maintenance from chore to craft