SC 016 — Data Repair Work
Identifies eight distinct ways data projects break and argues that understanding failure modes enables better maintenance strategies. Key solutions: deprecation processes for metric definition chan…
🌀 Hurricane Idalia passed by this week. My family left town, but the initial report is everyone is safe and our home is still standing. My thoughts and prayers are with those that were more severely impacted 🙏
Data Repair Work
Data projects / programs / products - whatever the name - are notoriously difficult to maintain. It is difficult because data can break in so many different ways.
In tackling this complexity I always find it helpful to enumerate the possibilities. I thought of 8 ways a data project could break.
Business
- Strategic — a shift in direction
- Definition — a misunderstood metric
Engineering
- Migration — a new source
- Schema — a new column
Orchestration
- Refresh — a stale dataset
- Grain — an unexpected fan out
Value
- Numeric — a nonsensical number
- Categorical — an unmapped value
Once you have the possibilites mapped out you can begin matching them up with a response plan. For example:
- A metric definition update could be put through a deprecation process. The new definition lives along side an old definition for a period of time to help stakeholder adjust to the impact of the change.
- With schema changes, you may want to be alerted, but not automatically sync new changes - otherwise you could end up with a surprisingly large Fivetran bill.
- It is easy to get out in front of unexpected granularity changes by defining surrogate keys and adding “uniqueness” & “not null” tests to the new column.
By having these response plans in place, and even some proactive measures, you will add stability to your data project. With stability comes trust from the business. Then with trust you will be given new opportunities to add value!
What other ways do you see data projects break?
What methods do you have in place to respond to a break?