Revenue seems unusually high. Patient numbers are lower than expected. A dashboard that has been reliable for months suddenly produces a result that nobody can explain.
Sometimes the number is correct. Sometimes it isn’t.
You’ve probably heard the term data swamp before. It’s what people call a data lake that’s become difficult to trust. The infrastructure is there, but confidence in the data slowly disappears because every report raises another question. Data quality testing is one of the simplest ways to prevent that from happening.
Governance and testing
Data governance and data quality testing are often discussed together because they solve two parts of the same problem.
Governance defines what good data looks like. It establishes the expectations the business has for its data, whether that’s requiring every customer to have a name, ensuring appointments always belong to a patient, or defining what a valid revenue figure should look like.
Without governance there’s nothing to measure against. Without testing there’s no way of knowing whether the standards you’ve defined are being maintained.
From expectations to tests
Once those expectations have been defined, they can be turned into automated checks. Frameworks such as dbt Tests, Great Expectations and Deequ all approach this slightly differently, but they’re solving the same problem. They inspect the data, compare it against an expected outcome, and alert you when something falls outside the rules you’ve established.
Some of those rules are technical. Required fields shouldn’t be empty, primary keys should remain unique, and relationships between tables should stay intact. Others come directly from the business. If quarterly revenue suddenly doubles, that may reflect exceptional performance, or it may simply mean duplicate records have been written into the warehouse. The test doesn’t decide which explanation is correct. It simply tells you the result deserves another look.
- Governance
Define what good data looks like.
- Testing
Measure the data against those expectations.
- Monitoring
Investigate anything that falls outside the expected range.
Building better tests
Introducing data quality testing doesn’t need to be a large project. Most teams start with the reporting tables they rely on most, define a handful of expectations, and automate those checks. As the business changes, the tests change with it. Thresholds become more accurate, new edge cases appear, and additional rules are added where they’re needed.
A small number of well-chosen tests will usually provide more value than hundreds of generic checks that nobody pays attention to.
Modern tooling has also made getting started much easier. With frameworks like dbt Tests and tools such as Claude Code, it’s possible to generate a useful first set of tests in a couple of hours and refine them over time.
Most data quality issues aren’t discovered by engineers. They’re usually discovered by someone in the business who notices that a number doesn’t feel right.
By that point, the real question isn’t whether the dashboard is wrong. It’s whether the reporting logic needs attention, the underlying data has changed, or the business itself is behaving differently than expected.
Why it matters
Data quality testing doesn’t prevent every issue, and it isn’t meant to. Its value comes from making problems easier to identify while they’re still small.
When something unexpected appears in a report, teams with good testing already know whether duplicate records appeared, required fields went missing, or key relationships between tables broke. Instead of investigating every stage of the pipeline, they can focus on the part that has changed.
Over time, that’s what keeps a data lake from becoming a data swamp. Not perfect data, but enough confidence in the data that people can spend less time questioning the numbers and more time using them.
If your reports raise more questions than they answer, we are happy to talk through what a first set of data quality tests would look like.