The Catalog Was Updated. The Data Was Still from 2016.

The catalog could tell me when it had seen a table. That was not the question I needed answered.

In the NYC Taxi sample environment for the Build with DataHub hackathon, the source data included records ending in March 2016. Recent catalog activity could sit next to that old data without anything looking malformed. Both timestamps were valid. They recorded different events.

That difference became the center of unmeasured, my freshness agent for the competition.

First, name the clock

A timestamp called lastObserved sounds useful when someone asks whether data is fresh. But it records an observation by the metadata system. It does not, by itself, establish when the underlying records were produced, arrived, or last changed.

If an agent uses the closest available timestamp, it can produce a perfectly formatted answer about the wrong event.

I wanted the report to put the business declaration beside the measurement. How often is this dataset supposed to update? What time does the selected field in its data actually represent? How far does that measurement sit from the reference time used for this run?

Without those answers, a freshness verdict would be a guess dressed as a calculation.

There is an important limit here. Old data is not automatically defective. A historical taxi dataset may be doing exactly what it is meant to do. The hackathon demonstration attaches an update expectation to the sample so that the agent can test a defined contract. In a real deployment, the owner would need to confirm the cadence and the field used to measure it.

A finding needs more than a number

The first design question was what the agent must refuse to say.

If it cannot measure the input, it should not substitute a number. If the catalog has never declared an update cadence, the agent should not invent one. If a requested target cannot be found, it should not quietly disappear from the summary.

That produced four states with different meanings:

A captured control run includes old sample data, a freshly seeded development control, and a nonexistent target. The point is not that the agent finds something stale. It is that the three inputs produce appropriately different answers, and the missing target remains visible in the coverage report.

The search has a boundary too

Even the list of things to inspect needs checking.

A catalog search can return one page of results while reporting a larger total. An agent that accepts that page as the whole inventory can examine every result it received and still overstate its coverage.

unmeasured follows the pages and compares the collected inventory with the server's declared total. The final report preserves the scope of the scan. “I found nothing wrong among the things I reached” cannot silently become “nothing is wrong.”

This is the same distinction that later mattered in an IvyBloom curriculum review: a resource absent from our mapping was not necessarily absent from the resource itself. The scope of the search belongs in the answer.

Follow the stale value through the pipeline

The sample pipeline spans SQLite and Postgres. A stale source can appear again in a downstream table and again in a summary. Treating every stale table as an independent failure would give an operator a noisy list without explaining the relationships.

The agent walks the declared lineage and distinguishes an upstream stale finding from downstream echoes. In one recorded scan, four stale tables were grouped around one upstream table, while two views remained unmeasured because the required cadence declaration was missing.

That is attribution within the available lineage, not proof of every physical cause of a pipeline failure. Missing or unreadable lineage must remain visible. A table that is stale below a healthy parent also needs its own attention; it should not be folded into an explanation that no longer fits.

The write response was not the finish line

I wanted the catalog to retain what the agent had learned. Otherwise, the next person opening it would know less than the last agent that inspected it.

The agent writes the findings back through a limited vocabulary of tags and structured evidence. When a state changes, it removes its own obsolete state tags instead of accumulating contradictions. Human-authored tags are left alone.

Then it reads the result back.

During development, I reported an upstream issue after different storage views returned different answers about the same metadata update. A success response from a write was not enough to establish what another reader would see. The DataHub issue records the reproduction and the maintainer's response.

I also submitted sample-setup corrections. Those are separate artifacts from the agent itself. Filing a fix and having it merged are different states, just as writing metadata and confirming the result are different states.

The loop therefore ends with a comparison between the intended write and the readback. A mismatch is a reported failure, even if the write request succeeded.

What the demonstration establishes

The repository contains captured runs against the hackathon sample environment, including fresh, stale, unmeasured, and uncovered situations. Those captures show the checks and the writeback behavior for that setup.

They do not establish that every organization's data semantics can be inferred automatically. A field containing event dates is not interchangeable with an ingestion timestamp. Historical backfills, missing profiles, access limits, and incomplete lineage all affect what a deployment can conclude.

I also keep the competition result separate from the engineering record. unmeasured was not awarded. The official results were published on September 10, 2026 and checked for this update on September 26. I have no published judging feedback from which to infer why it was not selected.

The part I want someone to inspect is the pair of values behind each verdict: the declared requirement and the actual measurement, with the clock meanings stated. If either side is unavailable, the report should make that absence easy to see.

An honest freshness report sometimes ends with work still to do: ask the owner what this table promises, select the right time field, or establish that a missing target was actually in scope.

That is why I called the project unmeasured. The missing measurement is part of the result.

Inspect the project, recorded runs, and source contracts.