How old can your data be when a decision uses it?
The right sync frequency between two systems is the one that fits the decision the data feeds, not the fastest one the technology allows. Every piece of information ages at its own pace. The architecture question is how old it can be when someone acts on it.
Why is "real time" not the default answer?
Because the right pace depends on the decision, not on the tool. Richard Wang and Diane Strong asked data consumers what quality means to them. The research started from 179 attributes gathered from 25 business data consumers and 112 MBA students with work experience, went through a survey with 355 valid responses, reached 20 dimensions, and ended with 15 grouped into four categories.
Timeliness landed in the contextual category, the one that can only be judged against the task at hand. The paper, published in the Journal of Management Information Systems, defines it as the extent to which the age of the data is appropriate for the task, and adopts the view that quality data is data that serves the people who use it.
That undoes the single rule. The problem is not a synchronization technology problem. It is an architecture problem: deciding the age of each piece of data. An inventory balance checked while promising a delivery date and an average cost read at month-end do not have the same requirement. Syncing both every minute takes build effort and monitoring effort, and only one of them benefits. That part is our reading, not the study's.
How fast does each piece of data age?
At different speeds, and you can estimate it per attribute. Heinrich, Kaiser and Klier proposed, at the ICIS conference in 2007, a metric in which the timeliness of a value is the probability that it is still valid. That probability falls exponentially with age, at a rate specific to each attribute. In the case the authors analyzed, a customer's first name has a rate of zero and the contracted tariff has 0.4 per year. At the same age, the two have very different validity.
The case is a campaign by a German mobile carrier, with anonymized figures. In an earlier campaign, aimed at 82,000 customers, the average success rate was about 8.5%. Among customers with estimated timeliness of 0.2 to 0.3, it was 4.4%. Among those at 0.9 to 1, it was 12.6%. By the authors' estimate, targeting only customers above 0.4 would raise the next campaign's profitability by 30 percentage points. The explanation is simple: a customer with an outdated address cannot even accept the offer.
The limit is this: the study covers a marketing campaign in Germany, and timeliness there is an estimated probability, not an age measured in hours. It shows that data age has a measurable effect on results and that the aging rate varies by attribute. It is not a benchmark for your process.
How old is the data when loads run periodically?
Between zero and the load interval, and on average at most half of it. Cho and Garcia-Molina, of Stanford, define the age of a copied item as zero when the copy is up to date and, otherwise, the time since the original changed. Applied to a fixed-interval sync T, a change made right after a load only arrives on the next one. The oldest change still missing from the copy is between 0 and T old, and the average is at most T/2 if reads are spread across the interval. That arithmetic is ours, not the paper's.
The effect adds up when systems are chained. If system A feeds B every hour and B feeds the dashboard once a night, the possible lag reaches 25 hours, plus the time each load takes to run. Whoever reads the dashboard just before the next load sees a snapshot that can be more than a day old, with nothing on the screen saying so.
Two dashboards can also disagree because of definitions, the subject of Management dashboards: why the numbers don't match. Here the disagreement is about time: the two sources are read at different ages, and nobody marked the age of each.
Does every piece of data need the same cadence?
No, and the idea of syncing more often whatever changes most does not always hold. The Cho and Garcia-Molina paper, dated 1999 and presented at SIGMOD 2000, deals with a related problem: with a limited budget of updates, how many to spend on each item in a copy. The authors' setting is copies of web pages, so the lesson works as a principle, not as a recipe.
The most counterintuitive result is that refreshing each item in proportion to how much it changes did worse than refreshing all of them at the same rate. In one of their examples, an item that changes nine times a day gains little from each refresh, because it goes stale soon after, while one that changes once a day gains more. In an estimate with real web data, which the authors themselves call crude, the proportional policy would leave on average 12% of items up to date, against 57% for the uniform one.
Our reading for an operation is another layer: the pace should not come only from how much the data changes, but from what the decision loses when it is old. Data that changes constantly and that nobody decides on within minutes can wait. Data that changes rarely but blocks an order when it is wrong deserves special attention.
A practical example: the stock balance the CRM cannot see
The scenario below is hypothetical. The numbers are illustrative assumptions, not measurements. A distributor with 60 people uses an ERP for stock and cost, a CRM for quotes and a spreadsheet for logistics. A nightly integration copies data from the ERP to the CRM at two in the morning.
At two in the afternoon, a salesperson promises a customer 40 units based on the two a.m. balance. By noon, another order had already consumed 30. The problem is not the integration, which runs as designed. It is treating three different pieces of data as if they had the same age.
The stock balance feeds a promise made on the spot, so the acceptable age is minutes, or a direct query to the ERP at the moment of promising. The customer's credit status changes a few times a month but blocks an order when it does, so a daily load works, as long as a block triggers an immediate update. Average cost per product is read at close, and a load at the end of the cycle is enough. Three pieces of data, three maximum ages. A single nightly integration serves cost well, serves credit only with an extra trigger, and fails on the one that matters most at the moment of sale.
How do you set the maximum age of each piece of data?
Start with a short inventory: the data that moves from one system to another and feeds a decision, five to ten to begin with. For each one, write who decides with it, at what moment, and what happens if it is old. The maximum acceptable age comes out of that answer, noted next to the data, in minutes or hours. Only then do you choose the sync mechanism.
Next, make the screen state the age. A field with the time of the last load gives the user what Wang and Strong place at the center of quality: the age of the data against the task. Finally, monitor the real delay against the agreed maximum. A sync that stopped without warning is the failure covered in Your automation stopped. When will you find out?.
In the Reatop case, hospital waste collectors work in areas without stable signal, and the app records weigh-ins offline and syncs the data automatically when it detects a connection. The cadence there came from the context of use: the data is born where there is no network and arrives when there is.
Frequently asked questions
Implementation will take too much of the team's time. Is that so?
An inventory of five to ten data items fits in one or two short conversations with the people who decide with them. That sizing is our design recommendation, not a research figure. The build work, if any, is limited to the data where age matters, which avoids rewriting integrations that already work.
I do not see how to measure the result. What should I measure?
Measure the age of the data at the moment of the decision and how many decisions were redone because of old data, such as an order promised and not fulfilled. Record both numbers before changing anything. The Heinrich study shows that the success rate varied with estimated timeliness, but it is not a benchmark for your process, so your own baseline matters more.
I worry this will not remove the real bottlenecks. Would real time solve it?
Not always. If the wait is in an approval or in a human review queue, syncing faster does not touch it. Before changing the cadence, confirm that the delay is in the data and not in the decision. When it is in the data, the maximum age per item tells you where to invest.
The next step
If your dashboard and your ERP show numbers of different ages without saying so, the Operational Architecture Diagnostic, a 30-minute conversation, is an invitation to map which data crosses your systems and how old each piece can be: talk to Yowpi.
Related case studies
Did you recognize your operation in this article?
Book the Operational Architecture Diagnostic: 30 minutes to map where your operation's bottleneck is. No proposal in the first conversation.
Book a diagnostic