Every analytics team knows this curve. Someone builds a model, a dashboard or a forecast. Over months they learn where it goes wrong and correct it. Then they move to another project, or leave, and what they learned leaves with them. The next person starts again, close to the bottom.
Agents can draw the other line, under one condition. What they learn has to be kept somewhere the next run, the next agent and the next person can read it. In practice that means tables: every decision, what happened afterwards, and how good the decision was. We call this always-learning analytics. It is the reason LakehouseBox is built the way it is, so we tested it on a real problem, with numbers.
The job: Madrid's air, decided by noon
The photo at the top is Madrid in 2017. The brown layer is nitrogen dioxide (NO2), mostly from traffic. That year the European Commission was pursuing Spain over NO2 levels in Madrid and Barcelona.
Madrid has done a lot since then, and it shows. In 2010, 18 of its stations were over the EU's annual NO2 limit. In 2022, for the first time, none was, and none has been since: 2025 was the fourth year in a row, according to the city's own summary. The city's open data, loaded into our catalog, tells the same story day by day.
The job is not finished, though. From 2030 the EU's rules get much stricter: no station should average more than 50 µg/m³ on more than 18 days a year. Madrid had 54 such days in 2025. And the city's protocol for bad days still stands: when pollution is forecast to be high, it must announce traffic restrictions before noon the day before. Someone has to make that call every day, with what is known at that hour.

The city publishes what its stations measure, every hour, as open data. So we gave an agent the city's job. Every day at noon it answers one question: will some station go over 50 tomorrow? It gives a probability, and the next day the stations score it.
How it runs
Everything is in one Apache Iceberg catalog on LakehouseBox:
- What happened. The city's hourly air quality since 2010 (20.8 million readings), its weather since 2019 and its traffic. Scheduled SQL fetches each source and keeps the tables fresh.
- What the agent said. A hosted agent runs next to the data at 12:00 Madrid time and appends its forecast to a table.
- How it did. A scheduled query joins each forecast to what the stations measured the next day. The score is one more table.
Nothing the agent learns lives inside the agent. It is all rows that the agent, another agent or a person can query.
The test: one winter, five kinds of memory
A live agent needs months before its learning shows. So we replayed a past winter, 1 October 2024 to 31 March 2025: 182 days. Each simulated day at noon, the forecasters saw only what was known at that hour. They all used the same model and got the same brief: the last week, this morning's readings, tomorrow's weather and calendar. Only their memory differed.
- No memory. Every day starts from zero.
- Notes. It keeps up to 12 lessons and rewrites them every day after seeing how it did. This is what "agent memory" usually means today.
- Its record, wiped monthly. It queries the table of its own scored forecasts, but the table is emptied on the 1st of every month: the person or the project changes.
- Its record, kept. The same, never emptied.
- Its record plus the city's tables. The same, and also the days of the last three years most like tomorrow, with what happened on them. The organisation's knowledge, from the first day.
What we found
Without memory, the agent makes the same mistake every day. The model knows the meteorology: calm air and cold clear nights build NO2 up. But it applied that too eagerly. It said 49% on average for something that happened on 35% of the days, and it said so all winter. Nothing told it otherwise.
Notes decay. October 2024 was clean, and the notebook learned that one day at a time. By 18 November, 11 of its 12 lessons capped the chance of an exceedance at 20% or less. That day it said 15%. The model with no memory said 90%, and a station went over. Over the winter, two thirds of all the lessons it wrote said the same thing. Notes are a summary of the last few weeks, written as if they were rules, and they are hard to unlearn. It is the human sawtooth again: knowledge kept as prose.
Forgetting costs a week, every time. In the first week of every month, the forecaster whose record had just been wiped did worse than the one that kept it: in all six months, and on average exactly as badly as no memory. By the third week it had caught up.
What compounds is what the organisation keeps. The more knowledge the forecaster could query, the less it got wrong.
The forecaster that read the city's tables did best: 18% less error than no memory, the one difference among these that is clearly larger than the noise of a single winter. It did not need a record of its own to start: in October, its first month, its error was 0.096 against 0.153 with no memory. It was also the only model to match the fixed "almost never" rule on the rarest question we asked, the parking surcharge, which happened twice all winter. Its own record could not teach it something that rare. Three years of the city's history could.
Memory is how an agent calibrates
A forecaster is calibrated when its numbers mean what they say: when it says 70%, it happens about 7 times in 10. A model on its own has no way to know. Ours said "40 to 60%" on 15 days, and it happened on 2.
An agent that reads its record does know. It reads "when you said 60 to 80%, it happened 22 times in 39" and adjusts, the way an experienced analyst learns how far to trust their own numbers. Every forecaster with a memory was better calibrated than the one without.
That is also why the gain from its own record came in weeks and then levelled off. Once the numbers are honest, more record does not make them more so. What kept adding was the organisation's tables. So the compounding is not one forecast getting better forever. It is that nothing starts over.

Why LakehouseBox is built this way
All five forecasters used the same model. What changed the result was what each could keep, and where it lived. That is the part a platform has to provide, and it is what LakehouseBox is: open Apache Iceberg tables where the data and the agents' work sit side by side, agents that run next to that data with a memory made for work that repeats, and pages that read the same tables in the browser.
Models and compute will keep changing. What an organisation has learned should not change with them.
Your agents bring the compute. We keep what they learn.
See it yourself
The live page shows the forecaster at work: today's air, Madrid under the 2030 rules, every forecast it has made, and how each one turned out. The repository has the queries, the agent's prompt and the replay.
LakehouseBox was born in Madrid, and we have a saying here: de Madrid al cielo. From Madrid to heaven. For fifteen years this city has measured its air, made hard decisions and kept at it, and you can see it when you look up. This post is also a thank-you.

