Dữ Liệu Kinh Tế is a product SealCore built and runs itself: a store of Vietnamese macroeconomic data, currently more than 93 tables across 15 indicator groups, updated monthly and free to use. This article covers the hardest part — which is not the interface.
The problem: unstable sources
The figures come from official publishing bodies, but the publishing is not uniform: some release spreadsheets, some release text documents, some change table structure between periods, and indicator names are sometimes written differently. A pipeline hard-coded to column positions breaks by the second release.
So the first design principle was: assume the source will change, and separate reading the source from storing the data, so that a change means editing one small place.
Three layers
- 1Collection — a dedicated reader per source, running on a schedule, keeping the original file untouched. The original is kept so that when something is found wrong, it can be traced back to where.
- 2Normalisation — mapping indicator names to internal standard codes, settling units and reporting periods, and handling the cases where a body revises a previously published figure.
- 3Serving — clean series, ready for the lookup pages and the charting tool.
The second layer takes the most work, exactly as described in Central data warehouses. The indicator-name mapping table is the single most valuable asset in the system.
Automated quality checks
With published data, one wrong figure costs all the credibility. So every update passes a check suite before it is allowed onto the site:
- Does the new value jump abnormally against the historical series? Past the threshold it stops and waits for a human.
- Are the reporting periods continuous, or is one missing?
- Do the internal relationships hold — does the sum of the components equal the published total?
- Have older figures changed since the last collection? If so, it is recorded as an official revision rather than silently overwritten.
Why pages are pre-built rather than queried on request
Each indicator is its own page, generated ahead of time on the server. The figures change once a month, so there is no reason to query a database on every visit. The result: pages open almost instantly, they survive the traffic spike when new figures land, and server costs stay low.
It is also the right decision for search: the content sits in the HTML, with the source and update date stated on each table — which is what both readers and search engines need before trusting a number.
The charting tool: where users stay longest
Looking up one number takes thirty seconds. What brings people back is being able to put indicators side by side: inflation next to retail sales, the exchange rate next to exports, credit next to property.
The technical difficulty is that indicators have different periods — monthly, quarterly, annual — and different units, so they cannot share one axis. The answer: allow multiple axes, suggest a period alignment automatically, and always show the unit next to the series name.
What this means for other data projects
- 1Keep the original of every collection run. Storage is cheap; being unable to trace the cause of an error is very expensive.
- 2Quality checks must be automated and must have the power to block. A check that cannot block is just logging.
- 3State the source and update time next to every figure. It is a small detail and it is the entire difference between a trustworthy data store and a website with a lot of numbers on it.
Want to talk specifics?
SealCore surveys at your premises and sends a fixed quote after the first session — including when the conclusion is that you do not need custom software.
