Eighteen open datasets in Apache Iceberg, ready for DuckDB and agents

Every cloud data warehouse has a public dataset catalog. Snowflake has its Marketplace, BigQuery has its Public Datasets program, and Databricks ships sample tables inside Unity Catalog. They are genuinely useful — as long as your data, your compute and your billing account already live inside that vendor's cloud.

If you are working in DuckDB on a laptop, running an autonomous agent in Claude Code or Codex, or building a pipeline outside a hyperscaler, those public catalogs might as well be behind a wall. To join your sales table with daily weather, exchange rates or gridded population, you are back to hunting down CSVs, ZIP archives and API documentation, writing a parser, and figuring out how to keep the table fresh next week.

We wanted those reference datasets to exist in the open: as public Apache Iceberg tables over plain HTTPS, kept fresh automatically from their upstream sources, and readable from anywhere in a few lines of SQL — with no account and no API key required.

Today LakehouseBox's public catalog (commons/pubdata) carries eighteen open datasets across twenty-three Iceberg tables (about 15 million rows in total). For many of them, this is the first time they have been published as live, queryable Apache Iceberg tables.

What is in the catalog

Every dataset in commons/pubdata is chosen for one job: giving your own tables context in a JOIN. Rather than dumping raw government archives, each recipe normalises join keys (country_code as ISO 3166-1 alpha-2, iso3 as alpha-3, currency as ISO 4217, day as DATE, h3_r4 as Uber H3 resolution-4 cell index) so the tables join cleanly with each other and with your data:

Weather — history and 16-day forecasts: Daily weather (weather.daily and weather.stations, from NOAA GHCN-Daily) gives daily temperature and rainfall for capital stations since 2020; Weather forecasts (16-day) (weather.forecast, from Open-Meteo) adds a rolling 16-day daily forecast and 7-day recent analysis for all 208 world capitals, refreshed every morning.
Population on a grid and by country: Population grid (H3) (population.grid_h3, from Kontur Population) maps 8.03 billion people onto 71,329 populated Uber H3 resolution-4 hexagons (~1,770 km² each) with land area and density; Population & economy (population.world_bank_indicators) tracks twelve World Bank WDI series by country from 1960 to the present.
Exchange rates, macro indicators and company financials: Foreign exchange rates (reference.fx_rates, from the European Central Bank) has daily fixing rates for 41 currencies since 1999 in both rate_per_eur and cross-computed usd_per_unit; Macroeconomic & interest rates (population.macro_rates, from FRED) tracks 11 US, Euro Area and global Brent oil series; US public company financials (population.sec_financials, from the SEC EDGAR XBRL Frames API) covers annual revenue, net income, assets, equity and cash for ~7,000 US-listed companies since 2019.
Places, boundaries and infrastructure: Countries (reference.countries), Countries & regions (places.admin_areas, from Overture Maps), Places of interest in Europe (places.places, 7.25 million shops, schools, hospitals and stations from Overture), Rail network of Europe (transport.rail_segments), Airports (transport.airports, 86,000 airports and heliports), World seaports (transport.seaports, 3,800 ports from the NGA World Port Index with vessel draft limits and facilities), and Power plants (environment.power_plants, 34,900 utility-scale plants from WRI).
Calendars, environment and mobility: Public holidays (calendars.public_holidays, 119 countries across 2020–2027), Earthquakes (environment.earthquakes, every M4.5+ event since 2000 from USGS), Air quality in Europe (environment.air_quality_daily and environment.air_quality_stations, from the European Environment Agency), and NYC yellow taxi trips (transport.nyc_taxi_trips and transport.nyc_taxi_zones, 3.3 million monthly trips from the NYC TLC).

World map of Kontur resident population on Uber H3 resolution-4 hexagons. — Global resident population on 71,329 Uber H3 resolution-4 hexagons (population.grid_h3), drawn directly from the public Iceberg table.

Iceberg v2 for tabular data, Iceberg v3 for native geometry

Each table is published in the Iceberg format version that matches its data:

Twenty tables use Iceberg format version 2. Tabular datasets (fx_rates, macro_rates, weather.forecast, sec_financials, nyc_taxi_trips), H3-indexed cells (population.grid_h3) and point tables (airports, seaports, power_plants, earthquakes) only need standard scalar types, and Iceberg v2 can be read by virtually every Iceberg client in use today.
Three tables use Iceberg format version 3 (places.admin_areas, places.places and transport.rail_segments). Polygon boundaries, rail line segments and 7.25 million European points of interest benefit from Iceberg v3's native geometry type and Parquet's geospatial statistics. We write these tables ordered along a Hilbert space-filling curve so that a spatial bounding-box filter in DuckDB 1.5.5+ skips irrelevant row groups over HTTPS and fetches only the megabytes covering your area of interest.

Every table also maintains a version-hint.text file at its root and is served through our CDN edge. When the daily refresh commits a new snapshot at 04:17 UTC, any reader pointing at the table's root path immediately sees the latest version without needing a catalog token or a pinned metadata filename.

Querying from DuckDB in three lines — no account, no key

Because the storage bucket is public and each table carries version-hint.text, DuckDB 1.5.5 or newer can query any table directly over anonymous HTTPS:

INSTALL iceberg; LOAD iceberg; INSTALL httpfs; LOAD httpfs;
CREATE SECRET pubdata (
  TYPE S3, KEY_ID '', SECRET '',
  ENDPOINT 'lakehousebox-public-data.b-cdn.net',
  URL_STYLE 'path', REGION 'us-east-1'
);

-- Next 7 days of rain and temperature forecast for European capitals, joined with public holidays
SELECT f.country_code, f.capital_city, f.day,
       f.tmax_c, f.precipitation_mm, f.precipitation_probability_pct,
       h.name AS holiday_name
FROM iceberg_scan('s3://commons--pubdata/weather/forecast') f
LEFT JOIN iceberg_scan('s3://commons--pubdata/calendars/public_holidays') h
  ON h.country_code = f.country_code
 AND h.date = f.day
 AND h.subdivision IS NULL
WHERE f.is_forecast
  AND f.day BETWEEN current_date AND current_date + INTERVAL 7 DAY
  AND f.precipitation_mm >= 5.0
ORDER BY f.precipitation_mm DESC
LIMIT 15;

If you have a LakehouseBox account, one CLI command attaches the entire catalog read-only as commons_pubdata alongside your own private tables:

lhbox duckdb --catalog commons/pubdata

Or in the browser, you can open any dataset directly in the in-browser DuckDB SQL explorer without installing anything.

Built so agents don't have to guess

When a human or an AI agent opens an unfamiliar catalog, table names alone are not enough. If you sum world_bank_indicators without filtering NOT is_aggregate, you double-count the world; if you join public_holidays without subdivision IS NULL, a regional holiday looks nationwide; if you call ST_Intersects on a 7.2-million-row Iceberg v3 table without geometry && <box>, you scan every file instead of using the bounding-box index.

We built four layers of metadata into commons/pubdata so an agent always has that context before it writes a query:

Native Iceberg table and column docs: All 23 tables carry a full table comment, and all 277 columns across the catalog carry native Iceberg field doc strings describing units, codes and null semantics.
The catalog as SQL tables (meta.datasets and meta.columns): The catalog describes itself in two Iceberg tables of its own. An agent can ask the catalog which tables share a join key or when each dataset was last refreshed using ordinary SQL:

SELECT dataset_id, title, refresh, row_count, join_keys
FROM iceberg_scan('s3://commons--pubdata/meta/datasets')
WHERE list_contains(join_keys, 'country_code')
ORDER BY row_count DESC;

Catalog AGENTS.md: A single 14 KB Markdown briefing returned automatically when an agent calls open_public_catalog("commons/pubdata") over MCP, with every table, its companion tables, a shared join-key map and the main query caveats.
Markdown data cards (/pubdata/cards/<id>.md): Every page on /public-data has a plain Markdown twin (/public-data.md for the catalog, /pubdata/cards/<id>.md for each dataset) with schema tables, "For agents" notes and ready-to-run DuckDB snippets.

Explore all eighteen datasets at lakehousebox.com/public-data, or query the catalog list directly at /pubdata/datasets.parquet.

Explore how it works