Engines

LakehouseBox is an Apache Iceberg REST catalog plus S3-compatible storage. Your engine talks to both directly. The DuckDB, PyIceberg, Spark and Snowflake recipes below are what lhbox connect --engine … prints (--catalog <name> with several catalogs), with the credential and the bucket filled in, and each carries the date it was verified against the service. The Trino and ClickHouse recipes are written from the catalog's REST contract and are marked not yet verified until a recorded run says otherwise. Catalog: https://catalog.lakehousebox.com. Storage: https://s3.lakehousebox.com.

DuckDB (1.5.5 or newer) — read and write

INSTALL iceberg; LOAD iceberg; INSTALL httpfs; LOAD httpfs;
CREATE SECRET lhbox (TYPE ICEBERG, CLIENT_ID '<client_id>', CLIENT_SECRET '<client_secret>',
                         OAUTH2_SERVER_URI 'https://catalog.lakehousebox.com/v1/oauth/tokens');
CREATE SECRET lhbox_s3 (TYPE S3, KEY_ID '<client_id>', SECRET '<client_secret>',
                            ENDPOINT 's3.lakehousebox.com', URL_STYLE 'path', USE_SSL true, REGION 'us-east-1');
ATTACH '<handle>--<catalog>' AS <catalog_name> (TYPE ICEBERG, ENDPOINT 'https://catalog.lakehousebox.com', SECRET lhbox);
-- '<handle>--<catalog>' is the recipe's warehouse_name (older catalogs keep w-<uuid>)
-- e.g. SELECT * FROM <catalog_name>.<namespace>.<table> LIMIT 10;

The lhbox_s3 secret is not needed to read or write tables through the catalog (vended credentials cover that); it lets read_blob('s3://acme--demo-data/**') and friends inspect the bucket. The recipe attaches the catalog under its own name (catalog_alias in the API answer): demo_data for the default catalog, which the examples below assume.

Importing files: one commit for many Parquet or CSV files

lhbox table import 'data/*.parquet' --namespace demo --name cities                       # a new table (--catalog <name> with several)
lhbox table import more/*.parquet --namespace demo --name cities --mode append            # into it
lhbox table import 'data/*.parquet' --namespace demo --name places --format-version 3     # geometry stays typed
lhbox table import 'exports/*.csv' --namespace demo --name orders                        # CSV, detected by the extension (or --format csv)
lhbox table import 'data/*.parquet' --python ~/venv/bin/python --namespace demo --name cities --full   # that venv's DuckDB; print the SQL

table import runs DuckDB on your machine (the duckdb python module of --python PATH or LHBOX_PYTHON when given, else of the interpreter running lhbox, else the duckdb binary; 1.5.5 or newer) over read_parquet([every file], union_by_name=true), or read_csv when every file ends in .csv or --format csv says so, in one statement, so the whole set is one Iceberg commit; importing files one statement at a time is one commit per file (44 files were 44 commits and 44 waits). Globs expand on your side; s3:// and https:// sources are read by DuckDB's httpfs. The report says rows, files, bytes, seconds, commits (1) and the path taken (--full adds the generated SQL statement): format version 2 (the catalog's default unless changed) is DuckDB's own CREATE TABLE … AS SELECT; format version 3 (--format-version 3, or the catalog's default) is created through POST /v1/tables with the schema DuckDB inferred, geometry columns typed, then filled with one INSERT. Measured 2026-09-21: three Parquet files, 3000 rows, one commit, 3.96 s from a laptop; append 1.21 s.

The same by hand, in any DuckDB session with the recipe pasted:

CREATE SCHEMA IF NOT EXISTS demo_data.demo;                  -- CTAS needs the schema; the API creates it on demand
CREATE TABLE demo_data.demo.cities AS SELECT * FROM read_parquet(['a.parquet', 'b.parquet'], union_by_name=true);
INSERT INTO demo_data.demo.cities SELECT * FROM read_parquet('more-cities.parquet');   -- into a table that exists
COPY demo_data.demo.cities FROM 'more-cities.parquet' (FORMAT PARQUET);
CREATE TABLE demo_data.demo.orders AS SELECT * FROM read_csv(['a.csv', 'b.csv'], union_by_name=true);   -- CSV: the list in ONE statement
-- CREATE OR REPLACE TABLE is not supported on an Iceberg catalog: DROP TABLE demo_data.demo.orders; then CREATE TABLE again

Geometry columns: create the table first, then INSERT

lhbox table create --catalog demo_data --namespace demo --name places --format-version 3     --column id:long:required:identifier --column name:string --column geom:geometry
-- then, in DuckDB with the recipe pasted (INSTALL spatial; LOAD spatial;):
INSERT INTO demo_data.demo.places SELECT id, name, ST_Point(lon, lat) FROM read_csv('places.csv');
SELECT id, ST_AsText(geom) FROM demo_data.demo.places LIMIT 3;   -- POINT (…)

A geometry column needs an Iceberg format-version 3 table, and DuckDB's own CREATE TABLE … AS SELECT ST_Point(…) cannot make one: DuckDB creates a version-2 table whatever the catalog's default_format_version, and the catalog answers the CTAS with a 400 that names no cause. Known limitation; the two-step path is the way: lhbox table create --format-version 3 --column geom:geometry (geometry(EPSG:xxxx) for a CRS) and then a plain INSERT … SELECT from DuckDB, measured at 0.8 s for a small table and read back as POINT. lhbox table import … --format-version 3 does exactly this for a GeoParquet.

Measured against the service: a million rows (18 MB of Parquet) in about 2.3 s from a laptop, three data files, one commit; format-version 2.

GeoParquet: the geo metadata becomes table properties

Until Iceberg v3 geometry is readable by every engine, the geo metadata of an imported GeoParquet lives in the table's properties, and DuckDB spatial can rebuild the GeoParquet from them on export. table import reads the file's geo key (parquet_kv_metadata) and sets, through PATCH /v1/table: geo.encoding (WKB), geo.primary_column, geo.crs (EPSG:xxxx when the PROJJSON carries an authority id, else the PROJJSON text; OGC:CRS84 when the file names none), geo.columns (the columns object as JSON: encoding, geometry_types, bbox -- merged across files), geo.version, and geo.crs.projjson when there was a PROJJSON. Into a format-version 2 table the geometry column is written as WKB binary (what geo.encoding says); into a format-version 3 table it stays a typed geometry(<crs>) column and the properties are the same. lhbox table get shows them; lhbox table set-properties --property geo.crs=EPSG:28992 sets or corrects any of them.

PyIceberg (0.12) — read and write

from pyiceberg.catalog.rest import RestCatalog
catalog = RestCatalog(name='lhbox', uri='https://catalog.lakehousebox.com',
                      warehouse='s3://<handle>--<catalog>/', credential='<client_id>:<client_secret>',
                      **{'s3.endpoint': 'https://s3.lakehousebox.com', 's3.path-style-access': 'true'})
table = catalog.load_table(('<namespace>', '<table>'))

# writing: create from an Arrow schema, then append
import pyarrow.parquet as pq
arrow = pq.read_table('cities.parquet')
tbl = catalog.create_table(('demo', 'cities'), schema=arrow.schema)
tbl.append(arrow)

PyIceberg writes one data file per append and a complete snapshot summary; DuckDB splits large writes into several files. Both are read identically by either engine.

Spark (3.5 + Iceberg 1.11) — read and write, including format-version 3

spark.sql.catalog.<catalog_name>=org.apache.iceberg.spark.SparkCatalog
spark.sql.catalog.<catalog_name>.type=rest
spark.sql.catalog.<catalog_name>.uri=https://catalog.lakehousebox.com
spark.sql.catalog.<catalog_name>.warehouse=s3://<handle>--<catalog>/
spark.sql.catalog.<catalog_name>.credential=<client_id>:<client_secret>
spark.sql.catalog.<catalog_name>.io-impl=org.apache.iceberg.aws.s3.S3FileIO
spark.sql.catalog.<catalog_name>.s3.endpoint=https://s3.lakehousebox.com
spark.sql.catalog.<catalog_name>.s3.path-style-access=true
spark.sql.catalog.<catalog_name>.client.region=us-east-1
spark.sql.catalog.<catalog_name>.header.X-Iceberg-Access-Delegation=vended-credentials

Pass these as --conf flags or in spark-defaults.conf; the catalog is then addressed by its name in Spark SQL (SELECT * FROM demo_data.demo.cities for the catalog demo_data). Jars: iceberg-spark-runtime-3.5_2.12 and iceberg-aws-bundle, both 1.11. Java 17 or newer: the Iceberg 1.11 runtime is compiled for Java 17 (class file 61.0) and dies with UnsupportedClassVersionError on the Java 11 that the apache/spark:3.5 images ship; 1.9.2 was the last Java 11 line. client.region=us-east-1 is the SigV4 region the store accepts, not a placement (the data stays in Nuremberg); without it the AWS SDK looks for a region on the JVM and fails before the first request.

Trino — recipe from the REST contract, not yet verified

# etc/catalog/<catalog_name>.properties
connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=https://catalog.lakehousebox.com
iceberg.rest-catalog.warehouse=s3://<handle>--<catalog>/
iceberg.rest-catalog.security=OAUTH2
iceberg.rest-catalog.oauth2.credential=<client_id>:<client_secret>
iceberg.rest-catalog.oauth2.server-uri=https://catalog.lakehousebox.com/v1/oauth/tokens
iceberg.rest-catalog.vended-credentials-enabled=true
fs.native-s3.enabled=true
s3.endpoint=https://s3.lakehousebox.com
s3.region=us-east-1
s3.path-style-access=true

Trino's Iceberg connector against a REST catalog with OAuth2 client credentials (iceberg.rest-catalog.oauth2.credential, …oauth2.server-uri) and vended credentials for the data path; s3.region=us-east-1 is the SigV4 region the store accepts. Not yet verified against the service: written from the catalog's contract (the token endpoint, the s3://<bucket>/ warehouse, path-style S3) and Trino's documented properties, no recorded run. If your run works or fails, tell us and this line changes.

ClickHouse — read recipe from the REST contract, not yet verified

SET allow_experimental_database_iceberg = 1;
CREATE DATABASE <catalog_name>
ENGINE = DataLakeCatalog('https://catalog.lakehousebox.com', '<client_id>', '<client_secret>')
SETTINGS catalog_type = 'rest',
         warehouse = 's3://<handle>--<catalog>/',
         catalog_credential = '<client_id>:<client_secret>',
         oauth_server_uri = 'https://catalog.lakehousebox.com/v1/oauth/tokens',
         storage_endpoint = 'https://s3.lakehousebox.com/<handle>--<catalog>';
SHOW TABLES FROM <catalog_name>;
SELECT count(*) FROM <catalog_name>.`<namespace>.<table>`;   -- one namespace level: backticks around namespace.table

ClickHouse's DataLakeCatalog database engine reads Iceberg tables through a REST catalog; it is experimental in ClickHouse (allow_experimental_database_iceberg) and reads only. ClickHouse has one namespace level, so a table is addressed as `<namespace>.<table>` in backticks. Not yet verified against the service: written from the catalog's contract and ClickHouse's documented settings, no recorded run.

Snowflake — data path not yet

The catalog integration connects (lhbox connect --engine snowflake prints it with the read-only identity, ALLOW_WRITES = FALSE). The data path is not available yet: Snowflake's S3-compatible external volumes need the endpoint allowed by Snowflake Support per account, and reading data needs work on both sides. On the roadmap; measured 2026-09-19.

Databricks — untested

No run against the service and no recipe. A Databricks cluster with the Iceberg REST catalog libraries should be able to use the Spark settings above; nothing here says that it does until a recorded run exists.

Gotchas, all of them measured

• DuckDB: ATTACH the bare bucket name. ATTACH 'acme--demo-data' is read-write. ATTACH 's3://acme--demo-data/' attaches READ-ONLY and every INSERT fails with "attached in read-only mode". PyIceberg and Spark want the s3://acme--demo-data/ form for warehouse, so copying one recipe's identifier into the other engine produces a false "DuckDB cannot write Iceberg".
• DuckDB 1.5.5 or newer. 1.5.0 writes manifest lists the catalog's maintenance cannot read, so tables it wrote were neither compacted nor expired by the scheduler. 1.5.5 writes the spec-compliant schema.
• Format version 2 by default; DuckDB 1.5.5+ also writes version 3. Every table has its own format version; lhbox table create without --format-version takes the catalog's default_format_version (2 unless you set it: lhbox catalog create <name> --default-format-version 3 or catalog update <name> --default-format-version 3; shown in catalog list and in the recipe's format_version_hint). An engine that creates a table itself (DuckDB CREATE TABLE … AS) chooses its own version, v2 today, and is never upgraded silently. Version 3 is needed for geometry/geography; it is written by DuckDB 1.5.5 or newer and by Spark 3.5 + Iceberg 1.11 (verified 2026-09-20 with a geometry(EPSG:28992) column loaded by a plain INSERT, CRS kept in the Parquet logical type). PyIceberg 0.12 reads v3 but cannot write it, and cannot load a geometry column with a non-default CRS. Who writes and who only reads each version: GET /v1/config/formats (no auth), echoed as writers/readers by table create and table get.
• The catalog only accepts its own tokens. Engines mint them from the catalog credential (catalog_credential in the recipe) at https://catalog.lakehousebox.com/v1/oauth/tokens (client_credentials; client_id = access key, client_secret = secret key) and refresh them themselves; they expire after 900 s. LakehouseBox API keys (al_live_…) and tokens from POST /v1/tokens are 401 at the catalog by design.
• PyIceberg asks for vended credentials on every request. That is the intended path: the catalog returns per-table storage credentials which override any static s3.* key you pass. If a cross-bucket copy fails with ACCESS_DENIED, pass "header.X-Iceberg-Access-Delegation": "none" to use your own keys instead.
• Table buckets accept only Iceberg files. Data files must be .parquet/.orc/.avro/.lance and metadata must look like Iceberg metadata, under <namespace>/<table>/(data|metadata)/; anything else is refused on write. Put photos, documents and other blobs in the catalog's blob bucket <handle>--<catalog>--blobs (same credential, S3 path-style, region us-east-1; lhbox connect prints a boto3 example) and index them in a table.
• CTAS makes every column optional. CREATE TABLE … AS SELECT from DuckDB cannot express identifier fields, required columns or partitioning. When you need them, lhbox table create --column name:type[:required][:identifier] first, then INSERT.
• Maintenance is on and not disableable. Compaction (128 MB target files), snapshot expiry (20 snapshots, 7 days) and orphan cleanup run on every catalog. A client that hard-codes vN.metadata.json file names will 404 after a maintenance commit; list the prefix or go through the catalog.
• Commits are optimistic: retry on 409. Iceberg writers commit against the table's current snapshot; when another writer or the platform's maintenance committed in between, the catalog answers 409 and DuckDB raises CommitFailedException … branch "main" has changed. DuckDB does not retry. An unattended job should catch that error, wait a few seconds and re-run the statement (measured 2026-09-20: two concurrent appenders with a client-side retry landed every row, no duplicates). Maintenance can commit into a table within seconds of its creation; a short blackout of the catalog after such a commit was also observed and is being fixed.
• CREATE OR REPLACE TABLE is not supported. DuckDB's iceberg extension refuses CREATE OR REPLACE TABLE on an attached Iceberg catalog. To redo a table, DROP TABLE demo_data.demo.cities; then CREATE TABLE … again (two commits, the second into a fresh table).
• TIMESTAMP WITH TIME ZONE values need pytz in Python. now() and every TIMESTAMP WITH TIME ZONE column come back through the Python client (duckdb.sql(…).fetchall(), .df()) only when the pytz module is installed, which a bare venv lacks: pip install pytz. Or print inside DuckDB with .show(), cast (now()::VARCHAR), or leave the column out of the SELECT. The duckdb binary and lhbox duckdb are not affected.
• Rotation. lhbox catalog rotate <name> replaces both catalog credentials: the old read/write key and every catalog token from it are refused at once; the old read-only key is deleted in the background right after the answer (about 10 s; warehouse.rotate_complete in the audit log records it); storage sessions already vended run out within 900 s. Fetch a fresh recipe afterwards.

Direct S3 access from a script

lhbox credentials --catalog demo_data --namespace demo --table cities (POST /v1/credentials) returns storage credentials scoped to that table's object prefix, with their expiry, for a boto3 or aws-cli session that needs the files themselves rather than the table.