Agent setup

0. Quick path

You are an agent setting up LakehouseBox for a person. Credentials never appear in the conversation: the CLI saves the key it receives to ~/.config/lhbox/credentials.json and every command reads it from there. LakehouseBox is a managed Apache Iceberg lakehouse operated in Europe: one catalog per organisation to start (an Iceberg REST catalog with its own storage; organisation -> catalog -> namespace -> table), DuckDB as the engine you run locally. The default catalog is named demo_data.

curl -fsSL https://lakehousebox.com/install.sh | sh          # one Python 3 file, no dependencies
lhbox login                                            # prints a link and a code; the person approves once in the browser
lhbox doctor                                           # PASS/FAIL per check; exit 1 on FAIL
pip install 'duckdb>=1.5.5'                                # writes need 1.5.5 or newer; doctor names which interpreter to upgrade
lhbox duckdb                                           # the DuckDB shell with the catalog attached (the recipe is never on screen)

In DuckDB (the catalog is attached under its own name):

SHOW ALL TABLES;
CREATE SCHEMA IF NOT EXISTS demo_data.raw;
CREATE TABLE demo_data.raw.trips AS SELECT * FROM read_csv('data/*.csv');   -- one statement for all the files = one commit
SELECT count(*) FROM demo_data.raw.trips;
DROP TABLE demo_data.analysis.daily;                                        -- when redoing a table: CREATE OR REPLACE is not supported
CREATE TABLE demo_data.analysis.daily AS SELECT ...;

From Python instead of the shell: lhbox --json connect --engine duckdb returns the recipe; its sql field is the ATTACH block. Files from the CLI in one commit: lhbox table import 'data/*.parquet' --namespace raw --name trips (CSV too, by extension or --format csv).

If login cannot block: lhbox login --no-wait, send the link, later lhbox login --resume. Several catalogs: lhbox catalog list, then --catalog <name> on connect, duckdb and table ….

What to say to the person: "Open this link and approve the connection in your browser" (repeat the code). No account yet: the same page creates one.

Slow operations are normal: catalog create and catalog rotate take about a minute (four identity writes of ~10 s each on the store today) and print progress while they run; a catalog may answer provisioning for a while: retry, do not conclude "cannot write" (section 8).

1. Install the CLI

curl -fsSL https://lakehousebox.com/install.sh | sh       # a single Python 3 file, no dependencies
lhbox --version

Or, if the person prefers a package manager: pipx install lakehousebox or pip install --user lhbox. The CLI needs Python 3.10 or newer, which macOS and every current Linux ship. API: https://api.lakehousebox.com. Site and docs: https://lakehousebox.com. The API calls a catalog a warehouse (/v1/warehouses, warehouse_id); the CLI says catalog and accepts warehouse too.

2. Connect this machine

lhbox login

What happens:

1. The CLI prints a link (https://lakehousebox.com/app/device?user_code=XXXX-XXXX) and the code, opens the browser when there is one, and waits (up to 15 minutes, the code's life; every 60 s it prints still waiting for approval at <url>, <n> min left on stderr). If you cannot block that long, lhbox login --no-wait prints the link and the code and exits 0 at once, keeping the pending login in the credentials file; later, lhbox login --resume polls it once (exit 0 connected, exit 3 still pending, with the minutes left) and lhbox login --resume --wait waits for the decision. The code lives 15 minutes either way.
2. Tell the person: "Open this link and approve the connection in your browser" (repeat the code in case the link does not open). If they have no account yet, the same page offers Create your account: they fill in their name, email and organisation, click the link in the mail, choose a password, and land back on the approval with the code intact. Nothing else is needed from them.
3. The person sees a consent screen: this machine's name, the catalog, and the access asked for (read and write by default; they may lower it to read only). They approve.
4. The CLI receives the key of a new agent of their organisation, named after this machine and holding exactly the approved access, saves it, and prints: Connected to demo_data (read and write) as <machine>. Saved to ~/.config/lhbox/credentials.json.

The key is never shown. Do not ask for it, do not look for it, do not cat the credentials file. Every lhbox command and the connection recipe below read it themselves. If the person wants to see or remove the connection later: their account page, Connections & tokens; revoking it stops this machine at once.

Running lhbox login again on a machine whose saved key still works answers Still connected … and starts nothing new. --level read asks for read only; --name <agent> names the agent; --new forces a fresh authorisation.

If the code expired before the person approved (15 minutes), run lhbox login again (or --no-wait again) and send the new link.

3. Check the path, then get the DuckDB recipe

lhbox doctor                          # API, credential, catalog, recipe and level, a read through the catalog,
                                          # write-policy (the bucket accepts the identity; no object written), the DuckDB here:
                                          # PASS/FAIL/SKIP, exit 1 on FAIL. `--write-probe` adds the real thing: a 0-byte
                                          # .parquet written and deleted as the recipe's identity, the store's error quoted on FAIL
lhbox whoami                          # who you are, the organisation, your level on each catalog, limits (with `enforced`),
                                          # the next step -- a short JSON object when piped; --full for the whole document
lhbox connect --engine duckdb         # a paste-ready DuckDB recipe; the only catalog is the default (--catalog <name> with several)
                                          # `lhbox connect --no-version-check` skips the DuckDB version check (either position works)
lhbox duckdb                          # or: the duckdb shell already attached (the recipe goes in through a private
                                          # init file, never on screen); `lhbox duckdb -- -c "SELECT ..."` runs one statement

doctor fails loudly on the one thing the rest of this page cannot fix (the credential, the grant, the DuckDB version) before you paste anything. Its write-policy line says the bucket accepts the identity and writes nothing; only lhbox doctor --write-probe proves the store takes a write (it PUTs and DELETEs a 0-byte _lhbox/_probe/data/probe.parquet as the recipe's identity and quotes the store's error verbatim when it refuses). The DuckDB line checks the interpreter running lhbox first, then python3 on PATH, then the duckdb binary, and names which one to upgrade and how. connect prints the SQL to attach the catalog: its REST endpoint, the storage endpoint, and a short-lived credential it fetched for you; it warns on stderr when the DuckDB on this machine is older than 1.5.5. Run that SQL in DuckDB (pip install 'duckdb>=1.5.5', or the duckdb binary). Do not copy the credential anywhere else; re-run lhbox connect when it expires. lhbox catalog list shows every catalog's name, id and status when you need to pick one, lhbox catalog get <name> one catalog with its storage figure; if the person needs a new one, lhbox catalog create <name> (about a minute; section 8). When stdout is not a terminal, connect prints a JSON object and the SQL is its sql field; lhbox connect --human prints the SQL alone.

Two ways to keep the credential out of the conversation altogether: lhbox duckdb opens the duckdb binary with the catalog attached (the recipe travels through a 0600 init file that is removed the moment duckdb has read it), and lhbox duckdb --persist writes two persistent DuckDB secrets (lhbox_<alias>, lhbox_<alias>_s3) so that any later session -- the shell, or import duckdb in python -- needs only the one ATTACH line it prints. The trade-off of --persist: the catalog credential then lives on disk in DuckDB's secret directory (~/.duckdb/stored_secrets, 0600), readable by any DuckDB this user runs; lhbox catalog rotate <name> or DROP PERSISTENT SECRET lhbox_<alias> revokes it.

4. Check that DuckDB can read and write

Attach with the bare bucket name the recipe gives (ATTACH '<handle>--<catalog>' AS demo_data (TYPE iceberg, …); always paste the recipe's warehouse_name), never s3://<handle>--<catalog>/ -- the s3:// form attaches read-only in DuckDB and every INSERT then fails with "attached in read-only mode". The recipe attaches the catalog under its own name (demo_data for the default catalog; the examples below use it), so two catalogs paste into one session. Then:

SHOW ALL TABLES;                                             -- one SELECT against the catalog
CREATE SCHEMA IF NOT EXISTS demo_data.scratch;
CREATE TABLE demo_data.scratch.setup_check AS SELECT 1 AS ok, now() AS at;
SELECT * FROM demo_data.scratch.setup_check;                 -- one INSERT read back
-- optional: DROP TABLE demo_data.scratch.setup_check;   if your rules forbid deleting, leave it and tell the person

If the CREATE fails with a 403 or AccessDenied, the connection is read only (the person lowered it, or holds read themselves). Say so; a person with write can grant it from the account page. Two DuckDB gotchas on this check (the full list is section 7): CREATE OR REPLACE TABLE is not supported on an Iceberg catalog, so redo a table with DROP TABLE then CREATE TABLE; and the now() column is a TIMESTAMP WITH TIME ZONE, which fetching into Python needs pytz for (pip install pytz; or .show(), now()::VARCHAR, or leave now() out).

Then the same check from the CLI side, creating a table from files in one commit:

lhbox table import 'data/*.parquet' --namespace scratch --name import_check   # many files, ONE statement, ONE commit
lhbox table get --catalog demo_data --namespace scratch --name import_check   # snapshots: 1, rows: the total

table import runs DuckDB on this machine (the duckdb python module of the interpreter running lhbox if importable, else the duckdb binary; --python PATH or LHBOX_PYTHON names another interpreter, for instance the venv where you installed duckdb>=1.5.5) over read_parquet([every file], union_by_name=true) -- or read_csv for CSV files, detected by extension or forced with --format csv -- and reports rows, files, bytes, seconds, commits (1) and which path it took; --full adds the generated SQL statement. Use table import (or one statement over the whole list) for more than one file: one statement per file is one commit per file (44 files were 44 commits and 44 waits, 2026-09-20).

5. The first task

Do what the person asked. The three common starts:

• Use their files. CSV, by hand in DuckDB with the recipe pasted: one statement over the file list, never one per file.

  CREATE SCHEMA IF NOT EXISTS demo_data.sales;
  CREATE TABLE demo_data.sales.orders AS
      SELECT * FROM read_csv(['orders-2025.csv', 'orders-2026.csv'], union_by_name=true);
  -- to redo it: DROP TABLE demo_data.sales.orders; then CREATE TABLE again (CREATE OR REPLACE is not supported)

  The same from the CLI, CSV or Parquet, one commit for the whole set: lhbox table import 'path/*.csv' --namespace <schema> --name <name> (CSV by extension, or --format csv), lhbox table import 'path/*.parquet' … for Parquet; --mode append adds to a table that exists; a GeoParquet keeps its geo metadata as geo.* table properties. For anything else DuckDB reads (read_json, a database attached next to the catalog), the same one-statement shape. Iceberg tables, partitioning optional. Then answer their question with SQL.

• Try an example dataset: DuckDB reads public Parquet over HTTPS; lhbox table import https://…/file.parquet … loads one into a table; show a few queries.

• Continue an existing project: SHOW ALL TABLES, describe what is there, and pick up from it.

• A producer with no engine (a device, a webhook, a job posting events): lhbox sink create <catalog> --table <ns>.<table> --name <name> --save <file> gives it one URL and a send-only key; it POSTs JSON arrays and the platform commits them on a rolling policy (https://lakehousebox.com/docs/ingest/). The table must exist first (lhbox table create), unpartitioned.

lhbox table list and lhbox table get show tables, schemas, row counts and properties from the API side; lhbox usage shows the free tier (5 GB), the headroom and which limits are enforced -- like whoami, a short JSON object when piped (--full for the API's whole document, --human for tables). lhbox catalog measure <name> measures a catalog's storage now instead of at the hourly sweep. A geometry column needs a format-version 3 table created first (section 7).

6. Reference: flags and environment

• Global flags -- --json, --human, --url, --api-key -- work before or after the subcommand: lhbox --json whoami and lhbox whoami --json are the same command. Per-command flags go after their command. Output is JSON whenever stdout is not a terminal (--json forces it, --human forces tables).

• Catalog names are positional: lhbox catalog create <name>, lhbox catalog get <name>, lhbox catalog rotate <name>, lhbox catalog publish <name> --confirm. --catalog <name> (and the older --name) still work as aliases; a UUID works wherever a name does; the destructive commands (catalog delete, catalog publish) want --confirm (or --yes). Commands that act inside a catalog (connect, duckdb, table …) take --catalog <name> and default to the only catalog.

• Environment:

  variable	meaning	default
  LHBOX_API_KEY	an API key; overrides the credentials file (--api-key overrides both)	none: the file is used
  LHBOX_CREDENTIALS	the path of the credentials file; a new path starts empty, and the legacy ~/.config/agentlake/credentials.json is read only when this is unset	~/.config/lhbox/credentials.json (0600)
  LHBOX_URL	the API (a self-hosted or local stack)	https://api.lakehousebox.com
  LHBOX_PYTHON	the interpreter whose duckdb module table import uses (--python PATH per run)	the interpreter running lhbox, else the duckdb binary

  You never need to read any of these; the CLI does.

• Every command's own flags: lhbox <command> --help, and the CLI page of the docs, https://lakehousebox.com/docs/cli/, rendered from the same --help.

7. Gotchas by engine

DuckDB (1.5.5 or newer to write; reads work on any 1.5)

• ATTACH the bare bucket name from the recipe ('<handle>--<catalog>'; catalogs created before 2026-09-20 keep w-<uuid>, so always paste the recipe's warehouse_name), never s3://…/: the s3:// form attaches read-only and every INSERT fails with "attached in read-only mode".
• 1.5.5 or newer to write. 1.5.0–1.5.4 write manifest lists the catalog's maintenance cannot read, so tables they write are never compacted or expired. doctor, connect and table import check.
• CREATE OR REPLACE TABLE is not supported on an Iceberg catalog: DROP TABLE then CREATE TABLE.
• A CTAS with a geometry column is refused by the catalog today (CREATE TABLE … AS SELECT ST_Point(…): DuckDB creates a format-version 2 table whatever the catalog's default, and the catalog answers a 400 that names no cause). Create the table first, then insert: lhbox table create --catalog demo_data --namespace demo --name places --format-version 3 --column id:long:required:identifier --column geom:geometry, then in DuckDB INSERT INTO demo_data.demo.places SELECT id, ST_Point(lon, lat) FROM … (0.8 s for a small table; reads back as POINT). lhbox table import … --format-version 3 does this for a GeoParquet. The recipe: https://lakehousebox.com/docs/engines/.
• now() and every TIMESTAMP WITH TIME ZONE column need pytz when fetched from Python (duckdb.sql(...).fetchall(), .df()), which a bare venv lacks: pip install pytz; or .show() inside DuckDB, cast (now()::VARCHAR), or leave the column out. The duckdb binary and lhbox duckdb are not affected.
• The first query on a table loads its metadata (~0.5 s); later ones on the same table take ~0.05 s.
• Commits are optimistic: when maintenance or another writer committed in between, the catalog answers 409 and DuckDB raises CommitFailedException … branch "main" has changed without retrying; wait a few seconds and re-run the statement.

PyIceberg (0.12)

• warehouse='s3://<handle>--<catalog>/': the s3:// form is required here (the opposite of DuckDB); copying one engine's identifier into the other produces a false "DuckDB cannot write Iceberg".
• Reads format-version 3, cannot write it, and cannot load a geometry column with a non-default CRS.
• Asks for vended credentials on every request (X-Iceberg-Access-Delegation), which override any static s3.* key you pass; a cross-bucket copy that fails ACCESS_DENIED wants "header.X-Iceberg-Access-Delegation": "none".

Spark (3.5 + Iceberg 1.11)

• Reads and writes, format-version 3 included (verified 2026-09-20 with a geometry(EPSG:28992) column); the s3:// form for warehouse, header.X-Iceberg-Access-Delegation=vended-credentials, client.region=us-east-1 (the SigV4 region the store accepts, not a placement; without it the AWS SDK fails before the first request).
• Java 17 or newer: the Iceberg 1.11 runtime jars are compiled for Java 17 and die with UnsupportedClassVersionError on Java 11 (what the apache/spark:3.5 images ship).
• Trino and ClickHouse recipes exist on the engines page but are not yet verified against the service; Snowflake's data path is not available yet; Databricks is untested. Say so if the person asks.

Formats. Every table has its own Iceberg format version: 2 by default (the catalog's default_format_version, changed with lhbox catalog create <name> --default-format-version 3 or catalog update), 3 for geometry/geography. An engine that creates a table itself (DuckDB CTAS) chooses its own version and is never upgraded silently. Who writes and who only reads each version: GET /v1/config/formats. All of it with the recipes: https://lakehousebox.com/docs/engines/.

8. Slow paths and what they look like

The store performs several identity writes per catalog operation, ~10 s each today, so some commands take a minute; the CLI prints progress while it waits. Measured 2026-09-21 against api.lakehousebox.com:

A catalog may answer provisioning for a while; retry. Creation ends with a write into each of the catalog's buckets. If the store refuses that first write, the catalog is created but its status is provisioning: catalog create says so with retry_after_seconds: 30, GET /v1/connection (and so connect) answers 503 provisioning with Retry-After: 30, and doctor fails the catalog line with the reason. The control plane retries every 30 s for up to 10 minutes and flips the catalog to ready; wait 30 s and run lhbox doctor (or lhbox catalog get <name>, which shows status) again. Do not conclude "cannot write". If the CLI ever reports a timeout, check lhbox catalog list before re-running: the operation usually completed.

9. Reading fast and bulk pulls

One HTTPS stream from a laptop is bounded by the network path, not by the store (measured 2026-09-21 from Madrid: 13 MB/s to Hetzner's own speed test, 39 MB/s to s3.lakehousebox.com with BBR on the host, 468 MB/s from a machine inside Hetzner). Parallel range requests for one file did not add up on that path (12.9 vs 12.8 MB/s) and HTTP/1.1 beat HTTP/2 for one big file (12.8 vs 10.0 MB/s). DuckDB already reads Parquet row groups in parallel; do not add threads or range tricks of your own. If the person needs a bulk pull of a large table, say so and run it from a machine in Europe, best inside Hetzner.

10. Environments and rules

Rules:

• Credentials stay out of the conversation. The CLI saves them; --show-key exists for a person who asks, not for you.
• One connection per machine; revoke from the account page, never by deleting the file (the agent would stay valid).
• Everything is in the open formats it says: Iceberg tables in an S3 bucket the person can take elsewhere.