# Importing files Turn Parquet or CSV files into an Iceberg table with one command. lhbox table import reads every file you name in one statement, so the whole set lands as one commit, however many files there are. Importing files one statement at a time costs one commit per file: 44 files were 44 commits and 44 waits. The import runs DuckDB on your machine, reads the files there, and writes the table in your catalog with your catalog's credential, which never appears on screen. A person can do the same from the account page: a catalog's Tables section has an Import files button that previews the files and writes the table from the browser tab. Both paths write the table the same way: large files, rows sorted by a date column, and a GeoParquet file's metadata kept as table properties. ## A first import, end to end You need DuckDB 1.5.5 or newer on the machine that holds the files: the duckdb Python module of the interpreter running lhbox, or the duckdb binary on your PATH. lhbox doctor shows which one it found. 1. Import the files as a new table. Name the table as catalog.namespace.table (or namespace.table with --catalog, or with no catalog at all when you have only one). The namespace is created if it does not exist. Quote globs so that lhbox expands them, not your shell: ``` lhbox table import analytics.sensors.readings 'data/2026/*.parquet' ``` 2. Add more files later with --mode append. This is again one commit: ``` lhbox table import analytics.sensors.readings 'data/2026-10/*.parquet' --mode append ``` 3. Check what landed. lhbox table get shows the schema, the format version, the snapshots and the row count: ``` lhbox table get analytics.sensors.readings ``` The import prints one summary line for a person, and JSON when its output is piped or with lhbox --json. The report includes rows, files, bytes, seconds, commits (counted from the table's snapshots before and after, not assumed), the format_version, the order_by column and the path it took. --full adds the generated SQL statements. A recorded run from a laptop on 2026-09-21 imported three Parquet files (3,000 rows) as one commit in 3.96 s, and appended them again in 1.21 s. ## What you can import | Files | How they are read | |---|---| | Parquet: .parquet, .parq, .pq | DuckDB read_parquet, columns matched by name across files | | CSV: .csv, .tsv, also compressed as .gz, .zst, .bz2 | DuckDB read_csv, columns matched by name across files | | Anything else | --format csv or --format parquet says what it is | - One format per import. A list that mixes CSV and Parquet stops with mixed_formats. Import each format as its own run, and use --mode append for the second one. A file whose extension is not recognised stops with unknown_format unless you pass --format. - Sources can be local files, globs (** works: 'data/**/*.parquet'), or URLs that DuckDB reads directly (s3://…, https://…). A pattern that matches nothing is reported; if nothing matches at all, the command stops before anything is written. - CSV has no options of its own. DuckDB detects the delimiter, the header and the column types over the whole list. If it guesses wrong, load the file yourself with read_csv(…) and its options (other formats, below). ## Creating or appending - --mode create (the default) makes a new table. If the table already exists, the import stops with table_exists (exit 2) and suggests --mode append. - --mode append writes into a table that exists. Columns are matched by name; a column the table has and the files lack is written as NULL. A column the files have and the table lacks stops the import (columns_not_in_table) before anything is written. - Importing needs a write credential on the catalog. With a read-only one the import stops with exit 3. - Every column of a table the import creates is optional. For required columns, identifier fields or a hand-written schema, create the table first with lhbox table create (CLI reference (https://lakehousebox.com/docs/cli/)), then import with --mode append. - If DuckDB reports an error during the commit, the import reads the table back before answering. Rows that did land are reported as landed. If it cannot tell, it answers import_outcome_unknown (exit 5) with the lhbox table get to run. Do not blindly re-run an append: a second run writes every row twice. ## Format version A new table gets the catalog's default format version (2 unless it was changed with lhbox catalog update --default-format-version 3). --format-version 2|3 overrides it for one import; it applies only to --mode create. - Version 2 is read by every engine the Engines page has verified, and written by DuckDB, PyIceberg, Spark, Trino and Polars. A geometry column is stored as WKB in a binary column. - Version 3 keeps geometry columns typed (geometry, or geometry(EPSG:xxxx)). DuckDB 1.5.5+ and Spark 3.5 with Iceberg 1.11 write it; PyIceberg 0.12 reads it but cannot write it. See Engines (https://lakehousebox.com/docs/engines/). ## Sorted rows and large files Every table the import writes gets the table property write.target-file-size-bytes = 134217728 (128 MiB) before its first row, so DuckDB writes a few large files instead of many small ones. An append into a table that already has its own value keeps it. The rows are written ORDER BY one column. By default, this is the first DATE or TIMESTAMP column in your files, so a query that filters on that column can skip whole files. - --order-by names another column: choose the one your queries filter on most. A column the files do not have stops the import before anything is written. - --no-order keeps the files' own row order. For why this matters, how much it saves, and how to load very large tables in chunks, see How tables are laid out (https://lakehousebox.com/docs/performance/). ## GeoParquet and geometry columns When a Parquet file carries GeoParquet metadata, the import keeps it as table properties: geo.encoding, geo.primary_column, geo.crs, geo.columns (with the bounding boxes merged across files), geo.version, and geo.crs.projjson when the file had a PROJJSON CRS. geo.crs is EPSG:xxxx when the file's CRS names an authority code, and OGC:CRS84 when the file names no CRS. The properties are set on version-2 and version-3 tables alike; lhbox table get shows them, and lhbox table set-properties corrects one (--property geo.crs=EPSG:28992). - Into a version-2 table the geometry is WKB: read it back with ST_GeomFromWKB(geom) in DuckDB spatial, or any WKB reader. - Into a version-3 table it stays a typed geometry column with its CRS: ``` lhbox table import analytics.geo.parcels 'parcels/*.parquet' --format-version 3 ``` A version-3 import needs a CRS with an authority code (such as EPSG:25830). A file whose CRS has none stops with geometry_crs_without_authority before the table is created. Import it with --format-version 2 (the full PROJJSON is kept in geo.crs.projjson), or re-tag the file's CRS with its code first. ## From the account page Open a catalog, go to Tables and choose Import files. The dialog has three steps: - Choose files: files on your computer (Parquet, CSV, or JSON: .json, .jsonl, .ndjson), or files in S3-compatible cloud storage. For cloud storage, the key you type stays in the tab and is never sent to LakehouseBox, and the bucket's CORS must allow the site to read. - Preview: the columns, their DuckDB and Iceberg types, and the first rows. For CSV and JSON, Ignore errors drops rows that fail to parse. - Destination: namespace, table name, Iceberg format version, and Order rows by. If the table exists, the dialog offers to append. Import into … writes the table from the tab, with your catalog credential, as one commit. Keep the tab open until it finishes. The files travel from your disk through the browser to the store, so a large import is bound by your upload speed. The browser engine is single-threaded with a 4 GB address space, so large sets of files are better imported with the CLI. Under Other ways to run this import, the dialog gives the same import as an lhbox table import command, as DuckDB statements, and as a prompt for your agent (with no credential in it). ## Other formats and your own SQL For JSON, other formats, or a transformation on the way in, use DuckDB with the catalog attached. lhbox duckdb opens a DuckDB shell with your catalog attached under its name, and never shows the credential. Write the whole file list in one statement, so it is one commit: ``` -- in `lhbox duckdb --catalog analytics` CREATE SCHEMA IF NOT EXISTS analytics.events; CREATE TABLE analytics.events.clicks AS SELECT * FROM read_json(['day1.jsonl', 'day2.jsonl']); INSERT INTO analytics.events.clicks SELECT * FROM read_json('day3.jsonl'); ``` - A table DuckDB creates itself is format version 2, whatever the catalog's default. It does not get the 128 MiB file size or the sort order: How tables are laid out (https://lakehousebox.com/docs/performance/) shows the CALL set_iceberg_table_properties and the ordered INSERT to add. - CREATE OR REPLACE TABLE is not supported on an Iceberg catalog: DROP TABLE, then CREATE TABLE again. - For a geometry column, create a version-3 table first, then insert into it: lhbox table create analytics.geo.places --format-version 3 --column id:long --column geom:geometry, then INSERT INTO analytics.geo.places SELECT id, ST_Point(lon, lat) FROM read_csv('places.csv'); with the spatial extension loaded. DuckDB's own CREATE TABLE … AS SELECT cannot make a geometry table. The recipes for PyIceberg, Spark and the other engines are on the Engines (https://lakehousebox.com/docs/engines/) page. --- HTML version: https://lakehousebox.com/docs/import/ · every page: https://lakehousebox.com/llms.txt · everything in one file: https://lakehousebox.com/llms-full.txt