# Files: what a catalog's file bucket takes Every catalog has two places for data: its tables, and a file bucket for the files around them (the source downloads a table was built from, rasters, tiles, photos from a camera, a data card). LakehouseBox stores data, so the file bucket takes data formats. An upload of any other kind of file is refused with an error that says so; if your data needs a format that is not on the list, write to hello@lakehousebox.com and we will look at adding it. Tables are not affected: they hold Iceberg tables, as always. ## What the file bucket takes The type is the file name's last extension, in any case (.parquet, .PARQUET). | kind | extensions | |---|---| | Tables and arrays | .parquet .arrow .feather .ipc .avro .orc .lance .duckdb .sqlite .nc .nc4 .h5 .hdf5 .he5 .grib .grib2 .grb .grb2 | | Rasters | .tif .tiff (Cloud-Optimized GeoTIFF included) .vrt .ovr .msk | | Vector and tiles | .fgb .pmtiles .mbtiles .gpkg .pbf .geojson .geojsonl .topojson .kml .kmz .gml .gpx | | Shapefiles | .shp .shx .dbf .prj .cpg .sbn .sbx .qix | | Point clouds | .las .laz (COPC included) | | Text data | .csv .tsv .txt .json .ndjson .jsonl .yaml .yml .xml .md .sql .ipynb .log .crc .qml .sld | | Spreadsheets | .xlsx .xls .ods | | Documents | .pdf | | Photos | .jpg .jpeg .png .webp .heic .heif | | Archives and compression | .zip .7z .tar .gz .tgz .zst .bz2 .xz (so .csv.gz and .json.zst too) | | Iceberg's own files | .avro .puffin .stats .text | Also taken, whatever the name: - Zarr stores: under a folder whose name ends in .zarr/, the Zarr metadata files (.zarray, .zattrs, .zgroup, .zmetadata, zarr.json) and chunks (0.0.1, c/0/3, c.0.0), which have no extension. - Small files: any file under 100 kB, so a pipeline's markers (COMPLETE, _SUCCESS, EDITION), folder markers (folder/) and small sidecar files keep working. Web pages, scripts and programs (.html, .js, .svg, .exe and the like) are refused at any size. A copy (CopyObject) and the start of a multipart upload do not count as small: their size is not known when the type is checked. Files already in a file bucket stay where they are, whatever their type; the rule applies to new uploads. ## What is refused, and the error Everything else: video, audio, web pages and scripts (.html, .js, .svg), executables, and files with an extension not on the list. The upload answers HTTP 415 with the S3 error code UnsupportedFileType, before any byte is stored: ``` UnsupportedFileTypeLakehouseBox stores data, and this file type is not one a catalog's files accept. … ``` Form uploads (S3's POST Object, used by browser forms) and other POST requests are refused with 501 NotImplemented: upload with PUT or a multipart upload, as every S3 client and SDK does by default. Multi-object deletes still work. The rule applies to every credential that can write files: the catalog's own credential and an uploader key. The FTP and SFTP drop box refuses video, audio, web pages and programs when the device uploads them, so the device sees the error (FTP 553). Any other file the bucket does not take is accepted by the drop box and then refused when it is passed on to the bucket. It is removed from the drop box and not retried. ## What a public folder serves A catalog that publishes a folder of its file bucket serves only that folder's data files to readers without an account. Photos, PDFs, spreadsheets and archives stay readable with your own credentials, and photos can be served publicly for organisations that ask for it. The list is on Public catalogs. ## Asking for a format If your data comes in a format the list does not have, write to hello@lakehousebox.com with the extension and what reads it. The list grows when a format is data that an engine or an agent reads. ## Not yet The check reads the file name, not the content. Checking that a file's content matches its extension is not built yet. --- HTML version: https://lakehousebox.com/docs/files/ · every page: https://lakehousebox.com/llms.txt · everything in one file: https://lakehousebox.com/llms-full.txt