polars.SQLContext.register_catalog#

SQLContext.register_catalog( name: str, catalog: pyiceberg.catalog.Catalog | IcebergCatalogConfig, *, default_namespace: str | Sequence[str] | None = None, default: bool = False, storage_options: StorageOptionsDict | None = None, ) → Self[source]#

Attach an Iceberg catalog, so that queries can read its tables by name.

Warning

This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.

A table name that starts with name refers to a table of this catalog (name.namespace.table). With default=True, so does a table name that starts with no attached catalog (namespace.table). A table name without a namespace gets default_namespace. Registered tables and CTEs take precedence over catalog tables, and an attached catalog takes precedence over a namespace of the same name in the default catalog.

Catalog tables are scanned lazily, as with scan_iceberg(): a table is loaded when the query’s schema is first needed, and a table that does not exist raises then. The rest of the table name is passed to the catalog as written; catalog identifiers are case-sensitive.

Parameters:

name: The catalog’s name in SQL. Registering the same name again replaces the catalog: catalog, default_namespace and storage_options take the new values, including their defaults when omitted, while the catalog stays the default if it was (see default). A name that differs from an attached catalog’s only in case is an error.
catalog: A PyIceberg catalog, or an IcebergCatalogConfig.
default_namespace: The namespace of table names without one: a string (split on “.”) or a sequence of namespace parts.
default: Make this catalog the default: table names that start with no attached catalog refer to it. At most one catalog is the default; a later default replaces it. False never removes the default: re-registering the default catalog keeps it the default, and re-registering another catalog leaves it. Only unregister_catalog leaves no default.
storage_options: Options for the storage of the catalog’s tables, as for scan_iceberg(). Pass the storage credentials here when a plan is serialized after its tables loaded: the serialized plan then reads the tables’ metadata directly, with these options rather than the catalog’s properties. DROP TABLE ... PURGE does not use them: the catalog deletes the files with its own properties.

Notes

A serialized query plan over catalog tables can contain the catalog’s properties and the storage options, which may include credentials: treat serialized plans as secret.

DROP TABLE [IF EXISTS] <name> [PURGE] resolves the name as a query does: a registered table of that name is dropped from the context, else the catalog table is dropped from its catalog, permanently. Once no registered table matches, DROP TABLE IF EXISTS of a name reaches the catalog. PURGE also asks the catalog to delete the table’s data files, and only applies to catalog tables. A DROP TABLE of a catalog table takes one name.

CREATE TABLE, TRUNCATE and DELETE on a table of an attached catalog are not supported. For CREATE TABLE that includes a name the default catalog applies to, even if a table of that name is registered: with a default catalog, CREATE TABLE tpch.t does not create a registered table; use CREATE TEMP TABLE tpch.t for that. TRUNCATE TABLE IF EXISTS of a name that does not start with an attached catalog only acts on a registered table, and is a no-op otherwise.

Catalogs attached inside a with SQLContext() block are detached on exit, like the tables registered there. If that detaches the default catalog, the default catalog on entry is restored.

SQLContext is not thread-safe: do not register catalogs or tables while another thread executes a query on the same context.

Examples

>>> from pyiceberg.catalog import load_catalog  
>>> ctx = pl.SQLContext()  
>>> ctx.register_catalog(
...     "lake", load_catalog("lake"), default_namespace="tpch", default=True
... )  
>>> ctx.execute("SELECT COUNT(*) FROM lineitem")  