polars.io.plugins.register_io_source#
- polars.io.plugins.register_io_source(
- io_source: Callable[[list[str] | None, Expr | None, int | None, int | None], Iterator[DataFrame]],
- *,
- schema: Callable[[], SchemaDict] | SchemaDict,
- validate_schema: bool = False,
Register your IO plugin and initialize a LazyFrame.
See the user guide for more information about plugins.
Warning
This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.
- Parameters:
- io_source
- Function that accepts the following arguments:
- with_columns
Columns that are projected. The reader must project these columns if applied
- predicate
Polars expression. The reader must filter their rows accordingly.
- n_rows
Materialize only n rows from the source. The reader can stop when
n_rowsare read.- batch_size
A hint of the ideal batch size the reader’s generator must produce.
The function should return a an iterator/generator that produces DataFrames.
- schema
Schema or function that when called produces the schema that the reader will produce before projection pushdown.
- validate_schema
Whether the engine should validate if the batches generated match the given schema. It’s an implementation error if this isn’t the case and can lead to bugs that are hard to solve.
- Returns:
- LazyFrame