polars.StreamingEngine#
- class polars.StreamingEngine(*, monitoring: bool | None = None)[source]#
Execute queries in a streaming fashion, supporting larger-than-memory data.
- Parameters:
- monitoringbool, default None
Whether to monitor queries run by this engine.
Noneuses the setting fromConfig.enable_monitoring();TrueorFalseoverrides it. Setting this toTruerequires thepolars-cloudpackage.
Examples
Monitor a single query, regardless of the configured default:
>>> lf.collect(engine=pl.StreamingEngine(monitoring=True))
Methods
__init__(*[, monitoring])collect(lf, *, optimizations[, background, ...])Execute
lf, returning aDataFrameor a background query handle.collect_all(lfs, *, optimizations)Execute several queries, potentially in parallel.
collect_all_async(lfs, *, optimizations[, ...])Execute several queries asynchronously.
collect_async(lf, *, optimizations[, gevent])Execute
lfasynchronously.collect_batches(lf, *, optimizations[, ...])Execute
lf, yielding its result in batches.execute(lf, *, optimizations)Execute the query into a
QueryResult.sink_batches(lf, function, *, chunk_size, ...)sink_csv(lf, path, *, include_bom, ...)sink_ipc(lf, path, *, compression, ...)sink_ndjson(lf, path, *, compression, ...)sink_parquet(lf, path, *, compression, ...)Attributes
monitoringWhether queries are monitored (
Noneuses the configured default).nameName of the engine.
plan_engineEngine name used to render query plans.