polars.collect_all#
- polars.collect_all(
- lazy_frames: Iterable[LazyFrame],
- *,
- type_coercion: bool = True,
- _type_check: bool = True,
- predicate_pushdown: bool = True,
- projection_pushdown: bool = True,
- simplify_expression: bool = True,
- no_optimization: bool = False,
- slice_pushdown: bool = True,
- comm_subplan_elim: bool = True,
- comm_subexpr_elim: bool = True,
- cluster_with_columns: bool = True,
- collapse_joins: bool = True,
- _check_order: bool = True,
- engine: EngineType = 'auto',
Collect multiple LazyFrames at the same time.
This can run all the computation graphs in parallel or combined.
Common Subplan Elimination is applied on the combined plan, meaning that diverging queries will run only once.
- Parameters:
- lazy_frames
A list of LazyFrames to collect.
- type_coercion
Do type coercion optimization.
- predicate_pushdown
Do predicate pushdown optimization.
- projection_pushdown
Do projection pushdown optimization.
- simplify_expression
Run simplify expressions optimization.
- no_optimization
Turn off optimizations.
- slice_pushdown
Slice pushdown optimization.
- comm_subplan_elim
Will try to cache branching subplans that occur on self-joins or unions.
- comm_subexpr_elim
Common subexpressions will be cached and reused.
- cluster_with_columns
Combine sequential independent calls to with_columns
- collapse_joins
Collapse a join and filters into a faster join
- engine
Select the engine used to process the query, optional. At the moment, if set to
"auto"(default), the query is run using the polars in-memory engine. Polars will also attempt to use the engine set by thePOLARS_ENGINE_AFFINITYenvironment variable. If it cannot run the query using the selected engine, the query is run using the polars in-memory engine.Note
The GPU engine does not support async, or running in the background. If either are enabled, then GPU execution is switched off.
- Returns:
- list of DataFrames
The collected DataFrames, returned in the same order as the input LazyFrames.