polars.collect_all#

polars.collect_all( lazy_frames: Iterable[LazyFrame], *, type_coercion: bool = True, predicate_pushdown: bool = True, projection_pushdown: bool = True, simplify_expression: bool = True, no_optimization: bool = False, slice_pushdown: bool = True, comm_subplan_elim: bool = True, comm_subexpr_elim: bool = True, cluster_with_columns: bool = True, collapse_joins: bool = True, optimizations: QueryOptFlags = (), engine: EngineType = 'auto', ) → list[DataFrame][source]#

Collect multiple LazyFrames at the same time.

This can run all the computation graphs in parallel or combined.

Common Subplan Elimination is applied on the combined plan, meaning that diverging queries will run only once.

Parameters:

lazy_frames: A list of LazyFrames to collect.
type_coercion: Do type coercion optimization.

Deprecated since version 1.30.0: Use the optimizations parameters.
predicate_pushdown: Do predicate pushdown optimization.

Deprecated since version 1.30.0: Use the optimizations parameters.
projection_pushdown: Do projection pushdown optimization.

Deprecated since version 1.30.0: Use the optimizations parameters.
simplify_expression: Run simplify expressions optimization.

Deprecated since version 1.30.0: Use the optimizations parameters.
no_optimization: Turn off optimizations.

Deprecated since version 1.30.0: Use the optimizations parameters.
slice_pushdown: Slice pushdown optimization.

Deprecated since version 1.30.0: Use the optimizations parameters.
comm_subplan_elim: Will try to cache branching subplans that occur on self-joins or unions.

Deprecated since version 1.30.0: Use the optimizations parameters.
comm_subexpr_elim: Common subexpressions will be cached and reused.

Deprecated since version 1.30.0: Use the optimizations parameters.
cluster_with_columns: Combine sequential independent calls to with_columns

Deprecated since version 1.30.0: Use the optimizations parameters.
collapse_joins: Collapse a join and filters into a faster join

Deprecated since version 1.30.0: Use the optimizations parameters.
optimizations: The optimization passes done during query optimization.

Warning

This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.
engine: Select the engine used to process the query, optional. At the moment, if set to "auto" (default), the query is run using the polars in-memory engine. Polars will also attempt to use the engine set by the POLARS_ENGINE_AFFINITY environment variable. If it cannot run the query using the selected engine, the query is run using the polars in-memory engine.

Note

The GPU engine does not support async, or running in the background. If either are enabled, then GPU execution is switched off.

Returns:

list of DataFrames: The collected DataFrames, returned in the same order as the input LazyFrames.