polars.collect_all#

polars.collect_all(
lazy_frames: Iterable[LazyFrame],
*,
type_coercion: bool = True,
predicate_pushdown: bool = True,
projection_pushdown: bool = True,
simplify_expression: bool = True,
no_optimization: bool = False,
slice_pushdown: bool = True,
comm_subplan_elim: bool = True,
comm_subexpr_elim: bool = True,
cluster_with_columns: bool = True,
streaming: bool = False,
) list[DataFrame][source]#

Collect multiple LazyFrames at the same time.

This runs all the computation graphs in parallel on the Polars threadpool.

Parameters:
lazy_frames

A list of LazyFrames to collect.

type_coercion

Do type coercion optimization.

predicate_pushdown

Do predicate pushdown optimization.

projection_pushdown

Do projection pushdown optimization.

simplify_expression

Run simplify expressions optimization.

no_optimization

Turn off optimizations.

slice_pushdown

Slice pushdown optimization.

comm_subplan_elim

Will try to cache branching subplans that occur on self-joins or unions.

comm_subexpr_elim

Common subexpressions will be cached and reused.

cluster_with_columns

Combine sequential independent calls to with_columns

streaming

Process the query in batches to handle larger-than-memory data. If set to False (default), the entire query is processed in a single batch.

Warning

Streaming mode is considered unstable. It may be changed at any point without it being considered a breaking change.

Note

Use explain() to see if Polars can process the query in streaming mode.

Returns:
list of DataFrames

The collected DataFrames, returned in the same order as the input LazyFrames.