polars.collect_all#
- polars.collect_all(
- lazy_frames: Iterable[LazyFrame],
- *,
- type_coercion: bool = True,
- predicate_pushdown: bool = True,
- projection_pushdown: bool = True,
- simplify_expression: bool = True,
- no_optimization: bool = False,
- slice_pushdown: bool = True,
- comm_subplan_elim: bool = True,
- comm_subexpr_elim: bool = True,
- cluster_with_columns: bool = True,
- streaming: bool = False,
Collect multiple LazyFrames at the same time.
This runs all the computation graphs in parallel on the Polars threadpool.
- Parameters:
- lazy_frames
A list of LazyFrames to collect.
- type_coercion
Do type coercion optimization.
- predicate_pushdown
Do predicate pushdown optimization.
- projection_pushdown
Do projection pushdown optimization.
- simplify_expression
Run simplify expressions optimization.
- no_optimization
Turn off optimizations.
- slice_pushdown
Slice pushdown optimization.
- comm_subplan_elim
Will try to cache branching subplans that occur on self-joins or unions.
- comm_subexpr_elim
Common subexpressions will be cached and reused.
- cluster_with_columns
Combine sequential independent calls to with_columns
- streaming
Process the query in batches to handle larger-than-memory data. If set to
False(default), the entire query is processed in a single batch.Warning
Streaming mode is considered unstable. It may be changed at any point without it being considered a breaking change.
Note
Use
explain()to see if Polars can process the query in streaming mode.
- Returns:
- list of DataFrames
The collected DataFrames, returned in the same order as the input LazyFrames.