polars.collect_all_async#

polars.collect_all_async(
lazy_frames: Iterable[LazyFrame],
*,
gevent: bool = False,
engine: EngineType = 'auto',
optimizations: QueryOptFlags = (),
) → Awaitable[list[DataFrame]] | _GeventDataFrameResult[list[DataFrame]][source]#

Collect multiple LazyFrames at the same time asynchronously in thread pool.

Warning

This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.

Collects into a list of DataFrame (like polars.collect_all()), but instead of returning them directly, they are scheduled to be collected inside thread pool, while this method returns almost instantly.

May be useful if you use gevent or asyncio and want to release control to other greenlets/tasks while LazyFrames are being collected.

Parameters:
lazy_frames

A list of LazyFrames to collect.

gevent

Return wrapper to gevent.event.AsyncResult instead of Awaitable

optimizations

The optimization passes done during query optimization.

Warning

This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.

engine

Select the engine used to process the query (default "auto"):

  • "auto": use the engine set by Config.set_engine_affinity or the POLARS_ENGINE_AFFINITY environment variable, falling back to "in-memory" if unset (this default may change in a future release).

  • "in-memory": use the in-memory engine, this is the default engine.

  • "streaming": use the streaming engine, which processes queries in batches, reducing memory pressure and often outperforming the in-memory engine. This will soon become the default engine of Polars.

  • "gpu": use the CUDA GPU engine (requires an Nvidia GPU and cudf-polars). Pass a GPUEngine object for fine-grained control (e.g. device selection on multi-GPU systems).

If the selected engine cannot run the query, Polars falls back to the in-memory engine.

Note

The GPU engine does not support async, or running in the background. If either are enabled, then GPU execution is switched off.

Returns:
If gevent=False (default) then returns awaitable.
If gevent=True then returns wrapper that has
.get(block=True, timeout=None) method.

See also

polars.collect_all

Collect multiple LazyFrames at the same time.

LazyFrame.collect_async

To collect single frame.

Notes

In case of error set_exception is used on asyncio.Future/gevent.event.AsyncResult and will be reraised by them.

On this page