polars.Config.set_streaming_chunk_size

classmethod Config.set_streaming_chunk_size(size: int) → type[polars.config.Config]

Overwrite chunk size used in streaming engine.

By default, the chunk size is determined by the schema and size of the thread pool. For some datasets (esp. when you have large string elements) this can be too optimistic and lead to Out of Memory errors.

Parameters:

size

Number of rows per chunk. Every thread will process chunks of this size.