Enum CompressedReader 

pub enum CompressedReader {
    Uncompressed {
        slice: MemSlice,
        offset: usize,
    },
    Gzip(MultiGzDecoder<MemReader>),
    Zlib(ZlibDecoder<MemReader>),
    Zstd(Decoder<'static, MemReader>),
}

Available on crate feature polars-io only.

Reader that implements a streaming read trait for uncompressed, gzip, zlib and zstd compression.

This allows handling decompression transparently in a streaming fashion.

Variants

Uncompressed

Fields

slice: MemSlice

offset: usize

Gzip(MultiGzDecoder<MemReader>)

Zlib(ZlibDecoder<MemReader>)

Zstd(Decoder<'static, MemReader>)

Implementations

impl CompressedReader

pub fn try_new(slice: MemSlice) -> Result<CompressedReader, PolarsError>

pub fn is_compressed(&self) -> bool

pub fn total_len_estimate(&self) -> usize

If possible returns the total number of bytes that will be produced by reading from the start to finish.

pub fn read_next_slice( &mut self, prev_leftover: &MemSlice, read_size: usize, ) -> Result<(MemSlice, usize), Error>

Reads exactly read_size bytes if possible from the internal readers and creates a new MemSlice with the content concat(prev_leftover, new_bytes).

Returns the new slice and the number of bytes read, which will be 0 when eof is reached and this function is called again.

If the underlying reader is uncompressed the operation is a cheap zero-copy MemSlice::slice operation.

By handling slice concatenation at this level we can implement zero-copy reading and make the interface easier to use.

It’s a logic bug if prev_leftover is neither empty nor the last slice returned by this function.