Enum CompressedReader 

pub enum CompressedReader {
    Uncompressed {
        slice: Buffer<u8>,
        offset: usize,
    },
    Gzip(MultiGzDecoder<Cursor<Buffer<u8>>>),
    Zlib(ZlibDecoder<Cursor<Buffer<u8>>>),
    Zstd(Decoder<'static, Cursor<Buffer<u8>>>),
}

Reader that implements a streaming read trait for uncompressed, gzip, zlib and zstd compression.

This allows handling decompression transparently in a streaming fashion.

Variants

Uncompressed

Fields

slice: Buffer<u8>

offset: usize

Gzip(MultiGzDecoder<Cursor<Buffer<u8>>>)

Available on crate feature decompress only.

Zlib(ZlibDecoder<Cursor<Buffer<u8>>>)

Available on crate feature decompress only.

Zstd(Decoder<'static, Cursor<Buffer<u8>>>)

Available on crate feature decompress only.

Implementations

impl CompressedReader

pub fn try_new(slice: Buffer<u8>) -> PolarsResult<Self>

pub fn is_compressed(&self) -> bool

pub const fn initial_read_size() -> usize

pub const fn ideal_read_size() -> usize

pub fn total_len_estimate(&self) -> usize

If possible returns the total number of bytes that will be produced by reading from the start to finish.

pub fn read_next_slice( &mut self, prev_leftover: &Buffer<u8>, read_size: usize, ) -> Result<(Buffer<u8>, usize)>

Reads exactly read_size bytes if possible from the internal readers and creates a new [Buffer] with the content concat(prev_leftover, new_bytes).

Returns the new slice and the number of bytes read, which will be 0 when eof is reached and this function is called again.

If the underlying reader is uncompressed the operation is a cheap zero-copy [Buffer::sliced] operation.

By handling slice concatenation at this level we can implement zero-copy reading and make the interface easier to use.

It’s a logic bug if prev_leftover is neither empty nor the last slice returned by this function.

Trait Implementations

impl Read for CompressedReader

This implementation is meant for compatibility. Use Self::read_next_slice for best performance.

fn read(&mut self, buf: &mut [u8]) -> Result<usize>

Pull some bytes from this source into the specified buffer, returning how many bytes were read.

fn read_vectored(&mut self, bufs: &mut [IoSliceMut<'_>]) -> Result<usize, Error>

Like read, except that it reads into a slice of buffers.

fn is_read_vectored(&self) -> bool

🔬This is a nightly-only experimental API. (can_vector)

Determines if this Reader has an efficient read_vectored implementation.

fn read_to_end(&mut self, buf: &mut Vec<u8>) -> Result<usize, Error>

Reads all bytes until EOF in this source, placing them into buf.

fn read_to_string(&mut self, buf: &mut String) -> Result<usize, Error>

Reads all bytes until EOF in this source, appending them to buf.

fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Error>

Reads the exact number of bytes required to fill buf.

fn read_buf(&mut self, buf: BorrowedCursor<'_>) -> Result<(), Error>

🔬This is a nightly-only experimental API. (read_buf)

Pull some bytes from this source into the specified buffer.

fn read_buf_exact(&mut self, cursor: BorrowedCursor<'_>) -> Result<(), Error>

🔬This is a nightly-only experimental API. (read_buf)

Reads the exact number of bytes required to fill cursor.

fn by_ref(&mut self) -> &mut Self

where Self: Sized,

Creates a “by reference” adapter for this instance of Read.

fn bytes(self) -> Bytes<Self>

where Self: Sized,

Transforms this Read instance to an Iterator over its bytes.

fn chain<R>(self, next: R) -> Chain<Self, R>

where R: Read, Self: Sized,

Creates an adapter which will chain this stream with another.

fn take(self, limit: u64) -> Take<Self>

where Self: Sized,

Creates an adapter which will read at most limit bytes from it.

fn read_array<const N: usize>(&mut self) -> Result<[u8; N], Error>

where Self: Sized,

🔬This is a nightly-only experimental API. (read_array)

Read and return a fixed array of bytes from this source.