Struct FileMetadata 

pub struct FileMetadata {
    pub version: i32,
    pub num_rows: usize,
    pub max_row_group_height: usize,
    pub created_by: Option<String>,
    pub row_groups: Vec<RowGroupMetadata>,
    pub key_value_metadata: Option<Vec<KeyValue>>,
    pub schema_descr: SchemaDescriptor,
    pub column_orders: Option<Vec<ColumnOrder>>,
    pub footer_buf: Buffer<u8>,
}

Available on crate feature polars-io only.

Metadata for a Parquet file.

Fields

version: i32

version of this file.

num_rows: usize

number of rows in the file.

max_row_group_height: usize

Max row group height, useful for sharing column materializations.

created_by: Option<String>

String message for application that wrote this file.

This should have the following format: <application> version <application version> (build <application build hash>).

parquet-mr version 1.8.0 (build 0fda28af84b9746396014ad6a415b90592a98b3b)

row_groups: Vec<RowGroupMetadata>

The row groups of this file

key_value_metadata: Option<Vec<KeyValue>>

key_value_metadata of this file.

schema_descr: SchemaDescriptor

schema descriptor.

column_orders: Option<Vec<ColumnOrder>>

Column (sort) order used for min_value and max_value of each leaf column in this file, one per leaf of schema_descr in the same order.

None when the file has no column orders, or not one per leaf; each column then has the undefined (legacy) column order.

footer_buf: Buffer<u8>

Footer bytes that back this file’s column-chunk statistics. Stats min_value / max_value are stored as (offset, len) ranges into this buffer; pass &self.footer_buf to [super::ColumnChunkMetadata::statistics] to materialise them.

Implementations

impl FileMetadata

pub fn schema(&self) -> &SchemaDescriptor

Returns the [SchemaDescriptor] that describes the schema of this file.

pub fn key_value_metadata(&self) -> &Option<Vec<KeyValue>>

Returns the file-level key-value metadata, if present.

pub fn column_order(&self, i: usize) -> ColumnOrder

Returns column order for the ith leaf column in this file. If column orders are not available, returns undefined (legacy) column order.

pub fn pruned( &self, keep_top_level_names: &[PlSmallStr], predicate_top_level_names: &[PlSmallStr], ) -> Result<FileMetadata, ParquetError>

Prune to projected columns, keeping statistics only for predicate columns.

Returns a new FileMetadata containing only:

• top-level schema fields whose name is in keep_top_level_names,
• row-group chunks corresponding to those fields’ leaves,
• statistics on chunks whose column is in predicate_top_level_names.

predicate_top_level_names is treated as a subset of keep_top_level_names; pass &[] to drop all stats. created_by and key_value_metadata are also dropped (not needed by the read hot path); column_orders is kept for the remaining leaves.

Returns Err only when [RowGroupMetadata::from_compact] rejects the rebuilt row group (chunks-vs-leaves desync). Callers can fall back to unpruned metadata; the unpruned form is always valid.

TODO: a planner-side pass could pre-evaluate static predicates against stats and drop fully-skipped row groups, removing stats from the wire for those cases.

Trait Implementations

impl Clone for FileMetadata

fn clone(&self) -> FileMetadata

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FileMetadata

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for FileMetadata

fn deserialize<D>(d: D) -> Result<FileMetadata, <D as Deserializer<'de>>::Error>

where D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for FileMetadata

fn serialize<S>( &self, s: S, ) -> Result<<S as Serializer>::Ok, <S as Serializer>::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.