Struct DataFrame

pub struct DataFrame { /* private fields */ }

A contiguous growable collection of Series that have the same length.

Use declarations

All the common tools can be found in crate::prelude (or in polars::prelude).

use polars_core::prelude::*; // if the crate polars-core is used directly
// use polars::prelude::*;      if the crate polars is used

Initialization

Default

A DataFrame can be initialized empty:

let df = DataFrame::default();
assert!(df.is_empty());

Wrapping a Vec<Series>

A DataFrame is built upon a Vec<Series> where the Series have the same length.

let s1 = Column::new("Fruit".into(), ["Apple", "Apple", "Pear"]);
let s2 = Column::new("Color".into(), ["Red", "Yellow", "Green"]);

let df: PolarsResult<DataFrame> = DataFrame::new(vec![s1, s2]);

Using a macro

The df! macro is a convenient method:

let df: PolarsResult<DataFrame> = df!("Fruit" => ["Apple", "Apple", "Pear"],
                                      "Color" => ["Red", "Yellow", "Green"]);

Using a CSV file

See the polars_io::csv::CsvReader.

Indexing

By a number

The Index<usize> is implemented for the DataFrame.

let df = df!("Fruit" => ["Apple", "Apple", "Pear"],
             "Color" => ["Red", "Yellow", "Green"])?;

assert_eq!(df[0], Column::new("Fruit".into(), &["Apple", "Apple", "Pear"]));
assert_eq!(df[1], Column::new("Color".into(), &["Red", "Yellow", "Green"]));

By a Series name

let df = df!("Fruit" => ["Apple", "Apple", "Pear"],
             "Color" => ["Red", "Yellow", "Green"])?;

assert_eq!(df["Fruit"], Column::new("Fruit".into(), &["Apple", "Apple", "Pear"]));
assert_eq!(df["Color"], Column::new("Color".into(), &["Red", "Yellow", "Green"]));

Implementations

impl DataFrame

pub fn to_ndarray<N>( &self, ordering: IndexOrder, ) -> PolarsResult<Array2<N::Native>>

where N: PolarsNumericType,

Available on crate feature ndarray only.

Create a 2D ndarray::Array from this DataFrame. This requires all columns in the DataFrame to be non-null and numeric. They will be casted to the same data type (if they aren’t already).

For floating point data we implicitly convert None to NaN without failure.

use polars_core::prelude::*;
let a = UInt32Chunked::new("a".into(), &[1, 2, 3]).into_column();
let b = Float64Chunked::new("b".into(), &[10., 8., 6.]).into_column();

let df = DataFrame::new(vec![a, b]).unwrap();
let ndarray = df.to_ndarray::<Float64Type>(IndexOrder::Fortran).unwrap();
println!("{:?}", ndarray);

Outputs:

[[1.0, 10.0],
 [2.0, 8.0],
 [3.0, 6.0]], shape=[3, 2], strides=[1, 3], layout=Ff (0xa), const ndim=2

impl DataFrame

pub fn sample_n( &self, n: &Series, with_replacement: bool, shuffle: bool, seed: Option<u64>, ) -> PolarsResult<Self>

Available on crate feature random only.

Sample n datapoints from this DataFrame.

pub fn sample_n_literal( &self, n: usize, with_replacement: bool, shuffle: bool, seed: Option<u64>, ) -> PolarsResult<Self>

Available on crate feature random only.

pub fn sample_frac( &self, frac: &Series, with_replacement: bool, shuffle: bool, seed: Option<u64>, ) -> PolarsResult<Self>

Available on crate feature random only.

Sample a fraction between 0.0-1.0 of this DataFrame.

impl DataFrame

pub fn split_chunks(&mut self) -> impl Iterator<Item = DataFrame> + '_

pub fn split_chunks_by_n(self, n: usize, parallel: bool) -> Vec<DataFrame>

impl DataFrame

pub fn explode_impl(&self, columns: Vec<Column>) -> PolarsResult<DataFrame>

pub fn explode<I, S>(&self, columns: I) -> PolarsResult<DataFrame>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Explode DataFrame to long format by exploding a column with Lists.

Example

let s0 = Series::new("a".into(), &[1i64, 2, 3]);
let s1 = Series::new("b".into(), &[1i64, 1, 1]);
let s2 = Series::new("c".into(), &[2i64, 2, 2]);
let list = Series::new("foo", &[s0, s1, s2]);

let s0 = Series::new("B".into(), [1, 2, 3]);
let s1 = Series::new("C".into(), [1, 1, 1]);
let df = DataFrame::new(vec![list, s0, s1])?;
let exploded = df.explode(["foo"])?;

println!("{:?}", df);
println!("{:?}", exploded);

Outputs:

 +-------------+-----+-----+
 | foo         | B   | C   |
 | ---         | --- | --- |
 | list [i64]  | i32 | i32 |
 +=============+=====+=====+
 | "[1, 2, 3]" | 1   | 1   |
 +-------------+-----+-----+
 | "[1, 1, 1]" | 2   | 1   |
 +-------------+-----+-----+
 | "[2, 2, 2]" | 3   | 1   |
 +-------------+-----+-----+

 +-----+-----+-----+
 | foo | B   | C   |
 | --- | --- | --- |
 | i64 | i32 | i32 |
 +=====+=====+=====+
 | 1   | 1   | 1   |
 +-----+-----+-----+
 | 2   | 1   | 1   |
 +-----+-----+-----+
 | 3   | 1   | 1   |
 +-----+-----+-----+
 | 1   | 2   | 1   |
 +-----+-----+-----+
 | 1   | 2   | 1   |
 +-----+-----+-----+
 | 1   | 2   | 1   |
 +-----+-----+-----+
 | 2   | 3   | 1   |
 +-----+-----+-----+
 | 2   | 3   | 1   |
 +-----+-----+-----+
 | 2   | 3   | 1   |
 +-----+-----+-----+

impl DataFrame

pub fn group_by_with_series( &self, by: Vec<Column>, multithreaded: bool, sorted: bool, ) -> PolarsResult<GroupBy<'_>>

Available on crate feature algorithm_group_by only.

pub fn group_by<I, S>(&self, by: I) -> PolarsResult<GroupBy<'_>>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Available on crate feature algorithm_group_by only.

Group DataFrame using a Series column.

Example

use polars_core::prelude::*;
fn group_by_sum(df: &DataFrame) -> PolarsResult<DataFrame> {
    df.group_by(["column_name"])?
    .select(["agg_column_name"])
    .sum()
}

pub fn group_by_stable<I, S>(&self, by: I) -> PolarsResult<GroupBy<'_>>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Available on crate feature algorithm_group_by only.

Group DataFrame using a Series column. The groups are ordered by their smallest row index.

impl DataFrame

pub unsafe fn hstack_mut_unchecked(&mut self, columns: &[Column]) -> &mut Self

Add columns horizontally.

Safety

The caller must ensure:

• the length of all Column is equal to the height of this DataFrame
• the columns names are unique

pub fn hstack_mut(&mut self, columns: &[Column]) -> PolarsResult<&mut Self>

Add multiple Column to a DataFrame. The added Series are required to have the same length.

Example

fn stack(df: &mut DataFrame, columns: &[Column]) {
    df.hstack_mut(columns);
}

impl DataFrame

pub fn get_row(&self, idx: usize) -> PolarsResult<Row<'_>>

Available on crate features rows or object only.

Get a row from a DataFrame. Use of this is discouraged as it will likely be slow.

pub fn get_row_amortized<'a>( &'a self, idx: usize, row: &mut Row<'a>, ) -> PolarsResult<()>

Available on crate features rows or object only.

Amortize allocations by reusing a row. The caller is responsible to make sure that the row has at least the capacity for the number of columns in the DataFrame

pub unsafe fn get_row_amortized_unchecked<'a>( &'a self, idx: usize, row: &mut Row<'a>, )

Available on crate features rows or object only.

Amortize allocations by reusing a row. The caller is responsible to make sure that the row has at least the capacity for the number of columns in the DataFrame

Safety

Does not do any bounds checking.

pub fn from_rows_and_schema( rows: &[Row<'_>], schema: &Schema, ) -> PolarsResult<Self>

Available on crate features rows or object only.

Create a new DataFrame from rows.

This should only be used when you have row wise data, as this is a lot slower than creating the Series in a columnar fashion

pub fn from_rows_iter_and_schema<'a, I>( rows: I, schema: &Schema, ) -> PolarsResult<Self>

where I: Iterator<Item = &'a Row<'a>>,

Available on crate features rows or object only.

Create a new DataFrame from an iterator over rows.

This should only be used when you have row wise data, as this is a lot slower than creating the Series in a columnar fashion.

pub fn try_from_rows_iter_and_schema<'a, I>( rows: I, schema: &Schema, ) -> PolarsResult<Self>

where I: Iterator<Item = PolarsResult<&'a Row<'a>>>,

Available on crate features rows or object only.

Create a new DataFrame from an iterator over rows. This should only be used when you have row wise data, as this is a lot slower than creating the Series in a columnar fashion

pub fn from_rows(rows: &[Row<'_>]) -> PolarsResult<Self>

Available on crate features rows or object only.

Create a new DataFrame from rows. This should only be used when you have row wise data, as this is a lot slower than creating the Series in a columnar fashion

impl DataFrame

pub fn transpose( &mut self, keep_names_as: Option<&str>, new_col_names: Option<Either<String, Vec<String>>>, ) -> PolarsResult<DataFrame>

Available on crate features rows or object only.

pub fn transpose_impl( &mut self, keep_names_as: Option<&str>, new_col_names: Option<Either<PlSmallStr, Vec<PlSmallStr>>>, ) -> PolarsResult<DataFrame>

Available on crate features rows or object only.

Transpose a DataFrame. This is a very expensive operation.

impl DataFrame

pub fn materialized_column_iter(&self) -> impl ExactSizeIterator<Item = &Series>

pub fn par_materialized_column_iter( &self, ) -> impl ParallelIterator<Item = &Series>

pub fn estimated_size(&self) -> usize

Returns an estimation of the total (heap) allocated size of the DataFrame in bytes.

Implementation

This estimation is the sum of the size of its buffers, validity, including nested arrays. Multiple arrays may share buffers and bitmaps. Therefore, the size of 2 arrays is not the sum of the sizes computed from this function. In particular, [StructArray]’s size is an upper bound.

When an array is sliced, its allocated size remains constant because the buffer unchanged. However, this function will yield a smaller number. This is because this function returns the visible size of the buffer, not its total capacity.

FFI buffers are included in this estimation.

pub fn _apply_columns(&self, func: &dyn Fn(&Column) -> Column) -> Vec<Column>

pub fn _apply_columns_par( &self, func: &(dyn Fn(&Column) -> Column + Send + Sync), ) -> Vec<Column>

pub fn new(columns: Vec<Column>) -> PolarsResult<Self>

Create a DataFrame from a Vector of Series.

Example

let s0 = Column::new("days".into(), [0, 1, 2].as_ref());
let s1 = Column::new("temp".into(), [22.1, 19.9, 7.].as_ref());

let df = DataFrame::new(vec![s0, s1])?;

pub fn new_with_broadcast(columns: Vec<Column>) -> PolarsResult<Self>

Converts a sequence of columns into a DataFrame, broadcasting length-1 columns to match the other columns.

pub fn new_with_broadcast_len( columns: Vec<Column>, broadcast_len: usize, ) -> PolarsResult<Self>

Converts a sequence of columns into a DataFrame, broadcasting length-1 columns to broadcast_len.

pub unsafe fn new_with_broadcast_no_namecheck( columns: Vec<Column>, broadcast_len: usize, ) -> PolarsResult<Self>

Converts a sequence of columns into a DataFrame, broadcasting length-1 columns to match the other columns.

Safety

Does not check that the column names are unique (which they must be).

pub const fn empty() -> Self

Creates an empty DataFrame usable in a compile time context (such as static initializers).

Example

use polars_core::prelude::DataFrame;
static EMPTY: DataFrame = DataFrame::empty();

pub fn empty_with_schema(schema: &Schema) -> Self

Create an empty DataFrame with empty columns as per the schema.

pub fn empty_with_arrow_schema(schema: &ArrowSchema) -> Self

Create an empty DataFrame with empty columns as per the schema.

pub fn pop(&mut self) -> Option<Column>

Removes the last Series from the DataFrame and returns it, or None if it is empty.

Example

let s1 = Column::new("Ocean".into(), ["Atlantic", "Indian"]);
let s2 = Column::new("Area (km²)".into(), [106_460_000, 70_560_000]);
let mut df = DataFrame::new(vec![s1.clone(), s2.clone()])?;

assert_eq!(df.pop(), Some(s2));
assert_eq!(df.pop(), Some(s1));
assert_eq!(df.pop(), None);
assert!(df.is_empty());

pub fn with_row_index( &self, name: PlSmallStr, offset: Option<IdxSize>, ) -> PolarsResult<Self>

Add a new column at index 0 that counts the rows.

Example

let df1: DataFrame = df!("Name" => ["James", "Mary", "John", "Patricia"])?;
assert_eq!(df1.shape(), (4, 1));

let df2: DataFrame = df1.with_row_index("Id".into(), None)?;
assert_eq!(df2.shape(), (4, 2));
println!("{}", df2);

Output:

 shape: (4, 2)
 +-----+----------+
 | Id  | Name     |
 | --- | ---      |
 | u32 | str      |
 +=====+==========+
 | 0   | James    |
 +-----+----------+
 | 1   | Mary     |
 +-----+----------+
 | 2   | John     |
 +-----+----------+
 | 3   | Patricia |
 +-----+----------+

pub fn with_row_index_mut( &mut self, name: PlSmallStr, offset: Option<IdxSize>, ) -> &mut Self

Add a row index column in place.

pub unsafe fn new_no_checks_height_from_first(columns: Vec<Column>) -> DataFrame

Create a new DataFrame but does not check the length or duplicate occurrence of the Series.

Calculates the height from the first column or 0 if no columns are given.

Safety

It is the callers responsibility to uphold the contract of all Series having an equal length and a unique name, if not this may panic down the line.

pub unsafe fn new_no_checks(height: usize, columns: Vec<Column>) -> DataFrame

Create a new DataFrame but does not check the length or duplicate occurrence of the Series.

It is advised to use DataFrame::new in favor of this method.

Safety

It is the callers responsibility to uphold the contract of all Series having an equal length and a unique name, if not this may panic down the line.

pub const unsafe fn _new_no_checks_impl( height: usize, columns: Vec<Column>, ) -> DataFrame

This will not panic even in debug mode - there are some (rare) use cases where a DataFrame is temporarily constructed containing duplicates for dispatching to functions. A DataFrame constructed with this method is generally highly unsafe and should not be long-lived.

pub unsafe fn new_no_length_checks( columns: Vec<Column>, ) -> PolarsResult<DataFrame>

Create a new DataFrame but does not check the length of the Series, only check for duplicates.

It is advised to use DataFrame::new in favor of this method.

Safety

It is the callers responsibility to uphold the contract of all Series having an equal length, if not this may panic down the line.

pub fn shrink_to_fit(&mut self)

Shrink the capacity of this DataFrame to fit its length.

pub fn as_single_chunk(&mut self) -> &mut Self

Aggregate all the chunks in the DataFrame to a single chunk.

pub fn as_single_chunk_par(&mut self) -> &mut Self

Aggregate all the chunks in the DataFrame to a single chunk in parallel. This may lead to more peak memory consumption.

pub fn rechunk_mut(&mut self)

Rechunks all columns to only have a single chunk.

pub fn rechunk_to_record_batch( self, compat_level: CompatLevel, ) -> RecordBatchT<Box<dyn Array>>

Rechunks all columns to only have a single chunk and turns it into a [RecordBatchT].

pub fn should_rechunk(&self) -> bool

Returns true if the chunks of the columns do not align and re-chunking should be done

pub fn align_chunks_par(&mut self) -> &mut Self

Ensure all the chunks in the DataFrame are aligned.

pub fn align_chunks(&mut self) -> &mut Self

pub fn schema(&self) -> Schema

Get the DataFrame schema.

Example

let df: DataFrame = df!("Thing" => ["Observable universe", "Human stupidity"],
                        "Diameter (m)" => [8.8e26, f64::INFINITY])?;

let f1: Field = Field::new("Thing".into(), DataType::String);
let f2: Field = Field::new("Diameter (m)".into(), DataType::Float64);
let sc: Schema = Schema::from_iter(vec![f1, f2]);

assert_eq!(df.schema(), sc);

pub fn get_columns(&self) -> &[Column]

Get a reference to the DataFrame columns.

Example

let df: DataFrame = df!("Name" => ["Adenine", "Cytosine", "Guanine", "Thymine"],
                        "Symbol" => ["A", "C", "G", "T"])?;
let columns: &[Column] = df.get_columns();

assert_eq!(columns[0].name(), "Name");
assert_eq!(columns[1].name(), "Symbol");

pub unsafe fn get_columns_mut(&mut self) -> &mut Vec<Column>

Get mutable access to the underlying columns.

Safety

The caller must ensure the length of all Series remains equal to height or DataFrame::set_height is called afterwards with the appropriate height.

pub fn clear_columns(&mut self)

Remove all the columns in the DataFrame but keep the height.

pub unsafe fn column_extend_unchecked( &mut self, iter: impl Iterator<Item = Column>, )

Extend the columns without checking for name collisions or height.

Safety

The caller needs to ensure that:

• Column names are unique within the resulting DataFrame.
• The length of each appended column matches the height of the DataFrame. For DataFrame]s with no columns (ZCDFs), it is important that the height is set afterwards with DataFrame::set_height.

pub fn take_columns(self) -> Vec<Column>

Take ownership of the underlying columns vec.

pub fn iter(&self) -> impl ExactSizeIterator<Item = &Series>

Iterator over the columns as Series.

Example

let s1 = Column::new("Name".into(), ["Pythagoras' theorem", "Shannon entropy"]);
let s2 = Column::new("Formula".into(), ["a²+b²=c²", "H=-Σ[P(x)log|P(x)|]"]);
let df: DataFrame = DataFrame::new(vec![s1.clone(), s2.clone()])?;

let mut iterator = df.iter();

assert_eq!(iterator.next(), Some(s1.as_materialized_series()));
assert_eq!(iterator.next(), Some(s2.as_materialized_series()));
assert_eq!(iterator.next(), None);

pub fn get_column_names(&self) -> Vec<&PlSmallStr>

Example

let df: DataFrame = df!("Language" => ["Rust", "Python"],
                        "Designer" => ["Graydon Hoare", "Guido van Rossum"])?;

assert_eq!(df.get_column_names(), &["Language", "Designer"]);

pub fn get_column_names_owned(&self) -> Vec<PlSmallStr>

Get the Vec<PlSmallStr> representing the column names.

pub fn get_column_names_str(&self) -> Vec<&str>

pub fn set_column_names<I, S>(&mut self, names: I) -> PolarsResult<()>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Set the column names.

Example

let mut df: DataFrame = df!("Mathematical set" => ["ℕ", "ℤ", "𝔻", "ℚ", "ℝ", "ℂ"])?;
df.set_column_names(["Set"])?;

assert_eq!(df.get_column_names(), &["Set"]);

pub fn dtypes(&self) -> Vec<DataType>

Get the data types of the columns in the DataFrame.

Example

let venus_air: DataFrame = df!("Element" => ["Carbon dioxide", "Nitrogen"],
                               "Fraction" => [0.965, 0.035])?;

assert_eq!(venus_air.dtypes(), &[DataType::String, DataType::Float64]);

pub fn first_col_n_chunks(&self) -> usize

The number of chunks for the first column.

pub fn max_n_chunks(&self) -> usize

The highest number of chunks for any column.

pub fn fields(&self) -> Vec<Field>

Get a reference to the schema fields of the DataFrame.

Example

let earth: DataFrame = df!("Surface type" => ["Water", "Land"],
                           "Fraction" => [0.708, 0.292])?;

let f1: Field = Field::new("Surface type".into(), DataType::String);
let f2: Field = Field::new("Fraction".into(), DataType::Float64);

assert_eq!(earth.fields(), &[f1, f2]);

pub fn shape(&self) -> (usize, usize)

Get (height, width) of the DataFrame.

Example

let df0: DataFrame = DataFrame::default();
let df1: DataFrame = df!("1" => [1, 2, 3, 4, 5])?;
let df2: DataFrame = df!("1" => [1, 2, 3, 4, 5],
                         "2" => [1, 2, 3, 4, 5])?;

assert_eq!(df0.shape(), (0 ,0));
assert_eq!(df1.shape(), (5, 1));
assert_eq!(df2.shape(), (5, 2));

pub fn width(&self) -> usize

Get the width of the DataFrame which is the number of columns.

Example

let df0: DataFrame = DataFrame::default();
let df1: DataFrame = df!("Series 1" => [0; 0])?;
let df2: DataFrame = df!("Series 1" => [0; 0],
                         "Series 2" => [0; 0])?;

assert_eq!(df0.width(), 0);
assert_eq!(df1.width(), 1);
assert_eq!(df2.width(), 2);

pub fn height(&self) -> usize

Get the height of the DataFrame which is the number of rows.

Example

let df0: DataFrame = DataFrame::default();
let df1: DataFrame = df!("Currency" => ["€", "$"])?;
let df2: DataFrame = df!("Currency" => ["€", "$", "¥", "£", "₿"])?;

assert_eq!(df0.height(), 0);
assert_eq!(df1.height(), 2);
assert_eq!(df2.height(), 5);

pub fn size(&self) -> usize

Returns the size as number of rows * number of columns

pub fn is_empty(&self) -> bool

Returns true if the DataFrame contains no rows.

Example

let df1: DataFrame = DataFrame::default();
assert!(df1.is_empty());

let df2: DataFrame = df!("First name" => ["Forever"],
                         "Last name" => ["Alone"])?;
assert!(!df2.is_empty());

pub unsafe fn set_height(&mut self, height: usize)

Set the height (i.e. number of rows) of this DataFrame.

Safety

This needs to be equal to the length of all the columns.

pub fn hstack(&self, columns: &[Column]) -> PolarsResult<Self>

Add multiple Series to a DataFrame. The added Series are required to have the same length.

Example

let df1: DataFrame = df!("Element" => ["Copper", "Silver", "Gold"])?;
let s1 = Column::new("Proton".into(), [29, 47, 79]);
let s2 = Column::new("Electron".into(), [29, 47, 79]);

let df2: DataFrame = df1.hstack(&[s1, s2])?;
assert_eq!(df2.shape(), (3, 3));
println!("{}", df2);

Output:

shape: (3, 3)
+---------+--------+----------+
| Element | Proton | Electron |
| ---     | ---    | ---      |
| str     | i32    | i32      |
+=========+========+==========+
| Copper  | 29     | 29       |
+---------+--------+----------+
| Silver  | 47     | 47       |
+---------+--------+----------+
| Gold    | 79     | 79       |
+---------+--------+----------+

pub fn vstack(&self, other: &DataFrame) -> PolarsResult<Self>

Concatenate a DataFrame to this DataFrame and return as newly allocated DataFrame.

If many vstack operations are done, it is recommended to call DataFrame::align_chunks_par.

Example

let df1: DataFrame = df!("Element" => ["Copper", "Silver", "Gold"],
                         "Melting Point (K)" => [1357.77, 1234.93, 1337.33])?;
let df2: DataFrame = df!("Element" => ["Platinum", "Palladium"],
                         "Melting Point (K)" => [2041.4, 1828.05])?;

let df3: DataFrame = df1.vstack(&df2)?;

assert_eq!(df3.shape(), (5, 2));
println!("{}", df3);

Output:

shape: (5, 2)
+-----------+-------------------+
| Element   | Melting Point (K) |
| ---       | ---               |
| str       | f64               |
+===========+===================+
| Copper    | 1357.77           |
+-----------+-------------------+
| Silver    | 1234.93           |
+-----------+-------------------+
| Gold      | 1337.33           |
+-----------+-------------------+
| Platinum  | 2041.4            |
+-----------+-------------------+
| Palladium | 1828.05           |
+-----------+-------------------+

pub fn vstack_mut(&mut self, other: &DataFrame) -> PolarsResult<&mut Self>

Concatenate a DataFrame to this DataFrame

If many vstack operations are done, it is recommended to call DataFrame::align_chunks_par.

Example

let mut df1: DataFrame = df!("Element" => ["Copper", "Silver", "Gold"],
                         "Melting Point (K)" => [1357.77, 1234.93, 1337.33])?;
let df2: DataFrame = df!("Element" => ["Platinum", "Palladium"],
                         "Melting Point (K)" => [2041.4, 1828.05])?;

df1.vstack_mut(&df2)?;

assert_eq!(df1.shape(), (5, 2));
println!("{}", df1);

Output:

shape: (5, 2)
+-----------+-------------------+
| Element   | Melting Point (K) |
| ---       | ---               |
| str       | f64               |
+===========+===================+
| Copper    | 1357.77           |
+-----------+-------------------+
| Silver    | 1234.93           |
+-----------+-------------------+
| Gold      | 1337.33           |
+-----------+-------------------+
| Platinum  | 2041.4            |
+-----------+-------------------+
| Palladium | 1828.05           |
+-----------+-------------------+

pub fn vstack_mut_unchecked(&mut self, other: &DataFrame)

Concatenate a DataFrame to this DataFrame

If many vstack operations are done, it is recommended to call DataFrame::align_chunks_par.

Panics

Panics if the schema’s don’t match.

pub fn extend(&mut self, other: &DataFrame) -> PolarsResult<()>

Extend the memory backed by this DataFrame with the values from other.

Different from vstack which adds the chunks from other to the chunks of this DataFrame extend appends the data from other to the underlying memory locations and thus may cause a reallocation.

If this does not cause a reallocation, the resulting data structure will not have any extra chunks and thus will yield faster queries.

Prefer extend over vstack when you want to do a query after a single append. For instance during online operations where you add n rows and rerun a query.

Prefer vstack over extend when you want to append many times before doing a query. For instance when you read in multiple files and when to store them in a single DataFrame. In the latter case, finish the sequence of append operations with a rechunk.

pub fn drop_in_place(&mut self, name: &str) -> PolarsResult<Column>

Remove a column by name and return the column removed.

Example

let mut df: DataFrame = df!("Animal" => ["Tiger", "Lion", "Great auk"],
                            "IUCN" => ["Endangered", "Vulnerable", "Extinct"])?;

let s1: PolarsResult<Column> = df.drop_in_place("Average weight");
assert!(s1.is_err());

let s2: Column = df.drop_in_place("Animal")?;
assert_eq!(s2, Column::new("Animal".into(), &["Tiger", "Lion", "Great auk"]));

pub fn drop_nulls<S>(&self, subset: Option<&[S]>) -> PolarsResult<Self>

where for<'a> &'a S: Into<PlSmallStr>,

Return a new DataFrame where all null values are dropped.

Example

let df1: DataFrame = df!("Country" => ["Malta", "Liechtenstein", "North Korea"],
                        "Tax revenue (% GDP)" => [Some(32.7), None, None])?;
assert_eq!(df1.shape(), (3, 2));

let df2: DataFrame = df1.drop_nulls::<String>(None)?;
assert_eq!(df2.shape(), (1, 2));
println!("{}", df2);

Output:

shape: (1, 2)
+---------+---------------------+
| Country | Tax revenue (% GDP) |
| ---     | ---                 |
| str     | f64                 |
+=========+=====================+
| Malta   | 32.7                |
+---------+---------------------+

pub fn drop(&self, name: &str) -> PolarsResult<Self>

Drop a column by name. This is a pure method and will return a new DataFrame instead of modifying the current one in place.

Example

let df1: DataFrame = df!("Ray type" => ["α", "β", "X", "γ"])?;
let df2: DataFrame = df1.drop("Ray type")?;

assert!(df2.is_empty());

pub fn drop_many<I, S>(&self, names: I) -> Self

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Drop columns that are in names.

pub fn drop_many_amortized(&self, names: &PlHashSet<PlSmallStr>) -> DataFrame

Drop columns that are in names without allocating a HashSet.

pub fn insert_column<S: IntoColumn>( &mut self, index: usize, column: S, ) -> PolarsResult<&mut Self>

Insert a new column at a given index.

pub fn with_column<C: IntoColumn>( &mut self, column: C, ) -> PolarsResult<&mut Self>

Add a new column to this DataFrame or replace an existing one.

pub unsafe fn with_column_unchecked(&mut self, column: Column) -> &mut Self

Adds a column to the DataFrame without doing any checks on length or duplicates.

Safety

The caller must ensure self.width() == 0 || column.len() == self.height() .

pub fn _add_series( &mut self, series: Vec<Series>, schema: &Schema, ) -> PolarsResult<()>

pub fn _add_columns( &mut self, columns: Vec<Column>, schema: &Schema, ) -> PolarsResult<()>

pub fn with_column_and_schema<C: IntoColumn>( &mut self, column: C, schema: &Schema, ) -> PolarsResult<&mut Self>

Add a new column to this DataFrame or replace an existing one. Uses an existing schema to amortize lookups. If the schema is incorrect, we will fallback to linear search.

Note: Schema can be both input or output_schema

pub fn get(&self, idx: usize) -> Option<Vec<AnyValue<'_>>>

Get a row in the DataFrame. Beware this is slow.

Example

fn example(df: &mut DataFrame, idx: usize) -> Option<Vec<AnyValue>> {
    df.get(idx)
}

pub fn select_at_idx(&self, idx: usize) -> Option<&Column>

Select a Series by index.

Example

let df: DataFrame = df!("Star" => ["Sun", "Betelgeuse", "Sirius A", "Sirius B"],
                        "Absolute magnitude" => [4.83, -5.85, 1.42, 11.18])?;

let s1: Option<&Column> = df.select_at_idx(0);
let s2 = Column::new("Star".into(), ["Sun", "Betelgeuse", "Sirius A", "Sirius B"]);

assert_eq!(s1, Some(&s2));

pub fn select_by_range<R>(&self, range: R) -> PolarsResult<Self>

where R: RangeBounds<usize>,

Select column(s) from this DataFrame by range and return a new DataFrame

Examples

let df = df! {
    "0" => [0, 0, 0],
    "1" => [1, 1, 1],
    "2" => [2, 2, 2]
}?;

assert!(df.select(["0", "1"])?.equals(&df.select_by_range(0..=1)?));
assert!(df.equals(&df.select_by_range(..)?));

pub fn get_column_index(&self, name: &str) -> Option<usize>

Get column index of a Series by name.

Example

let df: DataFrame = df!("Name" => ["Player 1", "Player 2", "Player 3"],
                        "Health" => [100, 200, 500],
                        "Mana" => [250, 100, 0],
                        "Strength" => [30, 150, 300])?;

assert_eq!(df.get_column_index("Name"), Some(0));
assert_eq!(df.get_column_index("Health"), Some(1));
assert_eq!(df.get_column_index("Mana"), Some(2));
assert_eq!(df.get_column_index("Strength"), Some(3));
assert_eq!(df.get_column_index("Haste"), None);

pub fn try_get_column_index(&self, name: &str) -> PolarsResult<usize>

Get column index of a Series by name.

pub fn column(&self, name: &str) -> PolarsResult<&Column>

Select a single column by name.

Example

let s1 = Column::new("Password".into(), ["123456", "[]B$u$g$s$B#u#n#n#y[]{}"]);
let s2 = Column::new("Robustness".into(), ["Weak", "Strong"]);
let df: DataFrame = DataFrame::new(vec![s1.clone(), s2])?;

assert_eq!(df.column("Password")?, &s1);

pub fn columns<I, S>(&self, names: I) -> PolarsResult<Vec<&Column>>

where I: IntoIterator<Item = S>, S: AsRef<str>,

Selected multiple columns by name.

Example

let df: DataFrame = df!("Latin name" => ["Oncorhynchus kisutch", "Salmo salar"],
                        "Max weight (kg)" => [16.0, 35.89])?;
let sv: Vec<&Column> = df.columns(["Latin name", "Max weight (kg)"])?;

assert_eq!(&df[0], sv[0]);
assert_eq!(&df[1], sv[1]);

pub fn select<I, S>(&self, selection: I) -> PolarsResult<Self>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Select column(s) from this DataFrame and return a new DataFrame.

Examples

fn example(df: &DataFrame) -> PolarsResult<DataFrame> {
    df.select(["foo", "bar"])
}

pub fn _select_impl(&self, cols: &[PlSmallStr]) -> PolarsResult<Self>

pub fn _select_impl_unchecked(&self, cols: &[PlSmallStr]) -> PolarsResult<Self>

pub fn select_with_schema<I, S>( &self, selection: I, schema: &SchemaRef, ) -> PolarsResult<Self>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Select with a known schema.

pub fn select_with_schema_unchecked<I, S>( &self, selection: I, schema: &Schema, ) -> PolarsResult<Self>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Select with a known schema. This doesn’t check for duplicates.

pub fn _select_with_schema_impl( &self, cols: &[PlSmallStr], schema: &Schema, check_duplicates: bool, ) -> PolarsResult<Self>

pub fn select_physical<I, S>(&self, selection: I) -> PolarsResult<Self>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

pub fn select_columns( &self, selection: impl IntoVec<PlSmallStr>, ) -> PolarsResult<Vec<Column>>

Select column(s) from this DataFrame and return them into a Vec.

Example

let df: DataFrame = df!("Name" => ["Methane", "Ethane", "Propane"],
                        "Carbon" => [1, 2, 3],
                        "Hydrogen" => [4, 6, 8])?;
let sv: Vec<Column> = df.select_columns(["Carbon", "Hydrogen"])?;

assert_eq!(df["Carbon"], sv[0]);
assert_eq!(df["Hydrogen"], sv[1]);

pub fn filter(&self, mask: &BooleanChunked) -> PolarsResult<Self>

Take the DataFrame rows by a boolean mask.

Example

fn example(df: &DataFrame) -> PolarsResult<DataFrame> {
    let mask = df.column("sepal_width")?.is_not_null();
    df.filter(&mask)
}

pub fn _filter_seq(&self, mask: &BooleanChunked) -> PolarsResult<Self>

Same as filter but does not parallelize.

pub fn take(&self, indices: &IdxCa) -> PolarsResult<Self>

Take DataFrame rows by index values.

Example

fn example(df: &DataFrame) -> PolarsResult<DataFrame> {
    let idx = IdxCa::new("idx".into(), [0, 1, 9]);
    df.take(&idx)
}

pub unsafe fn take_unchecked(&self, idx: &IdxCa) -> Self

Safety

The indices must be in-bounds.

pub unsafe fn take_unchecked_impl( &self, idx: &IdxCa, allow_threads: bool, ) -> Self

Safety

The indices must be in-bounds.

pub fn rename( &mut self, column: &str, name: PlSmallStr, ) -> PolarsResult<&mut Self>

Rename a column in the DataFrame.

Example

fn example(df: &mut DataFrame) -> PolarsResult<&mut DataFrame> {
    let original_name = "foo";
    let new_name = "bar";
    df.rename(original_name, new_name.into())
}

pub fn sort_in_place( &mut self, by: impl IntoVec<PlSmallStr>, sort_options: SortMultipleOptions, ) -> PolarsResult<&mut Self>

Sort DataFrame in place.

See DataFrame::sort for more instruction.

pub fn _to_metadata(&self) -> DataFrame

Create a DataFrame that has fields for all the known runtime metadata for each column.

This dataframe does not necessarily have a specified schema and may be changed at any point. It is primarily used for debugging.

pub fn sort( &self, by: impl IntoVec<PlSmallStr>, sort_options: SortMultipleOptions, ) -> PolarsResult<Self>

Return a sorted clone of this DataFrame.

Example

Sort by a single column with default options:

fn sort_by_sepal_width(df: &DataFrame) -> PolarsResult<DataFrame> {
    df.sort(["sepal_width"], Default::default())
}

Sort by a single column with specific order:

fn sort_with_specific_order(df: &DataFrame, descending: bool) -> PolarsResult<DataFrame> {
    df.sort(
        ["sepal_width"],
        SortMultipleOptions::new()
            .with_order_descending(descending)
    )
}

Sort by multiple columns with specifying order for each column:

fn sort_by_multiple_columns_with_specific_order(df: &DataFrame) -> PolarsResult<DataFrame> {
    df.sort(
        ["sepal_width", "sepal_length"],
        SortMultipleOptions::new()
            .with_order_descending_multi([false, true])
    )
}

See SortMultipleOptions for more options.

Also see DataFrame::sort_in_place.

pub fn replace<S: IntoSeries>( &mut self, column: &str, new_col: S, ) -> PolarsResult<&mut Self>

Replace a column with a Series.

Example

let mut df: DataFrame = df!("Country" => ["United States", "China"],
                        "Area (km²)" => [9_833_520, 9_596_961])?;
let s: Series = Series::new("Country".into(), ["USA", "PRC"]);

assert!(df.replace("Nation", s.clone()).is_err());
assert!(df.replace("Country", s).is_ok());

pub fn replace_or_add<S: IntoSeries>( &mut self, column: PlSmallStr, new_col: S, ) -> PolarsResult<&mut Self>

Replace or update a column. The difference between this method and DataFrame::with_column is that now the value of column: &str determines the name of the column and not the name of the Series passed to this method.

pub fn replace_column<C: IntoColumn>( &mut self, index: usize, new_column: C, ) -> PolarsResult<&mut Self>

Replace column at index idx with a Series.

Example

# use polars_core::prelude::*;
let s0 = Series::new("foo".into(), ["ham", "spam", "egg"]);
let s1 = Series::new("ascii".into(), [70, 79, 79]);
let mut df = DataFrame::new(vec![s0, s1])?;

// Add 32 to get lowercase ascii values
df.replace_column(1, df.select_at_idx(1).unwrap() + 32);
# Ok::<(), PolarsError>(())

pub fn apply<F, C>(&mut self, name: &str, f: F) -> PolarsResult<&mut Self>

where F: FnOnce(&Column) -> C, C: IntoColumn,

Apply a closure to a column. This is the recommended way to do in place modification.

Example

let s0 = Column::new("foo".into(), ["ham", "spam", "egg"]);
let s1 = Column::new("names".into(), ["Jean", "Claude", "van"]);
let mut df = DataFrame::new(vec![s0, s1])?;

fn str_to_len(str_val: &Column) -> Column {
    str_val.str()
        .unwrap()
        .into_iter()
        .map(|opt_name: Option<&str>| {
            opt_name.map(|name: &str| name.len() as u32)
         })
        .collect::<UInt32Chunked>()
        .into_column()
}

// Replace the names column by the length of the names.
df.apply("names", str_to_len);

Results in:

+--------+-------+
| foo    |       |
| ---    | names |
| str    | u32   |
+========+=======+
| "ham"  | 4     |
+--------+-------+
| "spam" | 6     |
+--------+-------+
| "egg"  | 3     |
+--------+-------+

pub fn apply_at_idx<F, C>( &mut self, idx: usize, f: F, ) -> PolarsResult<&mut Self>

where F: FnOnce(&Column) -> C, C: IntoColumn,

Apply a closure to a column at index idx. This is the recommended way to do in place modification.

Example

let s0 = Column::new("foo".into(), ["ham", "spam", "egg"]);
let s1 = Column::new("ascii".into(), [70, 79, 79]);
let mut df = DataFrame::new(vec![s0, s1])?;

// Add 32 to get lowercase ascii values
df.apply_at_idx(1, |s| s + 32);

Results in:

+--------+-------+
| foo    | ascii |
| ---    | ---   |
| str    | i32   |
+========+=======+
| "ham"  | 102   |
+--------+-------+
| "spam" | 111   |
+--------+-------+
| "egg"  | 111   |
+--------+-------+

pub fn try_apply_at_idx<F, C>( &mut self, idx: usize, f: F, ) -> PolarsResult<&mut Self>

where F: FnOnce(&Column) -> PolarsResult<C>, C: IntoColumn,

Apply a closure that may fail to a column at index idx. This is the recommended way to do in place modification.

Example

This is the idiomatic way to replace some values a column of a DataFrame given range of indexes.

let s0 = Column::new("foo".into(), ["ham", "spam", "egg", "bacon", "quack"]);
let s1 = Column::new("values".into(), [1, 2, 3, 4, 5]);
let mut df = DataFrame::new(vec![s0, s1])?;

let idx = vec![0, 1, 4];

df.try_apply("foo", |c| {
    c.str()?
    .scatter_with(idx, |opt_val| opt_val.map(|string| format!("{}-is-modified", string)))
});

Results in:

+---------------------+--------+
| foo                 | values |
| ---                 | ---    |
| str                 | i32    |
+=====================+========+
| "ham-is-modified"   | 1      |
+---------------------+--------+
| "spam-is-modified"  | 2      |
+---------------------+--------+
| "egg"               | 3      |
+---------------------+--------+
| "bacon"             | 4      |
+---------------------+--------+
| "quack-is-modified" | 5      |
+---------------------+--------+

pub fn try_apply<F, C>(&mut self, column: &str, f: F) -> PolarsResult<&mut Self>

where F: FnOnce(&Series) -> PolarsResult<C>, C: IntoColumn,

Apply a closure that may fail to a column. This is the recommended way to do in place modification.

Example

This is the idiomatic way to replace some values a column of a DataFrame given a boolean mask.

let s0 = Column::new("foo".into(), ["ham", "spam", "egg", "bacon", "quack"]);
let s1 = Column::new("values".into(), [1, 2, 3, 4, 5]);
let mut df = DataFrame::new(vec![s0, s1])?;

// create a mask
let values = df.column("values")?.as_materialized_series();
let mask = values.lt_eq(1)? | values.gt_eq(5_i32)?;

df.try_apply("foo", |c| {
    c.str()?
    .set(&mask, Some("not_within_bounds"))
});

Results in:

+---------------------+--------+
| foo                 | values |
| ---                 | ---    |
| str                 | i32    |
+=====================+========+
| "not_within_bounds" | 1      |
+---------------------+--------+
| "spam"              | 2      |
+---------------------+--------+
| "egg"               | 3      |
+---------------------+--------+
| "bacon"             | 4      |
+---------------------+--------+
| "not_within_bounds" | 5      |
+---------------------+--------+

pub fn slice(&self, offset: i64, length: usize) -> Self

Slice the DataFrame along the rows.

Example

let df: DataFrame = df!("Fruit" => ["Apple", "Grape", "Grape", "Fig", "Fig"],
                        "Color" => ["Green", "Red", "White", "White", "Red"])?;
let sl: DataFrame = df.slice(2, 3);

assert_eq!(sl.shape(), (3, 2));
println!("{}", sl);

Output:

shape: (3, 2)
+-------+-------+
| Fruit | Color |
| ---   | ---   |
| str   | str   |
+=======+=======+
| Grape | White |
+-------+-------+
| Fig   | White |
+-------+-------+
| Fig   | Red   |
+-------+-------+

pub fn split_at(&self, offset: i64) -> (Self, Self)

Split DataFrame at the given offset.

pub fn clear(&self) -> Self

pub fn slice_par(&self, offset: i64, length: usize) -> Self

pub fn _slice_and_realloc(&self, offset: i64, length: usize) -> Self

pub fn head(&self, length: Option<usize>) -> Self

Get the head of the DataFrame.

Example

let countries: DataFrame =
    df!("Rank by GDP (2021)" => [1, 2, 3, 4, 5],
        "Continent" => ["North America", "Asia", "Asia", "Europe", "Europe"],
        "Country" => ["United States", "China", "Japan", "Germany", "United Kingdom"],
        "Capital" => ["Washington", "Beijing", "Tokyo", "Berlin", "London"])?;
assert_eq!(countries.shape(), (5, 4));

println!("{}", countries.head(Some(3)));

Output:

shape: (3, 4)
+--------------------+---------------+---------------+------------+
| Rank by GDP (2021) | Continent     | Country       | Capital    |
| ---                | ---           | ---           | ---        |
| i32                | str           | str           | str        |
+====================+===============+===============+============+
| 1                  | North America | United States | Washington |
+--------------------+---------------+---------------+------------+
| 2                  | Asia          | China         | Beijing    |
+--------------------+---------------+---------------+------------+
| 3                  | Asia          | Japan         | Tokyo      |
+--------------------+---------------+---------------+------------+

pub fn tail(&self, length: Option<usize>) -> Self

Get the tail of the DataFrame.

Example

let countries: DataFrame =
    df!("Rank (2021)" => [105, 106, 107, 108, 109],
        "Apple Price (€/kg)" => [0.75, 0.70, 0.70, 0.65, 0.52],
        "Country" => ["Kosovo", "Moldova", "North Macedonia", "Syria", "Turkey"])?;
assert_eq!(countries.shape(), (5, 3));

println!("{}", countries.tail(Some(2)));

Output:

shape: (2, 3)
+-------------+--------------------+---------+
| Rank (2021) | Apple Price (€/kg) | Country |
| ---         | ---                | ---     |
| i32         | f64                | str     |
+=============+====================+=========+
| 108         | 0.63               | Syria   |
+-------------+--------------------+---------+
| 109         | 0.63               | Turkey  |
+-------------+--------------------+---------+

pub fn iter_chunks( &self, compat_level: CompatLevel, parallel: bool, ) -> RecordBatchIter<'_>

Iterator over the rows in this DataFrame as Arrow RecordBatches.

Panics

Panics if the DataFrame that is passed is not rechunked.

This responsibility is left to the caller as we don’t want to take mutable references here, but we also don’t want to rechunk here, as this operation is costly and would benefit the caller as well.

pub fn iter_chunks_physical(&self) -> PhysRecordBatchIter<'_>

Iterator over the rows in this DataFrame as Arrow RecordBatches as physical values.

Panics

Panics if the DataFrame that is passed is not rechunked.

This responsibility is left to the caller as we don’t want to take mutable references here, but we also don’t want to rechunk here, as this operation is costly and would benefit the caller as well.

pub fn reverse(&self) -> Self

Get a DataFrame with all the columns in reversed order.

pub fn shift(&self, periods: i64) -> Self

Shift the values by a given period and fill the parts that will be empty due to this operation with Nones.

See the method on Series for more info on the shift operation.

pub fn fill_null(&self, strategy: FillNullStrategy) -> PolarsResult<Self>

Replace None values with one of the following strategies:

• Forward fill (replace None with the previous value)
• Backward fill (replace None with the next value)
• Mean fill (replace None with the mean of the whole array)
• Min fill (replace None with the minimum of the whole array)
• Max fill (replace None with the maximum of the whole array)

See the method on Series for more info on the fill_null operation.

pub fn min_horizontal(&self) -> PolarsResult<Option<Column>>

Available on crate feature zip_with only.

Aggregate the column horizontally to their min values.

pub fn max_horizontal(&self) -> PolarsResult<Option<Column>>

Available on crate feature zip_with only.

Aggregate the column horizontally to their max values.

pub fn sum_horizontal( &self, null_strategy: NullStrategy, ) -> PolarsResult<Option<Series>>

Sum all values horizontally across columns.

pub fn mean_horizontal( &self, null_strategy: NullStrategy, ) -> PolarsResult<Option<Series>>

Compute the mean of all numeric values horizontally across columns.

pub fn pipe<F, B>(self, f: F) -> PolarsResult<B>

where F: Fn(DataFrame) -> PolarsResult<B>,

Pipe different functions/ closure operations that work on a DataFrame together.

pub fn pipe_mut<F, B>(&mut self, f: F) -> PolarsResult<B>

where F: Fn(&mut DataFrame) -> PolarsResult<B>,

Pipe different functions/ closure operations that work on a DataFrame together.

pub fn pipe_with_args<F, B, Args>(self, f: F, args: Args) -> PolarsResult<B>

where F: Fn(DataFrame, Args) -> PolarsResult<B>,

Pipe different functions/ closure operations that work on a DataFrame together.

pub fn unique_stable( &self, subset: Option<&[String]>, keep: UniqueKeepStrategy, slice: Option<(i64, usize)>, ) -> PolarsResult<DataFrame>

Available on crate feature algorithm_group_by only.

Drop duplicate rows from a DataFrame. This fails when there is a column of type List in DataFrame

Stable means that the order is maintained. This has a higher cost than an unstable distinct.

Example

let df = df! {
              "flt" => [1., 1., 2., 2., 3., 3.],
              "int" => [1, 1, 2, 2, 3, 3, ],
              "str" => ["a", "a", "b", "b", "c", "c"]
          }?;

println!("{}", df.unique_stable(None, UniqueKeepStrategy::First, None)?);

Returns

+-----+-----+-----+
| flt | int | str |
| --- | --- | --- |
| f64 | i32 | str |
+=====+=====+=====+
| 1   | 1   | "a" |
+-----+-----+-----+
| 2   | 2   | "b" |
+-----+-----+-----+
| 3   | 3   | "c" |
+-----+-----+-----+

pub fn unique<I, S>( &self, subset: Option<&[String]>, keep: UniqueKeepStrategy, slice: Option<(i64, usize)>, ) -> PolarsResult<DataFrame>

Available on crate feature algorithm_group_by only.

Unstable distinct. See DataFrame::unique_stable.

pub fn unique_impl( &self, maintain_order: bool, subset: Option<Vec<PlSmallStr>>, keep: UniqueKeepStrategy, slice: Option<(i64, usize)>, ) -> PolarsResult<Self>

Available on crate feature algorithm_group_by only.

pub fn is_unique(&self) -> PolarsResult<BooleanChunked>

Available on crate feature algorithm_group_by only.

Get a mask of all the unique rows in the DataFrame.

Example

let df: DataFrame = df!("Company" => ["Apple", "Microsoft"],
                        "ISIN" => ["US0378331005", "US5949181045"])?;
let ca: ChunkedArray<BooleanType> = df.is_unique()?;

assert!(ca.all());

pub fn is_duplicated(&self) -> PolarsResult<BooleanChunked>

Available on crate feature algorithm_group_by only.

Get a mask of all the duplicated rows in the DataFrame.

Example

let df: DataFrame = df!("Company" => ["Alphabet", "Alphabet"],
                        "ISIN" => ["US02079K3059", "US02079K1079"])?;
let ca: ChunkedArray<BooleanType> = df.is_duplicated()?;

assert!(!ca.all());

pub fn null_count(&self) -> Self

Create a new DataFrame that shows the null counts per column.

pub fn hash_rows( &mut self, hasher_builder: Option<PlRandomState>, ) -> PolarsResult<UInt64Chunked>

Available on crate feature row_hash only.

Hash and combine the row values

pub fn get_supertype(&self) -> Option<PolarsResult<DataType>>

Get the supertype of the columns in this DataFrame

pub fn partition_by<I, S>( &self, cols: I, include_key: bool, ) -> PolarsResult<Vec<DataFrame>>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Available on crate feature partition_by only.

Split into multiple DataFrames partitioned by groups

pub fn partition_by_stable<I, S>( &self, cols: I, include_key: bool, ) -> PolarsResult<Vec<DataFrame>>

where I: IntoIterator<Item = S>, S: Into<PlSmallStr>,

Available on crate feature partition_by only.

Split into multiple DataFrames partitioned by groups Order of the groups are maintained.

pub fn append_record_batch( &mut self, rb: RecordBatchT<ArrayRef>, ) -> PolarsResult<()>

impl DataFrame

pub fn schema_equal(&self, other: &DataFrame) -> PolarsResult<()>

Check if DataFrame’ schemas are equal.

pub fn equals(&self, other: &DataFrame) -> bool

Check if DataFrames are equal. Note that None == None evaluates to false

Example

let df1: DataFrame = df!("Atomic number" => &[1, 51, 300],
                        "Element" => &[Some("Hydrogen"), Some("Antimony"), None])?;
let df2: DataFrame = df!("Atomic number" => &[1, 51, 300],
                        "Element" => &[Some("Hydrogen"), Some("Antimony"), None])?;

assert!(!df1.equals(&df2));

pub fn equals_missing(&self, other: &DataFrame) -> bool

Check if all values in DataFrames are equal where None == None evaluates to true.

Example

let df1: DataFrame = df!("Atomic number" => &[1, 51, 300],
                        "Element" => &[Some("Hydrogen"), Some("Antimony"), None])?;
let df2: DataFrame = df!("Atomic number" => &[1, 51, 300],
                        "Element" => &[Some("Hydrogen"), Some("Antimony"), None])?;

assert!(df1.equals_missing(&df2));

Trait Implementations

impl Add<&DataFrame> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the + operator.

fn add(self, rhs: &DataFrame) -> Self::Output

Performs the + operation.

impl Add<&Series> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the + operator.

fn add(self, rhs: &Series) -> Self::Output

Performs the + operation.

impl Add<&Series> for DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the + operator.

fn add(self, rhs: &Series) -> Self::Output

Performs the + operation.

impl Clone for DataFrame

fn clone(&self) -> DataFrame

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Container for DataFrame

fn slice(&self, offset: i64, len: usize) -> Self

fn split_at(&self, offset: i64) -> (Self, Self)

fn len(&self) -> usize

fn iter_chunks(&self) -> impl Iterator<Item = Self>

fn n_chunks(&self) -> usize

fn chunk_lengths(&self) -> impl Iterator<Item = usize>

impl Debug for DataFrame

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DataFrame

fn default() -> Self

Returns the “default value” for a type.

impl Display for DataFrame

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Div<&DataFrame> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the / operator.

fn div(self, rhs: &DataFrame) -> Self::Output

Performs the / operation.

impl Div<&Series> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the / operator.

fn div(self, rhs: &Series) -> Self::Output

Performs the / operation.

impl Div<&Series> for DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the / operator.

fn div(self, rhs: &Series) -> Self::Output

Performs the / operation.

impl From<DataFrame> for Vec<Column>

fn from(df: DataFrame) -> Self

Converts to this type from the input type.

impl FromIterator<Column> for DataFrame

fn from_iter<T: IntoIterator<Item = Column>>(iter: T) -> Self

Panics

Panics if Column have different lengths.

impl FromIterator<Series> for DataFrame

fn from_iter<T: IntoIterator<Item = Series>>(iter: T) -> Self

Panics

Panics if Series have different lengths.

impl Index<&str> for DataFrame

type Output = Column

The returned type after indexing.

fn index(&self, index: &str) -> &Self::Output

Performs the indexing (container[index]) operation.

impl Index<Range<usize>> for DataFrame

type Output = [Column]

The returned type after indexing.

fn index(&self, index: Range<usize>) -> &Self::Output

Performs the indexing (container[index]) operation.

impl Index<RangeFrom<usize>> for DataFrame

type Output = [Column]

The returned type after indexing.

fn index(&self, index: RangeFrom<usize>) -> &Self::Output

Performs the indexing (container[index]) operation.

impl Index<RangeFull> for DataFrame

type Output = [Column]

The returned type after indexing.

fn index(&self, index: RangeFull) -> &Self::Output

Performs the indexing (container[index]) operation.

impl Index<RangeInclusive<usize>> for DataFrame

type Output = [Column]

The returned type after indexing.

fn index(&self, index: RangeInclusive<usize>) -> &Self::Output

Performs the indexing (container[index]) operation.

impl Index<RangeTo<usize>> for DataFrame

type Output = [Column]

The returned type after indexing.

fn index(&self, index: RangeTo<usize>) -> &Self::Output

Performs the indexing (container[index]) operation.

impl Index<RangeToInclusive<usize>> for DataFrame

type Output = [Column]

The returned type after indexing.

fn index(&self, index: RangeToInclusive<usize>) -> &Self::Output

Performs the indexing (container[index]) operation.

impl Index<usize> for DataFrame

type Output = Column

The returned type after indexing.

fn index(&self, index: usize) -> &Self::Output

Performs the indexing (container[index]) operation.

impl Mul<&DataFrame> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the * operator.

fn mul(self, rhs: &DataFrame) -> Self::Output

Performs the * operation.

impl Mul<&Series> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the * operator.

fn mul(self, rhs: &Series) -> Self::Output

Performs the * operation.

impl Mul<&Series> for DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the * operator.

fn mul(self, rhs: &Series) -> Self::Output

Performs the * operation.

impl PartialEq for DataFrame

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Rem<&DataFrame> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the % operator.

fn rem(self, rhs: &DataFrame) -> Self::Output

Performs the % operation.

impl Rem<&Series> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the % operator.

fn rem(self, rhs: &Series) -> Self::Output

Performs the % operation.

impl Rem<&Series> for DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the % operator.

fn rem(self, rhs: &Series) -> Self::Output

Performs the % operation.

impl Sub<&DataFrame> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the - operator.

fn sub(self, rhs: &DataFrame) -> Self::Output

Performs the - operation.

impl Sub<&Series> for &DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the - operator.

fn sub(self, rhs: &Series) -> Self::Output

Performs the - operation.

impl Sub<&Series> for DataFrame

Available on crate feature dataframe_arithmetic only.

type Output = Result<DataFrame, PolarsError>

The resulting type after applying the - operator.

fn sub(self, rhs: &Series) -> Self::Output

Performs the - operation.

impl TryExtend<RecordBatchT<Box<dyn Array>>> for DataFrame

fn try_extend<I: IntoIterator<Item = RecordBatchT<Box<dyn Array>>>>( &mut self, iter: I, ) -> PolarsResult<()>

Fallible version of Extend::extend.

impl TryExtend<Result<RecordBatchT<Box<dyn Array>>, PolarsError>> for DataFrame

fn try_extend<I: IntoIterator<Item = PolarsResult<RecordBatchT<Box<dyn Array>>>>>( &mut self, iter: I, ) -> PolarsResult<()>

Fallible version of Extend::extend.

impl TryFrom<(RecordBatchT<Box<dyn Array>>, &Schema<Field>)> for DataFrame

type Error = PolarsError

The type returned in the event of a conversion error.

fn try_from(arg: (RecordBatch, &ArrowSchema)) -> PolarsResult<DataFrame>

Performs the conversion.

impl TryFrom<StructArray> for DataFrame

type Error = PolarsError

The type returned in the event of a conversion error.

fn try_from(arr: StructArray) -> PolarsResult<Self>

Performs the conversion.