Skip to main content

DictionaryColumn

Struct DictionaryColumn 

Source
pub struct DictionaryColumn { /* private fields */ }
Expand description

A String Column held as codes into a SharedDictionary: row i, when valid, is dict.values()[codes[i]].

This is lazily materialized into a Series that gathers the values.

Implementations§

Source§

impl DictionaryColumn

Source

pub unsafe fn new_unchecked( name: PlSmallStr, dict: Arc<SharedDictionary>, codes: ChunkedArray<UInt32Type>, ) -> DictionaryColumn

§Safety

Every valid code must be in bounds for dict.

Source

pub fn try_new( name: PlSmallStr, dict: Arc<SharedDictionary>, codes: ChunkedArray<UInt32Type>, ) -> Result<DictionaryColumn, PolarsError>

Source

pub fn name(&self) -> &PlSmallStr

Source

pub fn rename(&mut self, name: PlSmallStr)

Source

pub fn dtype(&self) -> &DataType

Source

pub fn len(&self) -> usize

Source

pub fn is_empty(&self) -> bool

Source

pub fn null_count(&self) -> usize

Source

pub fn has_nulls(&self) -> bool

Source

pub fn dict(&self) -> &Arc<SharedDictionary> ⓘ

Source

pub fn codes(&self) -> &ChunkedArray<UInt32Type>

Source

pub fn is_sorted_flag(&self) -> IsSorted

Source

pub fn set_sorted_flag(&mut self, sorted: IsSorted)

Source

pub fn to_series(&self) -> Series

Gathers the values into a String Series.

Source

pub fn lazy_as_materialized_series(&self) -> Option<&Series>

Get the DictionaryColumn as Series if it was already materialized.

Source

pub fn as_materialized_series(&self) -> &Series

Get the DictionaryColumn as Series

This needs to materialize upon the first call. Afterwards, this is cached.

Source

pub fn take_materialized_series(self) -> Series

Take the DictionaryColumn and materialize as a Series if not already done.

Source

pub fn slice(&self, slice: impl Into<Slice>) -> DictionaryColumn

Source

pub fn filter( &self, filter: &ChunkedArray<BooleanType>, ) -> Result<DictionaryColumn, PolarsError>

Source

pub unsafe fn take_unchecked( &self, indices: &ChunkedArray<UInt32Type>, ) -> DictionaryColumn

§Safety

The indices must be in bounds.

Source

pub unsafe fn take_slice_unchecked(&self, indices: &[u32]) -> DictionaryColumn

§Safety

The indices must be in bounds.

Source

pub fn rechunk(&self) -> DictionaryColumn

Source

pub fn reverse(&self) -> DictionaryColumn

Source

pub fn new_empty(name: PlSmallStr) -> DictionaryColumn

An empty column that holds no dictionary of its own to keep alive: it adopts the dictionary of the first column appended to it.

Source

pub fn try_append(&mut self, other: &DictionaryColumn) -> bool

Appends other’s codes. With another dictionary, other’s values that this dictionary lacks are added to it (into a new dictionary, unless there are none) and other’s codes remapped, if the merged dictionary has no more values than the rows of both, as otherwise it costs more than materializing; returns false then.

Values are matched by value, so appending the chunks of many row groups whose dictionaries share their values keeps one dictionary of the distinct values. Once the lookup table exists, remapping costs the size of other’s dictionary and codes. Extending also copies the existing dictionary’s views, lookup and cached hashes, so this method returns false when that extension would copy more existing entries than other has rows.

Source

pub fn estimated_size(&self) -> usize

Estimated size of the column, see Series::estimated_size.

The dictionary is shared by every slice of a column, so each counts the part of it its rows would take materialized, at most all of it; counting all of it for each slice would overstate the memory of the morsels of a row group many times over.

Source

pub fn used_codes(&self, max_used: usize) -> Option<(Vec<u32>, Vec<u32>)>

The codes that valid rows use, in order of first use, and the position of each row’s code among them (0 for a null row). None once more than max_used codes are used.

The time this takes depends on the rows, not on the size of the dictionary, which can be far larger than the rows need after a filter or slice.

Source

pub fn apply_elementwise( &self, f: impl FnOnce(&Series) -> Result<Series, PolarsError>, ) -> Result<Column, PolarsError>

Evaluates an elementwise f once per dictionary value that a valid row uses, and once on a null if a row is null, and spreads the results to the rows by code. A String result stays dictionary-coded. f gets a String Series of this column’s name and returns one value per input value.

Only values that rows use are evaluated, so f errors exactly when it would on the rows. When that is not fewer values than rows, f is evaluated on the materialized column. A dictionary larger than the rows (a slice or filter) is not a reason: used_codes finds the used values in time linear in the rows.

Source

pub fn check_invariants(&self)

Checks that the materialized values match the codes.

Trait Implementations§

Source§

impl Clone for DictionaryColumn

Source§

fn clone(&self) -> DictionaryColumn

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for DictionaryColumn

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more
Source§

impl From<DictionaryColumn> for Column

Source§

fn from(value: DictionaryColumn) -> Column

Converts to this type from the input type.
Source§

impl IntoColumn for DictionaryColumn

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Key for T
where T: Clone,

Source§

fn align() -> usize

The alignment necessary for the key. Must return a power of two.
Source§

fn size(&self) -> usize

The size of the key in bytes.
Source§

unsafe fn init(&self, ptr: *mut u8)

Initialize the key in the given memory location. Read more
Source§

unsafe fn get<'a>(ptr: *const u8) -> &'a T

Get a reference to the key from the given memory location. Read more
Source§

unsafe fn drop_in_place(ptr: *mut u8)

Drop the key in place. Read more
§

impl<T> PlanCallbackArgs for T

§

impl<T> PlanCallbackOut for T

§

impl<T> Pointable for T

§

const ALIGN: usize

The alignment of pointer.
§

type Init = T

The type for initializers.
§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
§

impl<T> PolicyExt for T
where T: ?Sized,

§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] only if self and other return Action::Follow. Read more
§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] if either self or other returns Action::Follow. Read more
§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more