pub struct DictionaryColumn { /* private fields */ }Expand description
A String Column held as codes into a SharedDictionary: row i, when valid, is
dict.values()[codes[i]].
This is lazily materialized into a Series that gathers the values.
Implementations§
Source§impl DictionaryColumn
impl DictionaryColumn
Sourcepub unsafe fn new_unchecked(
name: PlSmallStr,
dict: Arc<SharedDictionary>,
codes: ChunkedArray<UInt32Type>,
) -> DictionaryColumn
pub unsafe fn new_unchecked( name: PlSmallStr, dict: Arc<SharedDictionary>, codes: ChunkedArray<UInt32Type>, ) -> DictionaryColumn
§Safety
Every valid code must be in bounds for dict.
pub fn try_new( name: PlSmallStr, dict: Arc<SharedDictionary>, codes: ChunkedArray<UInt32Type>, ) -> Result<DictionaryColumn, PolarsError>
pub fn name(&self) -> &PlSmallStr
pub fn rename(&mut self, name: PlSmallStr)
pub fn dtype(&self) -> &DataType
pub fn len(&self) -> usize
pub fn is_empty(&self) -> bool
pub fn null_count(&self) -> usize
pub fn has_nulls(&self) -> bool
pub fn dict(&self) -> &Arc<SharedDictionary> ⓘ
pub fn codes(&self) -> &ChunkedArray<UInt32Type>
pub fn is_sorted_flag(&self) -> IsSorted
pub fn set_sorted_flag(&mut self, sorted: IsSorted)
Sourcepub fn lazy_as_materialized_series(&self) -> Option<&Series>
pub fn lazy_as_materialized_series(&self) -> Option<&Series>
Get the DictionaryColumn as Series if it was already materialized.
Sourcepub fn as_materialized_series(&self) -> &Series
pub fn as_materialized_series(&self) -> &Series
Get the DictionaryColumn as Series
This needs to materialize upon the first call. Afterwards, this is cached.
Sourcepub fn take_materialized_series(self) -> Series
pub fn take_materialized_series(self) -> Series
Take the DictionaryColumn and materialize as a Series if not already done.
pub fn slice(&self, slice: impl Into<Slice>) -> DictionaryColumn
pub fn filter( &self, filter: &ChunkedArray<BooleanType>, ) -> Result<DictionaryColumn, PolarsError>
Sourcepub unsafe fn take_unchecked(
&self,
indices: &ChunkedArray<UInt32Type>,
) -> DictionaryColumn
pub unsafe fn take_unchecked( &self, indices: &ChunkedArray<UInt32Type>, ) -> DictionaryColumn
§Safety
The indices must be in bounds.
Sourcepub unsafe fn take_slice_unchecked(&self, indices: &[u32]) -> DictionaryColumn
pub unsafe fn take_slice_unchecked(&self, indices: &[u32]) -> DictionaryColumn
§Safety
The indices must be in bounds.
pub fn rechunk(&self) -> DictionaryColumn
pub fn reverse(&self) -> DictionaryColumn
Sourcepub fn new_empty(name: PlSmallStr) -> DictionaryColumn
pub fn new_empty(name: PlSmallStr) -> DictionaryColumn
An empty column that holds no dictionary of its own to keep alive: it adopts the dictionary of the first column appended to it.
Sourcepub fn try_append(&mut self, other: &DictionaryColumn) -> bool
pub fn try_append(&mut self, other: &DictionaryColumn) -> bool
Appends other’s codes. With another dictionary, other’s values that this dictionary
lacks are added to it (into a new dictionary, unless there are none) and other’s codes
remapped, if the merged dictionary has no more values than the rows of both, as otherwise
it costs more than materializing; returns false then.
Values are matched by value, so appending the chunks of many row groups whose dictionaries
share their values keeps one dictionary of the distinct values. Once the lookup table
exists, remapping costs the size of other’s dictionary and codes. Extending also copies
the existing dictionary’s views, lookup and cached hashes, so this method returns false
when that extension would copy more existing entries than other has rows.
Sourcepub fn estimated_size(&self) -> usize
pub fn estimated_size(&self) -> usize
Estimated size of the column, see Series::estimated_size.
The dictionary is shared by every slice of a column, so each counts the part of it its rows would take materialized, at most all of it; counting all of it for each slice would overstate the memory of the morsels of a row group many times over.
Sourcepub fn used_codes(&self, max_used: usize) -> Option<(Vec<u32>, Vec<u32>)>
pub fn used_codes(&self, max_used: usize) -> Option<(Vec<u32>, Vec<u32>)>
The codes that valid rows use, in order of first use, and the position of each row’s code
among them (0 for a null row). None once more than max_used codes are used.
The time this takes depends on the rows, not on the size of the dictionary, which can be far larger than the rows need after a filter or slice.
Sourcepub fn apply_elementwise(
&self,
f: impl FnOnce(&Series) -> Result<Series, PolarsError>,
) -> Result<Column, PolarsError>
pub fn apply_elementwise( &self, f: impl FnOnce(&Series) -> Result<Series, PolarsError>, ) -> Result<Column, PolarsError>
Evaluates an elementwise f once per dictionary value that a valid row uses, and once on
a null if a row is null, and spreads the results to the rows by code. A String result stays
dictionary-coded. f gets a String Series of this column’s name and returns one value
per input value.
Only values that rows use are evaluated, so f errors exactly when it would on the rows.
When that is not fewer values than rows, f is evaluated on the materialized column. A
dictionary larger than the rows (a slice or filter) is not a reason: used_codes finds the
used values in time linear in the rows.
Sourcepub fn check_invariants(&self)
pub fn check_invariants(&self)
Checks that the materialized values match the codes.
Trait Implementations§
Source§impl Clone for DictionaryColumn
impl Clone for DictionaryColumn
Source§fn clone(&self) -> DictionaryColumn
fn clone(&self) -> DictionaryColumn
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for DictionaryColumn
impl Debug for DictionaryColumn
Source§impl From<DictionaryColumn> for Column
impl From<DictionaryColumn> for Column
Source§fn from(value: DictionaryColumn) -> Column
fn from(value: DictionaryColumn) -> Column
Source§impl IntoColumn for DictionaryColumn
impl IntoColumn for DictionaryColumn
fn into_column(self) -> Column
Auto Trait Implementations§
impl !Freeze for DictionaryColumn
impl !RefUnwindSafe for DictionaryColumn
impl !UnwindSafe for DictionaryColumn
impl Send for DictionaryColumn
impl Sync for DictionaryColumn
impl Unpin for DictionaryColumn
impl UnsafeUnpin for DictionaryColumn
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more