Struct vortex_fastlanes::DeltaArray

pub struct DeltaArray(/* private fields */);

Implementations

impl DeltaArray

pub fn maybe_from(data: impl AsRef<ArrayData>) -> Option<Self>

Optionally downcast an ArrayData instance to a specific encoding.

Preferred in cases where a backtrace isn’t needed, like when trying multiple encoding to go down different code paths.

impl DeltaArray

A FastLanes-style delta-encoded array of primitive values.

A DeltaArray comprises a sequence of chunks each representing 1,024 delta-encoded values, except the last chunk which may represent from one to 1,024 values.

Examples

use vortex_fastlanes::DeltaArray;
let array = DeltaArray::try_from_vec(vec![1_u32, 2, 3, 5, 10, 11]).unwrap();

Details

To facilitate slicing, this array accepts an offset and logical_len. The offset must be strictly less than 1,024 and the sum of offset and logical_len must not exceed the length of the deltas array. These values permit logical slicing without modifying any chunk containing a kept value. In particular, we may defer decompresison until the array is canonicalized or indexed. The offset is a physical offset into the first chunk, which necessarily contains 1,024 values. The logical_len is the number of logical values following the offset, which may be less than the number of physically stored values.

Each chunk is stored as a vector of bases and a vector of deltas. If the chunk physically contains 1,024 vlaues, then there are as many bases as there are lanes of this type in a 1024-bit register. For example, for 64-bit values, there are 16 bases because there are 16 lanes. Each lane is a delta-encoding 1024 / bit_width long vector of values. The deltas are stored in the FastLanes order which splits the 1,024 values into one contiguous sub-sequence per-lane, thus permitting delta encoding.

If the chunk physically has fewer than 1,024 values, then it is stored as a traditional, non-SIMD-amenable, delta-encoded vector.

pub fn try_from_vec<T: NativePType>(vec: Vec<T>) -> VortexResult<Self>

pub fn try_from_primitive_array(array: &PrimitiveArray) -> VortexResult<Self>

pub fn try_from_delta_compress_parts( bases: ArrayData, deltas: ArrayData, validity: Validity, ) -> VortexResult<Self>

pub fn try_new( bases: ArrayData, deltas: ArrayData, validity: Validity, offset: usize, logical_len: usize, ) -> VortexResult<Self>

pub fn bases(&self) -> ArrayData

pub fn deltas(&self) -> ArrayData

pub fn offset(&self) -> usize

The logical offset into the first chunk of Self::deltas.

pub fn validity(&self) -> Validity

Trait Implementations

impl AsRef<ArrayData> for DeltaArray

fn as_ref(&self) -> &ArrayData

Converts this type into a shared reference of the (usually inferred) input type.

impl Clone for DeltaArray

fn clone(&self) -> DeltaArray

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DeltaArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl IntoArrayData for DeltaArray

fn into_array(self) -> ArrayData

impl IntoCanonical for DeltaArray

fn into_canonical(self) -> VortexResult<Canonical>

fn into_arrow(self) -> Result<Arc<dyn Array>, VortexError>

where Self: Sized,

impl PrimitiveArrayTrait for DeltaArray

fn ptype(&self) -> PType

The logical primitive type of the array.

impl ScalarAtFn<DeltaArray> for DeltaEncoding

fn scalar_at(&self, array: &DeltaArray, index: usize) -> VortexResult<Scalar>

impl SliceFn<DeltaArray> for DeltaEncoding

fn slice( &self, array: &DeltaArray, start: usize, stop: usize, ) -> VortexResult<ArrayData>

Return a zero-copy slice of an array, between start (inclusive) and end (exclusive). If start >= stop, returns an empty array of the same type as self. Assumes that start or stop are out of bounds, may panic otherwise.

impl StatisticsVTable<DeltaArray> for DeltaEncoding

fn compute_statistics( &self, _array: &Array, _stat: Stat, ) -> Result<StatsSet, VortexError>

Compute the requested statistic. Can return additional stats.

impl<'a> TryFrom<&'a ArrayData> for &'a DeltaArray

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(data: &'a ArrayData) -> VortexResult<Self>

Performs the conversion.

impl TryFrom<ArrayData> for DeltaArray

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(data: ArrayData) -> VortexResult<Self>

Performs the conversion.

impl ValidityVTable<DeltaArray> for DeltaEncoding

fn is_valid(&self, array: &DeltaArray, index: usize) -> bool

fn logical_validity(&self, array: &DeltaArray) -> LogicalValidity

impl VariantsVTable<DeltaArray> for DeltaEncoding

fn as_primitive_array<'a>( &self, array: &'a DeltaArray, ) -> Option<&'a dyn PrimitiveArrayTrait>

fn as_null_array<'a>(&self, _array: &'a Array) -> Option<&'a dyn NullArrayTrait>

fn as_bool_array<'a>(&self, _array: &'a Array) -> Option<&'a dyn BoolArrayTrait>

fn as_utf8_array<'a>(&self, _array: &'a Array) -> Option<&'a dyn Utf8ArrayTrait>

fn as_binary_array<'a>( &self, _array: &'a Array, ) -> Option<&'a dyn BinaryArrayTrait>

fn as_struct_array<'a>( &self, _array: &'a Array, ) -> Option<&'a dyn StructArrayTrait>

fn as_list_array<'a>(&self, _array: &'a Array) -> Option<&'a dyn ListArrayTrait>

fn as_extension_array<'a>( &self, _array: &'a Array, ) -> Option<&'a dyn ExtensionArrayTrait>

impl VisitorVTable<DeltaArray> for DeltaEncoding

fn accept( &self, array: &DeltaArray, visitor: &mut dyn ArrayVisitor, ) -> VortexResult<()>

impl ArrayTrait for DeltaArray