Struct vortex_array::ArrayData
source · pub struct ArrayData(/* private fields */);
Expand description
A central type for all Vortex arrays, which are known length sequences of typed and possibly compressed data.
This is the main entrypoint for working with in-memory Vortex data, and dispatches work over the underlying encoding or memory representations.
Implementations§
source§impl ArrayData
impl ArrayData
pub fn try_new_owned( encoding: EncodingRef, dtype: DType, len: usize, metadata: Arc<dyn ArrayMetadata>, buffer: Option<Buffer>, children: Arc<[ArrayData]>, statistics: StatsSet, ) -> VortexResult<Self>
pub fn try_new_viewed<F>( ctx: Arc<Context>, dtype: DType, len: usize, flatbuffer: Buffer, flatbuffer_init: F, buffers: Vec<Buffer>, ) -> VortexResult<Self>
sourcepub fn encoding(&self) -> EncodingRef
pub fn encoding(&self) -> EncodingRef
Return the array’s encoding
sourcepub fn is_canonical(&self) -> bool
pub fn is_canonical(&self) -> bool
Whether the array is of a canonical encoding.
sourcepub fn is_arrow(&self) -> bool
pub fn is_arrow(&self) -> bool
Whether the array is fully zero-copy to Arrow (including children). This means any nested types, like Structs, Lists, and Extensions are not present.
sourcepub fn is_constant(&self) -> bool
pub fn is_constant(&self) -> bool
Return whether the array is constant.
sourcepub fn as_constant(&self) -> Option<Scalar>
pub fn as_constant(&self) -> Option<Scalar>
Return scalar value of this array if the array is constant
pub fn child<'a>( &'a self, idx: usize, dtype: &'a DType, len: usize, ) -> VortexResult<Self>
sourcepub fn children(&self) -> Vec<ArrayData>
pub fn children(&self) -> Vec<ArrayData>
Returns a Vec of Arrays with all the array’s child arrays.
sourcepub fn named_children(&self) -> Vec<(String, ArrayData)>
pub fn named_children(&self) -> Vec<(String, ArrayData)>
Returns a Vec of Arrays with all the array’s child arrays.
pub fn depth_first_traversal(&self) -> ArrayChildrenIterator ⓘ
sourcepub fn cumulative_nbuffers(&self) -> usize
pub fn cumulative_nbuffers(&self) -> usize
Count the number of cumulative buffers encoded by self.
sourcepub fn all_buffer_offsets(&self, alignment: usize) -> Vec<u64>
pub fn all_buffer_offsets(&self, alignment: usize) -> Vec<u64>
Return the buffer offsets and the total length of all buffers, assuming the given alignment. This includes all child buffers.
pub fn array_metadata(&self) -> &dyn ArrayMetadata
pub fn metadata<M: ArrayMetadata + Clone + for<'m> TryDeserializeArrayMetadata<'m>>( &self, ) -> VortexResult<&M>
sourcepub fn metadata_bytes(&self) -> VortexResult<Cow<'_, [u8]>>
pub fn metadata_bytes(&self) -> VortexResult<Cow<'_, [u8]>>
Get back the (possibly owned) metadata for the array.
View arrays will return a reference to their bytes, while heap-backed arrays must first serialize their metadata, returning an owned byte array to the caller.
pub fn buffer(&self) -> Option<&Buffer>
pub fn into_buffer(self) -> Option<Buffer>
pub fn into_array_iterator(self) -> impl ArrayIterator
pub fn into_array_stream(self) -> impl ArrayStream
sourcepub fn is_encoding(&self, id: EncodingId) -> bool
pub fn is_encoding(&self, id: EncodingId) -> bool
Checks whether array is of a given encoding.
source§impl ArrayData
impl ArrayData
pub fn tree_display(&self) -> TreeDisplayWrapper<'_>
source§impl ArrayData
impl ArrayData
Provide functions on type-erased ArrayData to downcast into dtype-specific array variants.
pub fn as_null_array(&self) -> Option<&dyn NullArrayTrait>
pub fn as_bool_array(&self) -> Option<&dyn BoolArrayTrait>
pub fn as_primitive_array(&self) -> Option<&dyn PrimitiveArrayTrait>
pub fn as_utf8_array(&self) -> Option<&dyn Utf8ArrayTrait>
pub fn as_binary_array(&self) -> Option<&dyn BinaryArrayTrait>
pub fn as_struct_array(&self) -> Option<&dyn StructArrayTrait>
pub fn as_list_array(&self) -> Option<&dyn ListArrayTrait>
pub fn as_extension_array(&self) -> Option<&dyn ExtensionArrayTrait>
Trait Implementations§
source§impl<'a> Arbitrary<'a> for ArrayData
impl<'a> Arbitrary<'a> for ArrayData
source§fn arbitrary(u: &mut Unstructured<'a>) -> Result<Self>
fn arbitrary(u: &mut Unstructured<'a>) -> Result<Self>
Self
from the given unstructured data. Read more§fn arbitrary_take_rest(u: Unstructured<'a>) -> Result<Self, Error>
fn arbitrary_take_rest(u: Unstructured<'a>) -> Result<Self, Error>
Self
from the entirety of the given
unstructured data. Read moresource§impl AsRef<ArrayData> for ChunkedArray
impl AsRef<ArrayData> for ChunkedArray
source§impl AsRef<ArrayData> for ConstantArray
impl AsRef<ArrayData> for ConstantArray
source§impl AsRef<ArrayData> for ExtensionArray
impl AsRef<ArrayData> for ExtensionArray
source§impl AsRef<ArrayData> for PrimitiveArray
impl AsRef<ArrayData> for PrimitiveArray
source§impl AsRef<ArrayData> for SparseArray
impl AsRef<ArrayData> for SparseArray
source§impl AsRef<ArrayData> for StructArray
impl AsRef<ArrayData> for StructArray
source§impl AsRef<ArrayData> for VarBinArray
impl AsRef<ArrayData> for VarBinArray
source§impl AsRef<ArrayData> for VarBinViewArray
impl AsRef<ArrayData> for VarBinViewArray
source§impl<E> BinaryBooleanFn<ArrayData> for E
impl<E> BinaryBooleanFn<ArrayData> for E
fn binary_boolean( &self, lhs: &ArrayData, rhs: &ArrayData, op: BinaryOperator, ) -> VortexResult<Option<ArrayData>>
source§impl<E> BinaryNumericFn<ArrayData> for E
impl<E> BinaryNumericFn<ArrayData> for E
source§impl<E> FillForwardFn<ArrayData> for E
impl<E> FillForwardFn<ArrayData> for E
fn fill_forward(&self, array: &ArrayData) -> VortexResult<ArrayData>
source§impl<E> FillNullFn<ArrayData> for E
impl<E> FillNullFn<ArrayData> for E
source§impl From<BooleanBuffer> for ArrayData
impl From<BooleanBuffer> for ArrayData
source§fn from(value: BooleanBuffer) -> Self
fn from(value: BooleanBuffer) -> Self
source§impl From<Canonical> for ArrayData
impl From<Canonical> for ArrayData
This conversion is always “free” and should not touch underlying data. All it does is create an owned pointer to the underlying concrete array type.
This combined with the above IntoCanonical impl for ArrayData allows simple two-way conversions between arbitrary Vortex encodings and canonical Arrow-compatible encodings.
source§impl From<TemporalArray> for ArrayData
impl From<TemporalArray> for ArrayData
source§fn from(value: TemporalArray) -> Self
fn from(value: TemporalArray) -> Self
source§impl FromArrowArray<&BooleanArray> for ArrayData
impl FromArrowArray<&BooleanArray> for ArrayData
fn from_arrow(value: &ArrowBooleanArray, nullable: bool) -> Self
source§impl<T: ByteArrayType> FromArrowArray<&GenericByteArray<T>> for ArrayDatawhere
<T as ByteArrayType>::Offset: NativePType,
impl<T: ByteArrayType> FromArrowArray<&GenericByteArray<T>> for ArrayDatawhere
<T as ByteArrayType>::Offset: NativePType,
fn from_arrow(value: &GenericByteArray<T>, nullable: bool) -> Self
source§impl<T: ByteViewType> FromArrowArray<&GenericByteViewArray<T>> for ArrayData
impl<T: ByteViewType> FromArrowArray<&GenericByteViewArray<T>> for ArrayData
fn from_arrow(value: &GenericByteViewArray<T>, nullable: bool) -> Self
source§impl<O: OffsetSizeTrait + NativePType> FromArrowArray<&GenericListArray<O>> for ArrayData
impl<O: OffsetSizeTrait + NativePType> FromArrowArray<&GenericListArray<O>> for ArrayData
fn from_arrow(value: &GenericListArray<O>, nullable: bool) -> Self
source§impl FromArrowArray<&NullArray> for ArrayData
impl FromArrowArray<&NullArray> for ArrayData
fn from_arrow(value: &ArrowNullArray, nullable: bool) -> Self
source§impl<T: ArrowPrimitiveType> FromArrowArray<&PrimitiveArray<T>> for ArrayDatawhere
<T as ArrowPrimitiveType>::Native: NativePType,
impl<T: ArrowPrimitiveType> FromArrowArray<&PrimitiveArray<T>> for ArrayDatawhere
<T as ArrowPrimitiveType>::Native: NativePType,
fn from_arrow(value: &ArrowPrimitiveArray<T>, nullable: bool) -> Self
source§impl FromArrowArray<&StructArray> for ArrayData
impl FromArrowArray<&StructArray> for ArrayData
fn from_arrow(value: &ArrowStructArray, nullable: bool) -> Self
source§impl FromArrowArray<Arc<dyn Array>> for ArrayData
impl FromArrowArray<Arc<dyn Array>> for ArrayData
fn from_arrow(array: ArrowArrayRef, nullable: bool) -> Self
source§impl FromIterator<ArrayData> for ChunkedArray
impl FromIterator<ArrayData> for ChunkedArray
source§impl FromIterator<Option<Buffer>> for ArrayData
impl FromIterator<Option<Buffer>> for ArrayData
source§impl FromIterator<Option<BufferString>> for ArrayData
impl FromIterator<Option<BufferString>> for ArrayData
source§impl IndexOrd<Scalar> for ArrayData
impl IndexOrd<Scalar> for ArrayData
source§fn index_cmp(&self, idx: usize, elem: &Scalar) -> Option<Ordering>
fn index_cmp(&self, idx: usize, elem: &Scalar) -> Option<Ordering>
idx
with elem
.
For example, if self[idx] > elem, return Some(Greater).fn index_lt(&self, idx: usize, elem: &V) -> bool
fn index_le(&self, idx: usize, elem: &V) -> bool
fn index_gt(&self, idx: usize, elem: &V) -> bool
fn index_ge(&self, idx: usize, elem: &V) -> bool
source§impl IntoCanonical for ArrayData
impl IntoCanonical for ArrayData
IntoCanonical implementation for Array.
Canonicalizing an array requires potentially decompressing, so this requires a roundtrip through the array’s internal codec.
fn into_canonical(self) -> VortexResult<Canonical>
fn into_arrow(self) -> VortexResult<ArrayRef>
source§impl<E> ScalarAtFn<ArrayData> for E
impl<E> ScalarAtFn<ArrayData> for E
source§impl<E> SearchSortedFn<ArrayData> for E
impl<E> SearchSortedFn<ArrayData> for E
fn search_sorted( &self, array: &ArrayData, value: &Scalar, side: SearchSortedSide, ) -> VortexResult<SearchResult>
source§fn search_sorted_many(
&self,
array: &ArrayData,
values: &[Scalar],
side: SearchSortedSide,
) -> VortexResult<Vec<SearchResult>>
fn search_sorted_many( &self, array: &ArrayData, values: &[Scalar], side: SearchSortedSide, ) -> VortexResult<Vec<SearchResult>>
source§impl<E> SearchSortedUsizeFn<ArrayData> for E
impl<E> SearchSortedUsizeFn<ArrayData> for E
fn search_sorted_usize( &self, array: &ArrayData, value: usize, side: SearchSortedSide, ) -> VortexResult<SearchResult>
fn search_sorted_usize_many( &self, array: &ArrayData, values: &[usize], side: SearchSortedSide, ) -> VortexResult<Vec<SearchResult>>
source§impl<E> SliceFn<ArrayData> for E
impl<E> SliceFn<ArrayData> for E
source§fn slice(
&self,
array: &ArrayData,
start: usize,
stop: usize,
) -> VortexResult<ArrayData>
fn slice( &self, array: &ArrayData, start: usize, stop: usize, ) -> VortexResult<ArrayData>
start
(inclusive) and end
(exclusive).
If start >= stop, returns an empty array of the same type as self
.
Assumes that start or stop are out of bounds, may panic otherwise.source§impl<E> StatisticsVTable<ArrayData> for E
impl<E> StatisticsVTable<ArrayData> for E
source§impl StatisticsVTable<ArrayData> for OpaqueEncoding
impl StatisticsVTable<ArrayData> for OpaqueEncoding
source§impl<E> TakeFn<ArrayData> for E
impl<E> TakeFn<ArrayData> for E
source§impl<'a> TryFrom<&'a ArrayData> for &'a ChunkedArray
impl<'a> TryFrom<&'a ArrayData> for &'a ChunkedArray
source§impl<'a> TryFrom<&'a ArrayData> for &'a ConstantArray
impl<'a> TryFrom<&'a ArrayData> for &'a ConstantArray
source§impl<'a> TryFrom<&'a ArrayData> for &'a ExtensionArray
impl<'a> TryFrom<&'a ArrayData> for &'a ExtensionArray
source§impl<'a> TryFrom<&'a ArrayData> for &'a PrimitiveArray
impl<'a> TryFrom<&'a ArrayData> for &'a PrimitiveArray
source§impl<'a> TryFrom<&'a ArrayData> for &'a SparseArray
impl<'a> TryFrom<&'a ArrayData> for &'a SparseArray
source§impl<'a> TryFrom<&'a ArrayData> for &'a StructArray
impl<'a> TryFrom<&'a ArrayData> for &'a StructArray
source§impl<'a> TryFrom<&'a ArrayData> for &'a VarBinArray
impl<'a> TryFrom<&'a ArrayData> for &'a VarBinArray
source§impl<'a> TryFrom<&'a ArrayData> for &'a VarBinViewArray
impl<'a> TryFrom<&'a ArrayData> for &'a VarBinViewArray
source§impl TryFrom<ArrayData> for ChunkedArray
impl TryFrom<ArrayData> for ChunkedArray
source§impl TryFrom<ArrayData> for ConstantArray
impl TryFrom<ArrayData> for ConstantArray
source§impl TryFrom<ArrayData> for ExtensionArray
impl TryFrom<ArrayData> for ExtensionArray
source§impl TryFrom<ArrayData> for FilterMask
impl TryFrom<ArrayData> for FilterMask
source§impl TryFrom<ArrayData> for LogicalValidity
impl TryFrom<ArrayData> for LogicalValidity
source§impl TryFrom<ArrayData> for PrimitiveArray
impl TryFrom<ArrayData> for PrimitiveArray
source§impl TryFrom<ArrayData> for SparseArray
impl TryFrom<ArrayData> for SparseArray
source§impl TryFrom<ArrayData> for StructArray
impl TryFrom<ArrayData> for StructArray
source§impl TryFrom<ArrayData> for TemporalArray
impl TryFrom<ArrayData> for TemporalArray
source§fn try_from(value: ArrayData) -> Result<Self, Self::Error>
fn try_from(value: ArrayData) -> Result<Self, Self::Error>
Try to specialize a generic Vortex array as a TemporalArray.
§Errors
If the provided Array does not have vortex.ext
encoding, an error will be returned.
If the provided Array does not have recognized ExtMetadata corresponding to one of the known
TemporalMetadata
variants, an error is returned.
source§impl TryFrom<ArrayData> for VarBinArray
impl TryFrom<ArrayData> for VarBinArray
source§impl TryFrom<ArrayData> for VarBinViewArray
impl TryFrom<ArrayData> for VarBinViewArray
source§impl<E> ValidityVTable<ArrayData> for E
impl<E> ValidityVTable<ArrayData> for E
source§impl ValidityVTable<ArrayData> for OpaqueEncoding
impl ValidityVTable<ArrayData> for OpaqueEncoding
source§impl<E> VariantsVTable<ArrayData> for E
impl<E> VariantsVTable<ArrayData> for E
fn as_null_array<'a>( &self, array: &'a ArrayData, ) -> Option<&'a dyn NullArrayTrait>
fn as_bool_array<'a>( &self, array: &'a ArrayData, ) -> Option<&'a dyn BoolArrayTrait>
fn as_primitive_array<'a>( &self, array: &'a ArrayData, ) -> Option<&'a dyn PrimitiveArrayTrait>
fn as_utf8_array<'a>( &self, array: &'a ArrayData, ) -> Option<&'a dyn Utf8ArrayTrait>
fn as_binary_array<'a>( &self, array: &'a ArrayData, ) -> Option<&'a dyn BinaryArrayTrait>
fn as_struct_array<'a>( &self, array: &'a ArrayData, ) -> Option<&'a dyn StructArrayTrait>
fn as_list_array<'a>( &self, array: &'a ArrayData, ) -> Option<&'a dyn ListArrayTrait>
fn as_extension_array<'a>( &self, array: &'a ArrayData, ) -> Option<&'a dyn ExtensionArrayTrait>
source§impl VariantsVTable<ArrayData> for OpaqueEncoding
impl VariantsVTable<ArrayData> for OpaqueEncoding
fn as_null_array<'a>(&self, _array: &'a Array) -> Option<&'a dyn NullArrayTrait>
fn as_bool_array<'a>(&self, _array: &'a Array) -> Option<&'a dyn BoolArrayTrait>
fn as_primitive_array<'a>( &self, _array: &'a Array, ) -> Option<&'a dyn PrimitiveArrayTrait>
fn as_utf8_array<'a>(&self, _array: &'a Array) -> Option<&'a dyn Utf8ArrayTrait>
fn as_binary_array<'a>( &self, _array: &'a Array, ) -> Option<&'a dyn BinaryArrayTrait>
fn as_struct_array<'a>( &self, _array: &'a Array, ) -> Option<&'a dyn StructArrayTrait>
fn as_list_array<'a>(&self, _array: &'a Array) -> Option<&'a dyn ListArrayTrait>
fn as_extension_array<'a>( &self, _array: &'a Array, ) -> Option<&'a dyn ExtensionArrayTrait>
source§impl<E> VisitorVTable<ArrayData> for E
impl<E> VisitorVTable<ArrayData> for E
source§impl VisitorVTable<ArrayData> for OpaqueEncoding
impl VisitorVTable<ArrayData> for OpaqueEncoding
Auto Trait Implementations§
impl !Freeze for ArrayData
impl !RefUnwindSafe for ArrayData
impl Send for ArrayData
impl Sync for ArrayData
impl Unpin for ArrayData
impl !UnwindSafe for ArrayData
Blanket Implementations§
source§impl<T> ArrayEncodingRef for T
impl<T> ArrayEncodingRef for T
fn encoding(&self) -> &'static dyn EncodingVTable
source§impl<A> ArrayNBytes for A
impl<A> ArrayNBytes for A
source§impl<T> ArrayStatistics for T
impl<T> ArrayStatistics for T
fn statistics(&self) -> &dyn Statistics
fn inherit_statistics(&self, parent: &dyn Statistics)
source§impl<A> ArrayValidity for A
impl<A> ArrayValidity for A
source§fn is_valid(&self, index: usize) -> bool
fn is_valid(&self, index: usize) -> bool
Return whether the element at the given index is valid (true) or null (false).
source§fn logical_validity(&self) -> LogicalValidity
fn logical_validity(&self) -> LogicalValidity
Return the logical validity of the array.
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
source§unsafe fn clone_to_uninit(&self, dst: *mut T)
unsafe fn clone_to_uninit(&self, dst: *mut T)
clone_to_uninit
)§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoArrayVariant for Twhere
T: IntoCanonical,
impl<T> IntoArrayVariant for Twhere
T: IntoCanonical,
fn into_null(self) -> Result<NullArray, VortexError>
fn into_bool(self) -> Result<BoolArray, VortexError>
fn into_primitive(self) -> Result<PrimitiveArray, VortexError>
fn into_struct(self) -> Result<StructArray, VortexError>
fn into_list(self) -> Result<ListArray, VortexError>
fn into_varbinview(self) -> Result<VarBinViewArray, VortexError>
fn into_extension(self) -> Result<ExtensionArray, VortexError>
source§impl<T> IntoEither for T
impl<T> IntoEither for T
source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read more