Struct vortex_alp::RDEncoder

pub struct RDEncoder { /* private fields */ }

Encoder for ALP-RD (“real doubles”) values.

The encoder calculates its parameters from a single sample of floating-point values, and then can be applied to many vectors.

ALP-RD uses the algorithm outlined in Section 3.4 of the paper. The crux of it is that the front (most significant) bits of many double vectors tend to be the same, i.e. most doubles in a vector often use the same exponent and front bits. Compression proceeds by finding the best prefix of up to 16 bits that can be collapsed into a dictionary of up to 8 elements. Each double can then be broken into the front/left L bits, which neatly bit-packs down to 1-3 bits per element (depending on the actual dictionary size). The remaining R bits naturally bit-pack.

In the ideal case, this scheme allows us to store a sequence of doubles in 49 bits-per-value.

Our implementation draws on the MIT-licensed C++ implementation provided by the original authors.

Implementations

impl RDEncoder

pub fn new<T>(sample: &[T]) -> Self

where T: ALPRDFloat + NativePType, T::UINT: NativePType,

Build a new encoder from a sample of doubles.

pub fn encode(&self, array: &PrimitiveArray) -> ALPRDArray

Encode a set of floating point values with ALP-RD.

Each value will be split into a left and right component, which are compressed individually.