Struct vortex_datafusion::persistent::format::VortexFormat
source · pub struct VortexFormat { /* private fields */ }
Implementations§
Trait Implementations§
source§impl Debug for VortexFormat
impl Debug for VortexFormat
source§impl Default for VortexFormat
impl Default for VortexFormat
source§impl FileFormat for VortexFormat
impl FileFormat for VortexFormat
source§fn as_any(&self) -> &dyn Any
fn as_any(&self) -> &dyn Any
Returns the table provider as
Any
so that it can be
downcast to a specific implementation.source§fn get_ext_with_compression(
&self,
file_compression_type: &FileCompressionType,
) -> DFResult<String>
fn get_ext_with_compression( &self, file_compression_type: &FileCompressionType, ) -> DFResult<String>
Returns the extension for this FileFormat when compressed, e.g. “file.csv.gz” -> csv
source§fn infer_schema<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
_state: &'life1 SessionState,
store: &'life2 Arc<dyn ObjectStore>,
objects: &'life3 [ObjectMeta],
) -> Pin<Box<dyn Future<Output = DFResult<SchemaRef>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
fn infer_schema<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
_state: &'life1 SessionState,
store: &'life2 Arc<dyn ObjectStore>,
objects: &'life3 [ObjectMeta],
) -> Pin<Box<dyn Future<Output = DFResult<SchemaRef>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
Infer the common schema of the provided objects. The objects will usually
be analysed up to a given number of records or files (as specified in the
format config) then give the estimated common schema. This might fail if
the files have schemas that cannot be merged.
source§fn infer_stats<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
_state: &'life1 SessionState,
store: &'life2 Arc<dyn ObjectStore>,
table_schema: SchemaRef,
object: &'life3 ObjectMeta,
) -> Pin<Box<dyn Future<Output = DFResult<Statistics>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
fn infer_stats<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
_state: &'life1 SessionState,
store: &'life2 Arc<dyn ObjectStore>,
table_schema: SchemaRef,
object: &'life3 ObjectMeta,
) -> Pin<Box<dyn Future<Output = DFResult<Statistics>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
Infer the statistics for the provided object. The cost and accuracy of the
estimated statistics might vary greatly between file formats. Read more
source§fn create_physical_plan<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
_state: &'life1 SessionState,
file_scan_config: FileScanConfig,
filters: Option<&'life2 Arc<dyn PhysicalExpr>>,
) -> Pin<Box<dyn Future<Output = DFResult<Arc<dyn ExecutionPlan>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
fn create_physical_plan<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
_state: &'life1 SessionState,
file_scan_config: FileScanConfig,
filters: Option<&'life2 Arc<dyn PhysicalExpr>>,
) -> Pin<Box<dyn Future<Output = DFResult<Arc<dyn ExecutionPlan>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Take a list of files and convert it to the appropriate executor
according to this file format.
source§fn create_writer_physical_plan<'life0, 'life1, 'async_trait>(
&'life0 self,
_input: Arc<dyn ExecutionPlan>,
_state: &'life1 SessionState,
_conf: FileSinkConfig,
_order_requirements: Option<LexRequirement>,
) -> Pin<Box<dyn Future<Output = DFResult<Arc<dyn ExecutionPlan>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
fn create_writer_physical_plan<'life0, 'life1, 'async_trait>(
&'life0 self,
_input: Arc<dyn ExecutionPlan>,
_state: &'life1 SessionState,
_conf: FileSinkConfig,
_order_requirements: Option<LexRequirement>,
) -> Pin<Box<dyn Future<Output = DFResult<Arc<dyn ExecutionPlan>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
Take a list of files and the configuration to convert it to the
appropriate writer executor according to this file format.
source§fn supports_filters_pushdown(
&self,
_file_schema: &Schema,
table_schema: &Schema,
filters: &[&Expr],
) -> DFResult<FilePushdownSupport>
fn supports_filters_pushdown( &self, _file_schema: &Schema, table_schema: &Schema, filters: &[&Expr], ) -> DFResult<FilePushdownSupport>
Check if the specified file format has support for pushing down the provided filters within
the given schemas. Added initially to support the Parquet file format’s ability to do this.
Auto Trait Implementations§
impl Freeze for VortexFormat
impl !RefUnwindSafe for VortexFormat
impl Send for VortexFormat
impl Sync for VortexFormat
impl Unpin for VortexFormat
impl !UnwindSafe for VortexFormat
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoEither for T
impl<T> IntoEither for T
source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read more