One post tagged with "streaming"

IndexTables 0.5.4 introduces streaming companion synchronization, unified native memory management, and Arrow FFI across all data paths. Together these changes simplify the architecture, reduce serialization overhead, and make IndexTables behave more like a native Spark component rather than an external indexing engine.

More importantly, this release continues a broader shift in IndexTables toward incremental lakehouse indexing and fully columnar execution. Companion indexes can now stay synchronized with table changes continuously, while Arrow-native data movement and unified memory management allow Spark to treat indexing and search workloads as first-class citizens inside the execution engine.

This release also adds a streaming columnar reader shared across all split types and introduces a new complete read mode designed for ETL workloads.