IndexTables 0.5.5 continues the shift toward a fully native execution stack.
The headline change is a complete reimplementation of the transaction log in Rust, replacing the previous Scala-based design with a native module built on Arrow FFI. This release also introduces a DataSource short name and adds built-in profiling for the FFI read path.
IndexTables 0.5.4 introduces streaming companion synchronization, unified native memory management, and Arrow FFI across all data paths. Together these changes simplify the architecture, reduce serialization overhead, and make IndexTables behave more like a native Spark component rather than an external indexing engine.
More importantly, this release continues a broader shift in IndexTables toward incremental lakehouse indexing and fully columnar execution. Companion indexes can now stay synchronized with table changes continuously, while Arrow-native data movement and unified memory management allow Spark to treat indexing and search workloads as first-class citizens inside the execution engine.
This release also adds a streaming columnar reader shared across all split types and introduces a new complete read mode designed for ETL workloads.