2 posts tagged with "companion-mode"

IndexTables 0.5.4 introduces streaming companion synchronization, unified native memory management, and Arrow FFI across all data paths. Together these changes simplify the architecture, reduce serialization overhead, and make IndexTables behave more like a native Spark component rather than an external indexing engine.

More importantly, this release continues a broader shift in IndexTables toward incremental lakehouse indexing and fully columnar execution. Companion indexes can now stay synchronized with table changes continuously, while Arrow-native data movement and unified memory management allow Spark to treat indexing and search workloads as first-class citizens inside the execution engine.

This release also adds a streaming columnar reader shared across all split types and introduces a new complete read mode designed for ETL workloads.

Every database has indexes.

You don’t copy the table into the index.
You don’t govern it separately.
You don’t build ETL pipelines to keep it in sync.

An index is an acceleration structure, not a second source of truth.

So why does every search platform ask you to copy your data first?

With IndexTables 0.5.0, that changes.

Introducing Companion Mode — a fundamentally new way to add full-text search to your existing Delta Lake tables, Apache Iceberg tables, or raw Parquet datasets, without duplicating your data.