Each test is 5 questions with varying difficulty.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.
Vector index rebuilding and compaction are critical operations in modern vector databases, ensuring that Approximate Nearest Neighbor (ANN) search remains performant as datasets evolve. As data is inserted, deleted, or updated, indices like HNSW (Hierarchical Navigable Small World) or IVF (Inverted File Index) accumulate 'staleness'—a state where the index structure no longer optimally represents the underlying vector distribution. In 2026, with the explosion of real-time RAG applications, maintaining search accuracy and low latency during these background maintenance tasks is a core requirement for AI engineers and database architects. Interviewers focus on this topic to assess a candidate's understanding of how storage engines manage memory, disk I/O, and concurrency. Junior candidates are expected to understand the basic need for compaction to reclaim space and improve recall, while senior candidates must demonstrate expertise in zero-downtime strategies, such as log-structured merge-tree (LSM) approaches for vectors, segment merging, and the trade-offs between background task resource consumption and query latency.
In production environments, vector indices are rarely static. As new documents are ingested, the index structure becomes fragmented, leading to degraded recall and increased latency. Compaction is the process of merging smaller index segments into larger, more optimized structures, while rebuilding is the full reconstruction of the graph or partition. Without efficient compaction, a vector database can experience a 'performance cliff' where search latency spikes due to suboptimal traversal paths in the HNSW graph. For large-scale systems, performing these operations without downtime is a significant engineering challenge. A strong candidate must explain how to balance background compaction throughput with foreground query performance, often involving techniques like read-copy-update or multi-versioning. This topic is high-signal because it forces candidates to move beyond black-box usage of vector databases and into the internals of how data is physically laid out, indexed, and maintained under high-concurrency write loads. In 2026, as vector databases move from niche tools to enterprise-grade infrastructure, the ability to architect these maintenance pipelines is a key differentiator for senior engineering talent.
The vector database architecture typically employs a tiered storage model where incoming vectors are first written to an immutable segment. These segments are periodically compacted by a background worker into larger, optimized indices (like HNSW graphs). Queries are routed through a search coordinator that performs a scatter-gather operation across both active segments and the compacted index, merging results before returning them to the user.
Incoming vectors are logged to the WAL and stored in a memory buffer. Once the buffer reaches a threshold, it is flushed to an immutable segment. The background compactor periodically selects these segments, merges them into a new, larger index, and replaces the old segments. The search coordinator queries all segments and the compacted index concurrently.
Incoming Vectors
↓
[Write-Ahead Log]
↓
[Memory Buffer]
↓
[Immutable Segments] → [Background Compactor]
↓ ↓
[Search Coordinator] ← [Compacted Index]
↓
Query ResultTreating vector indices as LSM trees where writes are buffered in memory and flushed to immutable files, which are then merged in the background.
Trade-offs: High write throughput, but read performance requires merging multiple segments.
Updating the reference to the active index structure atomically after a rebuild, ensuring zero-downtime.
Trade-offs: Requires careful memory management to avoid premature deallocation of the old index.
Allowing readers to access the old index version while a background process creates a new version, then switching.
Trade-offs: Increases memory usage during the transition period.
| Reliability | Use write-ahead logs to recover index state after crashes and ensure atomic segment swaps to prevent partial indices. |
| Scalability | Scale by sharding the vector space and performing compaction independently on each shard. |
| Performance | Limit the number of segments queried in parallel; optimize HNSW graph connectivity during compaction. |
| Cost | Reduce storage costs by aggressive compaction of deleted vectors and using compressed vector formats. |
| Security | Ensure index files are encrypted at rest and access to compaction logs is restricted. |
| Monitoring | Track 'segment_count', 'compaction_latency', 'tombstone_ratio', and 'search_p99_latency'. |
Performance degrades due to index staleness, where frequent insertions, updates, and deletions cause the index structure (like an HNSW graph) to become fragmented and suboptimal for search traversal, leading to increased latency and lower recall.
Rebuilding is the process of creating a new index from scratch, often used for major version changes or significant data shifts. Compaction is an incremental, background process that merges segments and cleans up tombstones to optimize an existing index.
Zero-downtime maintenance is typically achieved using MVCC or RCU patterns, where queries continue to serve from the old index while a background process builds the new one, followed by an atomic pointer swap to make the new index active.
Tombstones are markers used to indicate that a vector has been deleted. They allow the database to handle deletions without immediately rebuilding the entire index, which is computationally expensive.
Compaction is necessary if your workload involves frequent updates or deletions. For static datasets that are rarely modified, the initial index build may be sufficient, though compaction can still be used to optimize for specific query patterns.
Segment merging reduces the number of individual index structures the search coordinator must query. By combining smaller segments into one, the engine performs fewer scatter-gather operations, resulting in lower overall latency.
The WAL ensures that if the system crashes during a compaction task, the database can recover the state of the index and the data, preventing data loss and ensuring consistency.
Yes, modern vector databases perform compaction in the background. By using techniques like segment-based storage and versioning, they allow concurrent read and write operations while the background task optimizes the index.
Index fragmentation occurs when the physical layout of the index, such as the graph structure in HNSW, no longer aligns with the logical data, causing inefficient traversal paths and increased cache misses during search.
Monitor metrics such as the ratio of deleted vectors (tombstones), segment count, compaction latency, and P99 search latency. An increase in these metrics often signals that compaction is required.
HNSW indices are graphs that require complex node re-linking during maintenance, whereas IVF indices are partition-based and typically require re-clustering or re-training of centroids, which are different maintenance challenges.
Yes, compaction generally improves search recall. By removing stale nodes and optimizing the index structure, the search algorithm can find more accurate neighbors, reversing the recall degradation caused by index staleness.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.