The DataPart System

DataParts are the units that allows us to be columnar and have version control

Intelligent DataPart Merging

We have developed policies and algorithms to intelligently merge DataParts in the background. The goal is to:

• Automatically compact multiple DataParts into fewer ones
• Detect hot paths and frequently accessed subgraphs
• Optimize for query throughput and storage locality

Commits start as multiple data parts for fast ingestion and converge toward compact forms for analytical speed, combining the best of both worlds.

Summary

Feature	Benefit
Immutable DataParts	Safe versioning and reuse
Parallel write ingestion	High-performance batch processing
Shared storage across commits	Lower memory usage, fast snapshots
Merge roadmap	Compact layout for ultimate read speed

TuringDB uses DataParts to balance high-speed writes, versioned safety, and read-optimized performance, all in a single, cohesive engine.

Related Concepts

ClickHouse: Parts , A similar model used in high-performance columnar stores to enable immutability, versioning, and efficient compaction.