Siodb has a unique way of storing data. This unique storage design is a combination of various modern concepts that together enable versioning, high performance, data TTL, and partitioning. Let’s discover deeper how works this storage architecture.
First of all, Siodb stores its data per columns as opposed to row storage (discover the difference here). Then, each column from each table has its own set of data files. Furthermore, Siodb indexes each column by design according to their type.
Siodb has an automatic data type size for text and blob. Thanks to this feature, Siodb takes care of the data size without having to declare column size nor impacting performance. For instance, you just have to declare text and blob column like this:
siocli> create table table_test ( post text, video blob ) ;
Siodb writes data from a column in data files called partitions. All partitions have the same size. Siodb fills the partition from the beginning to the end with the data from clients.
A storage representation for a table of 4 columns:
This storage architecture enables high throughput by leveraging the OS multi-threading capacity. Indeed, for any kind of operation, multiples threads work in parallel to go through each partition of requested columns.
Siodb converts each DML transaction into records. Then, Siodb appends those records in the columns’ data files. Therefore, Siodb has always a consistent view of data at any time natively.
This mechanism eliminates the need for a logging system and for an undo space. Thus, it reduces I/Os considerably at the expense of more disk space consumption. We choose that mechanism for the convenience it provides while nowadays disk space is cheap.
Therefore, Siodb provides natively versioning of both, the data model and the data. That makes possible to create tags, flashback the database at any time, and query the past without any additional configuration nor mechanism.
Application load profile
OLAP applications execute queries across various columns from multiple large tables. Hence, the column storage, automatically indexed and partitioned by blocks makes Siodb very fast on large query processing.
OLTP applications are not forgotten. Siodb maintains a master column on each table that traces a virtual view of data by row. Therefore, whenever Siodb accesses a cell from any row in a table, it has direct access to all other cells of that row.
Your feedback is WANTED. Please give us your input by commenting below.
🧑Design, automate, and scale data technologies in the cloud for my clients.
📱 Text me to get me deploying Siodb for you 👉👉 +41 78 853 85 07
🚀 Contribute to the Siodb project 👉👉 https://github.com/siodb/siodb