Siodb Data Storage

Siodb has a unique way of storing data. This unique storage design is a combination of various modern concepts that together enable versioning, high performance, data TTL, and partitioning. Let’s discover deeper how works this storage architecture.

Column storage

First of all, Siodb stores its data per columns as opposed to row storage (discover the difference here). Then, each column from each table has its own set of data files. Furthermore, Siodb indexes each column by design according to their type.

Data auto-sizing

Siodb has an automatic data type size for text and blob. Thanks to this feature, Siodb takes care of the data size without having to declare column size nor impacting performance. For instance, you just have to declare text and blob column like this:

siocli> create table table_test ( 
             post text, 
             video blob 
        ) ;

Block partitioning

Siodb writes data from a column in data files called partitions. All partitions have the same size. Siodb fills the partition from the beginning to the end with the data from clients.

A storage representation for a table of 4 columns:

This storage architecture enables high throughput by leveraging the OS multi-threading capacity. Indeed, for any kind of operation, multiples threads work in parallel to go through each partition of requested columns.

Database versioning

Siodb converts each DML transaction into records. Then, Siodb appends those records in the columns’ data files. Therefore, Siodb has always a consistent view of data at any time natively.

This mechanism eliminates the need for a logging system and for an undo space. Thus, it reduces I/Os considerably at the expense of more disk space consumption. We choose that mechanism for the convenience it provides while nowadays disk space is cheap.

Therefore, Siodb provides natively versioning of both, the data model and the data. That makes possible to create tags, flashback the database at any time, and query the past without any additional configuration nor mechanism.

Application load profile

This storage architecture answers to both load profiles from applications: OLTP and OLAP.

OLAP applications execute queries across various columns from multiple large tables. Hence, the column storage, automatically indexed and partitioned by blocks makes Siodb very fast on large query processing.

OLTP applications are not forgotten. Siodb maintains a master column on each table that traces a virtual view of data by row. Therefore, whenever Siodb accesses a cell from any row in a table, it has direct access to all other cells of that row.

Quick start now
3 minutes – no login

Leave a Reply

Your email address will not be published.