Siodb has a unique way of storing data. This unique storage design is a combination of various modern concepts which together provide high performance. Let’s discover deeper how works this storage architecture.
First of all, Siodb stores its data by column (as opposed to row storage where one storage unit stores multiple columns). Each column from each table has its own data files. Furthermore, Siodb indexes each column by design according to their type.
Also, we designed the automatic data type size to be flexible on data type sizes. Thanks to this feature, Siodb take care of the data size without having to declare column size nor impacting performance. We designed this feature to avoid the waste of space with empty cells or with heterogeneous data size in columns.
Siodb writes data from a column in data files called blocks. all blocks have the same size. Siodb fills the block from the beginning to the end with the data from clients. No matter the type of transaction (insert, update, delete), Siodb generates records and then append those records into the current block.
A storage representation of a table of 4 columns:
This storage architecture enables high throughput. Indeed, for any kinds of operation, multiples threads work in parallel to go through each block of requested columns.
Then, once a block is full, Siodb computes a block key and stores it into the next block. This block to block validation form a chain guaranteeing the authenticity of the data.
This storage architecture answer to both biggest load types in application: OLTP and OLAP.
Indeed, OLAP applications execute large queries across various columns from various big tables. Hence, the column storage, automatically indexed and partitioned by blocks make Siodb very fast on large query processing.
Nevertheless, OLTP applications are not forgotten. Indeed, a master column on each table keeps a virtual view of data by row. Therefore, whenever Siodb accesses a cell from any row in a table, it has direct access on all other cells of that row.
In brief, the internal storage architecture of Siodb reduces I/Os, increases performance and avoid wasting space.
Please comment below should you need more details.