How does Cassandra store data? – A simple explanation

dml_write-process_12

Every time a record is inserted into Cassandra – it follows the write-path as per the diagram above. First, the record is written to a commit log (on disk). All records irrespective of schema tables  are written to the commit log. The commit log enables recovery of memtable in case of hardware failure. Second,  the record is written to a memtable (in memory, specific to schema table). Third, when memtable reaches a particular size or meets flush requirement, it is flushed to SSTable (on disk, specific to schema table). In addition to SSTable data a number of other SSTable structures such as, primary/secondary index files, compression info, checksum data, etc. are also written to assist read operations.

Records in the commit log are purged after its corresponding data in the memtable is flushed to the SSTable.

As mentioned above, memtables and SSTables are maintained per table and the commit log is shared among tables.

SSTables are immutable, so once memtable is flushed to an SSTable file nothing is written to it again. Thus a schema table is typically stored across multiple SSTable files.

To improve read performance as well as to utilize disk space, Cassandra periodically (per compaction strategy) compacts multiple old SSTables files and creates a new consolidated  SSTable file.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s