To maintain write throughput in the face of node failures, Cassandra uses "hinted handoffs." A node receiving a write request will attempt to deliver the request to the replica node responsible for the data. If that fails, the recipient node (referred to as the "coordinator node") will save the request as a "hint" — a reminder to replay the write operation when the unreachable replica node becomes available. If the coordinator node knows beforehand that the replica node is unreachable, the hint is saved immediately.
Hinted handoffs are one of Cassandra's consistency repair features. Another, called "read repair," comes into play during read request processing. Depending on the consistency level chosen (explained below), Cassandra may satisfy a read request by reading only one of the replica nodes. Even so, it will issue background reads to all the replica nodes, and verify that all have the latest version of the data. Those that don't are sent write operations to ensure that all copies of the data are up-to-date and consistent.
Consistency or speed
A prominent benefit of an RDBMS is its adherence to ACID — atomicity, consistency, isolation, and durability — principles, which guarantees repeatable, deterministic behavior in a multiclient setting and helps ensure data safety in spite of system failure. Nonrelational databases like Cassandra eschew ACID guarantees on the basis that they become performance-limiting as the database scales in both quantity of data and I/O requests.
Cassandra is described as being "eventually consistent." When data is written to Cassandra, that data is not necessarily written simultaneously on all replica nodes. As described earlier, some cluster members might be temporarily unreachable. However, hinted handoffs ensure all nodes eventually catch up, and the system becomes consistent. Similarly, read repairs catch and correct inconsistencies when the data moves in the other direction, from Cassandra to the outside world.
This notion that different nodes in a cluster might possess inconsistent copies of a given data element might make you uneasy. The good news is that you can tune Cassandra's consistency level. For instance, you can control the level of consistency that a write operation has achieved — how many replica nodes have written the data — before the write is acknowledged as successful to the issuing client application.
Similarly, on read operations, you can control how many replica nodes have responded before the response is returned to the client. This tunable consistency level ranges from Any, which means the request completes if any node responds, to All, which means the request only completes if all replica nodes have responded. Midway between Any and All are consistency levels such as Quorum, which allows requests to complete if a majority of replica nodes have responded. Cassandra's tunable consistency is a powerful feature that lets you balance speed and consistency or trade one for the other. Want speed? Pick Any. Want full consistency? Pick All.
Sign up for CIO Asia eNewsletters.