Reading Summary 2019.05.15

· by Aldrin Montana

Sinfonia: a new paradigm for building scalable distributed systems

Overall Evaluation

I really like this paper for the idea of minitransactions and the approach to distinguishing between application nodes and memory nodes. By using semantics of a transaction to be able to execute final operations concurrently the authors show that it is possible to reduce the number of network round-trips. And the shift from bringing data to application nodes to bringing logic to memory (or storage) nodes. The idea of having memory nodes vs application nodes seems similar in spirit to mapreduce, though I think that Sinfonia offers more flexibility.

I also thought it was funny that “possible applications for MapReduce include distributed grep, sort and computing inverted indexes.”

Strong Points

1. The discussion of the redo log managed by participants and how recovery is handled for coordinators is quite interesting and seemed fairly novel to me.

2. I think it is fairly interesting how minitransactions grab locks in the first phase if all is well, and how optimistic this seems to allow the minitransaction protocol to be. Locks only need to be held for 1 full round trip, and because they are at a high resolution, the likelihood of lock conflicts seems low.

3. The group communication service seems to highlight an elegant use case for Sinfonia and the way in which the authors were able to de-couple dependencies as much as possible.

Weak Points

1. I think the use of a recovery coordinator seems brittle but also does not seem to adequately handle coordination failure on it’s own. I feel like there are a lot of details here that are relevant but were left out.

Questions Raised

1. I wonder if minitransactions provide a good abstraction for managing NVRAM, specifically, by viewing NVRAM devices as memory nodes themselves. My initial thought is that in Sinfonia, reliability of memory nodes are important because that is where most state is maintained. And, since NVRAM maintains state even in crashes, perhaps it can be physically co-located with an application node and essentially serve as transactional memory (or at least, my understanding of it).

2. It is not clear to me that Sinfonia provides distributed transactions, and so I wonder if more complex transactions must be expressed as a chain of minitransactions and how that affects the type of consistency and atomicity guarantees Sinfonia is able to provide.

Research Connections

I am interested in trying to figure out ways of getting a storage system to help an application with incremental computation over data. I wonder if the use of minitransactions can be an efficient way to propagate computations without holding locks for too long and it seems to be an approach that could be extended to Ceph or a similar programmable storage system fairly easily.