Topics in Distributed Systems

Note: This list of topics of interest is constantly changing, so check back frequently for updates! Suggested readings for each topic will also be expanded and modified. Please report expired links here.

What is a Distributed System?

  • Consistency vs Availability
  • Weak, Strong, and Eventual Consistency

Suggested Readings

Ousterhout & Rosenblum (1991) The Design and Implementation of a Log-Structured File System

Terry, et al. (1994) Session Guarantees for Weakly Consistent Replicated Data

Terry, et al. (1995) Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System

Petersen, et al. (1997) Flexible Update Propagation for Weakly Consistent Replication

Glendenning, et al. (2011) Scalable Consistency in Scatter

Bailis, et al. (2012) Probabilistically Bounded Staleness for Practical Partial Quorums

Hewitt, et al. (2012) The Actor Model (everything you wanted to know…)

Bernstein, et al. (2014) Orleans: Distributed virtual actors for programmability and scalability

Raju, et al. (2017) PebblesDB: Building Key-Value Stores using Fragmented Log-Structured Merge Trees

Distributed Systems in the Wild

  • Data infrastructures of social media apps
  • Affordances and limitations of commercial cloud offerings (AWS, Google, etc)

Suggested Readings

Ghemawat, et al. (2003) The Google File System

Corbett, et al. (2012) Spanner: Google’s Globally-Distributed Database

Muralidhar, et al. (2014) f4: Facebook’s Warm BLOB Storage System

Consensus Algorithms

  • What is consensus?
  • Key algorithms: Paxos, Raft, and more
  • Implementation challenges

Suggested Readings

Lamport, et al. (1982) The Byzantine Generals Problem

Lamport (2001) Paxos Made Simple

Ongaro & Ousterhout (2013) In Search of an Understandable Consensus Algorithm

Bilbro & Bengfort (2021) Concur: An API for Consensus

Machine Learning

  • Training models in a distributed system
  • Optimizing distributed systems

Suggested Readings

Annamualai, et al. (2018) Sharding the Shards: Managing Datastore Locality at Scale with Akkio

Bender, et al. (2021) On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?

Bengfort, et al. (2019) Anti-Entropy Bandits for Geo-Replicated Consistency

Hilprecht & Binnig (2021). One Model to Rule them All: Towards Zero-Shot Learning for Databases

Schmied et al. (2021). Towards a General Framework for ML-based Self-tuning Databases

Somashekar & Gandhi (2021). Towards Optimal Configuration of Microservices

Zaharia, et al. (2012) Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing

Energy Efficiency

  • How can distributed systems be made more energy efficient?
  • Is there a more efficient way to do consensus (e.g. emergent consensus)?

Suggested Readings

Bender, et al. (2021) On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?

Charapko (2021) Scalable but Wasteful, Or Why Fast Replication Protocols are Actually Slow

Data Security and Privacy

  • Public key cryptography
  • Symmetric and asymmetric encryption
  • Notable breaches and their causes
  • Privacy-first infrastructures

Suggested Readings

Diffie & Hellman (1976) New Directions in Cryptography

Li, et al. (2004) Secure Untrusted Data Repository (SUNDR)

Nakov (2019) Practical Cryptograpy for Developers. Chapter 6 - Encryption: Symmetric and Asymmetric

McCandless & Evans (2021) World’s Biggest Data Breaches & Hacks

  • Understanding GDPR (e.g data processors vs consumers)
  • Cryptocurrency and the Travel Rule

Suggested Readings

Buterin, et al. (2013) The Ethereum Whitepaper

Linden, et al. (2019) The Privacy Policy Landscape After the GDPR

TRISA Working Group (2020) The TRISA Whitepaper

User Experience

  • Balancing UI/UX in distributed systems
  • Localization (l10n) and internationalization (I18n)

Suggested Readings

DePalma, et al. (2006) Can’t Read, Won’t Buy

Scott (2013) Characters, Symbols and the Unicode Miracle

Patch (2014) Unicode Beyond Just Characters: Localization with the CLDR

The GNU gettext authors GNU gettext utilities

CLDR Unicode Common Locale Data Repository

Languages and APIs

  • What languages are best suited to distributed systems?
  • gRPC and Protocol Buffers

Suggested Readings

Bal, et al. (1989) Programming Languages for Distributed Computing Systems

Ferreira, et al. (2020) Programming Languages for Distributed Systems and Distributed Data Management

gRPC Authors (2021) Introduction to gRPC

Nally (2020) gRPC vs REST: Understanding gRPC, OpenAPI and REST and when to use them in API design

Bilbro (2021) What are Protocol Buffers?