Distributed Systems

A distributed system is a set of independent computers that coordinate over a network to appear as a single coherent system. Most software running at meaningful scale today is distributed, including web applications, databases, message queues, content delivery, and machine learning training.

How it works

Distributed systems are built around two recurring patterns: replication (multiple copies of the same data, for availability and read scaling) and partitioning (splitting data across machines, for write scaling and capacity). The CAP theorem describes their fundamental tradeoff: under a network partition, a system must choose between consistency and availability.

Coordination across nodes uses a consensus protocol such as Raft or Paxos for strongly consistent decisions, or eventual consistency mechanisms (gossip, CRDTs) where some divergence is acceptable. Failure detection, retries, idempotency, and back-pressure are the load-bearing primitives that hold the system together.