Interested in the future, that’s where we are going to spend the rest of our lives.
From Exploding Engines to the Industrial Engineering
The two key enabling technologies of steam power were the governor and the safety valve. These took steam engines from being a dangerous toy to the key enabling technology for the industrial revolution. But they also led to thinking about systems, and how to make them robust, something that has not yet taken off in distributed systems design. The safety valve was invented by Denis Papin, a frenchman who invented the pressure cooker on a visit to London in the late seventeenth century. It is based in the simple idea that an explosion is not the most helpful way to respond to overload, and it is better to lose a bit of potential work. We don’t usually look at the overload behaviour of our code. Let’s take a look at some failure modes, test our software until it breaks, something the Victorians also invented after the Tay Bridge disaster. We can see that software does not fail nicely. Crash only software is fine for random failures with a load balancer, but failure under load will be made worse. The governor introduces feedback, probably the most important thing when thinking about systems. Indeed the governor was the ultimate inspiration for Charles Darwin’s theory of evolution. Feedback is what takes a system from being a collection of parts to a whole that works together and is controllable. The governor is even older than the safety valve, having first been invented to build water clocks, and the form used for the steam engine was adapted by James Watt from the control system for windmills. While we rely on some key feedback systems, such as TCP, in the foundations of the internet, they are not yet tools for every distributed system. Resilience, self-organization and hierarchy are the key principles of systems that survive. Every microservice should be stand alone, able to survive catastrophic failure, and able to give feedback to other components. We can look at what this communication model looks like in an application.