Mission Critical Processes

Published on 20 MAY 2019 by Joe

I’ve had some conversations lately w/r/t the level of scrutiny that gets applied to every suggestion at Parity. This is apparently not normal in most startups, where ideas are met with enthusiasm, but this was always SOP for me. I started thinking about what “mission critical” really means.

Back in my aerospace days, we seemed to have regular Roger Boisjoly[1] reminders about what it really means to critique a decision.

What is Mission Critical?

Although the term mission critical is usually associated with engineering, it is not related to anything technical. It’s a decision making process. The journalist Walter Lippman wrote,

“It does not matter that the news is not susceptible of mathematical statement. In fact, just because news is complex and slippery, good reporting requires the exercise of the highest scientific virtues.”

If journalism is mission-critical to democracy,[2] then mission-critical processes apply.

While most discussions around mission-criticality tend to focus on “catastrophic consequences” (e.g. X people dying, more often X dollars lost), I focus on the process and not the failure consequences. Consequences are subjective, and if people decide that they are severe enough, they tend to adopt MC processes around them.

A mission-critical process is a process of evaluating proposed changes, where each change leads to a marginally better system if it works, but system failure if it does not.[3]

As an example, weight is usually a design driver for anything that flies (for reasons that should be obvious). Installing lighter valves or avionics units can improve performance,[4] which means that the {space,air}craft can be marginally more useful.[5] But if a valve leaks fuel or the avionics shut down in the radiation of space, you get total mission failure.

So why not just take the conservative choice every time? If you’re designing an aircraft and always choose the stronger bracket, it will be so heavy that it could never get off the ground. If it can, then it will be expensive and inefficient. Making air travel “green” won’t come from making planes heavier. Physics or economics will always force you to make optimizations. Mission-critical practices must detect when to forego an optimization because the risk is too large.

Mission Critical for Whom?

A common root cause for catastrophic failures is that the people making decisions are not the same people for whom the failed system is mission-critical. Take the 2008 housing crisis. People (unsurprisingly) consider their homes as mission-critical to their way of life. Most of the resentment revolved around the audacity of the decision makers who did not employ mission-critical processes in the treatment of their mortgages.

Many people rely on mission critical systems without understanding them or realizing how dependent their lives are on those system. Which is to say, we put a lot of trust in people and organizations who have opaque decision-making processes (or worse, no processes at all).

Many of the web’s underlying protocols have gone unmaintained for decades. When profit motives didn’t incent the development of privacy-focused protocols, the web’s protocols were captured by corporations whose revenue streams were orthogonal to their product streams. One of the oft-repeated goals of the Web 3 movement is to add financial incentives and transparency[6] to these layers.

Designing Mission Critical Blockchain Systems

As tech giants weave themselves further into the fabric of society, Web 3 companies can feel like they are in a race to save the world from tech dystopia. So how do you level mission-critical methodology with the capital-S Startup’s Weltanschauung of “go fast and break things”?

For me, this is part of the excitement of working in this domain. It’s a constant yet exhilarating challenge.

Although many of the same mission-critical processes from satellite launch and cycling[7] apply, working on software systems is a departure for me.

For one, software is always on. A satellite launch was stressful, but once it was done you could sit back for a moment. But a blockchain is always online. There aren’t discreet events. In a way, you’re just always waiting for that zero day.

Second, the “for whom” question is absolutely enormous. When we ask, “for whom is a blockchain mission criticial,” it could be “anyone who uses the internet, money, or a government.” So, everyone. I think that this is an area we can do much better than traditional jurisdictions and technologies by allowing anyone the opportunity to have a stake and voice in the system, but more on that another time.

Because of these two factors, the decision-making process w/r/t these systems is critical. Decentralized protocols offer the choice to participate in the governance of the system. That is, as long as governance protocols are inherent to the messaging protocols.

Except that, right now, those processes are being designed and implemented. We’re using old tools to build new tools, so we must make them generic enough to adapt to the future. We’re still in an alpha-software phase. A lot of rockets exploded before we put humans on them. But they are quite reliable now, and hopefully a decentralized web will be, too.

Notes

  1. Roger Boisjoly tried, but ultimately failed, to prevent the launch of the Challenger space shuttle, because there was no test data to support the performance of the solid rocket booster O-rings at the low temperature of launch day. Management’s decision to launch came down to the fact that Boisjoly didn’t have the data to prove that launching was dangerous. You know the rest of the story: Challenger exploded 73 seconds after liftoff, killing everyone on board.
  2. I would argue that it is, but that’s besides the point.
  3. As a work environment, this tends to manifest as your collegues trying to break your idea. Which is fine, just don’t take it personally.
  4. If you’re interested, “specific impulse” in the case of rockets.
  5. “More useful” in this case generally means a heavier payload. A lighter rocket means more sensors on the satellite. A lighter aircraft means that airlines can cram another row in economy save on fuel costs.
  6. The “and” here is essential. Financial incentives without transparency essentially amount to what we have today: mega-corporations that demand our complete trust. Transparency without financial incentives is charity, and while good, only allows those who can aford to be charitable to participate. Having both empowers individuals to have a voice.
  7. I include cycling, not that anyone else (except my mom) cares if I win a bike race, but a race is essentially a stream of decisions that result in a marginal energy savings or crashing. Decisions like, how much to slow down for this corner?