In 1961, Alan Shepard became the first American to reach space aboard NASA’s Freedom 7.

NASA took a large, albeit calculated, risk in putting Alan Shepard aboard that spacecraft. What reassurances did they have regarding his safety?

According to a NASA archive, “the key factor in enabling NASA to take such a risk was the redundancy built into the orbiter.”

The spacecraft had four computers (and an extra backup) loaded with identical software. If one failed, the next computer would take control. System redundancy, in this case, was a matter of life or death.

From Outer Space to the Factory Floor

The operations managers I know aren’t dealing with such high-risk systems. However, despite lower stakes, they rely on redundancies because it’s foolish not to.

• One building manager keeps his commercial tenants warm with a system of failover water boilers and pumps.
• A plant supervisor at a plastics company keeps injection blow molding machines running with a backup air compressor.
• A hospital’s facilities manager guarantees access to life-saving oxygen with multiple redundancies.

In this post, we’ll take a closer look at these three system redundancy examples. We will consider their shortcomings. And we’ll examine opportunities to make sure your operations never miss a beat.

Three Redundancy Examples from Three Industries

1. The Building Manager

From the man himself:

In one of my buildings, there are four water heaters. And two pumps. When one fails, the other one kicks on.

His problem?

I have no way of knowing if one of the heaters fails and I’m running the failover. And then sometimes even the failover fails, and that puts me in a tough spot. I usually don’t know something is wrong until a tenant calls to let me know the building is freezing.

2. The Plant Supervisor

One plastics manufacturer couldn’t keep up with demand. Unfortunately, it wasn’t always because business was booming.

They used injection blow molding machines. These use an air compressor to create the product’s final shape. You can bet they had a backup air compressor since forced air was critical to their manufacturing process.

Unfortunately, their problem was similar to the building manager we discussed above. They had no (good) way of knowing that their main air compressor had failed and that they were relying on the backup.

How did they know the backup compressor was running?

Well, the backup air compressor tended to smoke after a few hours of use. Yikes.

3. The Hospital Facilities Manager

Patients can die when a hospital’s oxygen supply cuts off.

Our facilities manager designed the hospital’s system to reflect the high stakes.

The hospital had a primary oxygen supply tank, a reserve oxygen supply tank, and a third supply system. Each supply tank had its own internal set of redundancies.

The third supply system was in a separate location from the primary and reserve tanks, and had an independent feed line.

But you can never be too sure with lives on the line.

The facilities team made regular physical inspections. They did this even though it took up a considerable part of their day.

Of course we’d love to have that time back. But this could be the difference between life and death. Just because we have multiple redundancies doesn’t mean we can get lazy and rely on them.

Why Redundancy Isn’t Enough

System redundancies put Alan Shepard in space and safely brought him home. They’re what enable the three people profiled above to rest easier.

But as we saw with each of them, there is still room for improvement in ensuring a system is reliable.

Waiting until an angry tenant tells you it’s too cold is unacceptable. Allowing clouds of black smoke from the backup compressor to fill your factory is out of the question.

We could discuss machine learning, predictive maintenance techniques, and a slew of other measures one could take to keep a system operational.

But what if you don’t need to, metaphorically, go to the moon?

What can you do when you’re short on time and a full-service automation or facility management system is too expensive?

Know What’s Up When Systems Go Down

Our building manager and the plant supervisor could solve their problems with a simple hardware setup:

• A current monitor could detect when the power supply for a failover pump turns on.
• A similar vibration sensor could detect when a backup air compressor roars to life.

In the spirit of redundancy, you could outfit both the primary and backup equipment with these to guarantee you don’t miss a change in behavior.

Is this something you would be interested in?

If so, here is specific hardware we’ve had success with at Temboo:

What I like most about each of these sensors from National Control Devices is that they come ready to go off the shelf. They’re also straightforward to set up. That’s more than I could say if I tried building them myself with individual hardware components.

What About Our Hospital Facilities Manager?

I would still recommend that our hospital facilities manager add something like the vibration sensor to their oxygen supply tanks. That would let them know when one tank had failed and the backup had turned on.

This addition doesn’t mean that their staff is off the hook for doing manual inspections. What it does is provide yet another layer of redundancy to a process responsible for keeping people alive.

Which Projects are Worth the Effort?

An angry tenant telling you the building is too cold and a factory filled with smoke are clear consequences of unplanned downtime.

Tackling low-hanging fruit, like the examples outlined in this post, is a great way to secure some quick wins.

If you’re not sure that fixing the problem is worth the hassle, or if your situation demands a more complex solution, check out our 8-Step Formula to Calculate the Costs of Unplanned Downtime.