Speaking to the Associated Press, Delta CEO Ed Bastian said, “We did not believe, by any means, that we had this type of vulnerability.”
And the world agreed.
How does a company the size of DELTA Airlines get caught with its pants down to such an extent that from August 8th – 10th of this year, over 2300 flights had to be cancelled and thousand others delayed? How could someone this large NOT have tested the backup systems and rechecked all the inter dependencies to make sure in case of a disaster or in this case a simple electrical piece shutting down a transformer bringing electricity to their datacenter not all be working?
How?
Because in the case of DELTA (like the rest of us), they thought they had figured it all out.
Surely this scenario couldn’t happen?
Oh, Really?
When the backup power came on, not all the servers including the reservation and some operation servers were connected to it. The failures started a “snowball effect,” and it would take them almost a week to get everything back to normal. How could someone not check to make sure all servers had access to the backup power?
Because they didn’t. They assumed, and their assumptions cost them millions.
But as a business owner and one whose company works and deals with helping SMBs plan for disasters, is there anything we can all learn from this?
I believe 4 VERY IMPORTANT lessons.
Strive for 100% redundancy
According to Delta’s chief information officer, a power failure caused the company’s data center to crash, grounding thousands of would-be passengers. Although power was restored six hours after the incident, critical systems and network equipment failed to switch to a secondary site, corrupting valuable data in the process. And while some systems failed over, other vital applications didn’t; this created bottlenecks, decreased revenue, and diminished customers’ confidence.
Delta’s case is a massive wakeup call not just for the airline industry but for every business — large and small. Companies must implement disaster recovery plans for their data centers, on-site technology, and Cloud applications to continue servicing customers while fixing the main issue with their primary systems. Companies also need to get rid of the false notion that redundancy plans to assure service continuity is restricted to larger corporations. DR and business continuity solutions are extremely affordable today, and a partnership with a provider can help you in more ways than one (more on this later).
Always test your backups
So although Delta had a plan to bring its business back to normalcy, the DR plan left a lot to be desired in practice. This begs the question as to whether the airline company is actually testing, reviewing, and reinforcing its vulnerabilities to different disasters.
The point is that even though your company may have a failover protocol in place, that protocol adds no value to your business unless it has been rigorously tried and tested. In order to avoid the same fate as Delta, make sure to find out whether your disaster recovery plan is capable of running mission-critical applications like email and customer service applications before — not after — downtime occurs.
Account for different types of vulnerability
In an interview with the Associated Press, Delta CEO Ed Bastian basically told the reporters they never saw it coming. Indeed, it’s often hard to foresee what threats and vulnerabilities a natural disaster, power outage, or hacker can produce. But it’s not impossible.
By conducting a comprehensive audit of your data center security and disaster protocols, your business will be more aware and adept at minimizing the risk of potential disasters. This also means evaluating and preparing for disasters that are likely to happen to your business depending on its geographic location. Southern US, for instance, is prone to hurricanes and flooding.
Call for help
These lessons and strategies are all crucially important, but pulling off a DR and business continuity solution on your own may be difficult. For this reason, it’s critical to have a planned partnership with a managed services provider that can assess, plan, test and install the continuity solutions your business needs in order to minimize the impact and avoid encountering a Delta IT outage of your own.
To find out more about business continuity and guaranteeing complete IT redundancy, contact us today, 336-303-1730 or email [email protected].