Prepare for More Catastrophic Airline Computer Outages Like Delta’s

Photo by Rob Wilson / Shutterstock.com

Catastrophic computer outages that paralyze an entire airline are few and far between. Except this summer.

In July, Southwest Airlines canceled 2,300 flights after a router in one of its data centers failed, delaying hundreds of thousands of passengers. Then, earlier this month, Delta Air Lines suffered a massive computer outage, which triggered the cancellation of 451 flights in a single morning.

A rare look behind the curtain at Southwest’s meltdown offers several important customer-service lessons for passengers who experience similar delays in the future. And in an industry that depends on finicky information systems, these incidents are bound to repeat themselves. They’ve left customers wondering how to avoid getting stuck in another IT collapse and what, if anything, an airline can do to make up for such an event.

Jack Russell, who was scheduled to fly from St. Louis to Las Vegas last month, had a front-row seat for Southwest’s IT issues, which an employee euphemistically blamed on a “software problem.” The airline’s proposed fix: Fly him to Vegas four days later.

As the executive vice president of a software company in St. Louis, Russell knows a thing or two about computers that go on the blink. But he’s less understanding about Southwest’s IT implosion, which he says left him with little choice but to pay an extra $1,800 to reach his destination.

“I spent twice as much money as I thought I would to get to Las Vegas,” Russell says. “If my customers had an outage created by my company and I said, ‘Sorry, it was a freak occurrence,’ they would be waiting at my doorstep with their lawyer.”

The root of the problem

The Southwest systems problem suggests how fragile even the best-run airlines can be. It started in the early afternoon of July 20, when one of its small Cisco routers, out of about 2,000 such pieces of hardware that direct the airline’s network traffic, failed.

This router broke in an unusual way. Instead of registering the error, which would have allowed network administrators in Southwest’s Dallas data center to take it offline immediately and replace it with a working router, it behaved as if it was still operating normally. Only it wasn’t directing any traffic.

Although network administrators spotted the error within half an hour, enough traffic had backed up that critical systems needed to be rebooted — a process that took a full 12 hours and affected critical functions, including the airline’s website, its smartphone app and several internal systems used by Southwest employees to handle reservations. It was as if someone had turned off the lights for half a day.

When the systems flickered back to life, the problems continued. The airline still didn’t have enough information to restart all flights. Because its systems had been down for so long, it couldn’t be sure whether some of its crews had taken enough rest, as required by the Federal Aviation Administration. That forced Southwest to cancel more flights on June 21 and 22.

Brandwatch, a social-tracking service, charted a corresponding tsunami of anger on Southwest’s social media channels. The airline drew 36,905 mentions in a single day on July 21, an almost 20-fold increase from normal levels.

Unprepared for a ‘thousand-year flood’

“The spike in incoming volume that this received was incredible,” says Joshua March, the chief executive of Conversocial, which offers customer-service software to travel companies. “But the really significant piece in this instance was the inability to effectively scale the response.”

Southwest had no script for handling an event of this magnitude.

“It was really rough,” says Robert Jordan, the airline’s executive vice president and chief commercial officer, who describes the IT catastrophe as a “thousand-year flood.” The airline sent 50-percent-off vouchers to passengers affected by the outage, and in some cases paid for them to fly to their destinations on other airlines. All told, he says Southwest spent “tens of millions of dollars” trying to make amends.

“We know we messed up,” he adds. “We know we have to work really hard to regain our passengers’ trust.”

Southwest is still cleaning up. Russell’s delayed flight to Las Vegas is among the thousands of cases still being processed. Under most circumstances, a full refund for a replacement flight would be a tall order, but these are not normal circumstances.

IT disasters of this scale are unusual. Back in 2012, United Airlines experienced several days of delayed flights and sluggish customer service as it struggled to integrate the IT systems of United and Continental Airlines. Last July, United also suffered an outage that made it cancel hundreds of flights after a network router stopped working.

Asked if passengers could have done anything to get to their destinations faster during such a systems collapse, Jordan pauses. So many things went wrong during the event that the normal tricks didn’t work. You couldn’t fall back on calling the airline because even the call-center employees didn’t have access to their IT systems.

“There just isn’t a good answer,” he says.

What customers and customer service reps can do

That’s the consensus of the customer service experts, too.

Elaine Allison, a former flight attendant and on-board service manager who now offers training courses in customer service, says passengers are powerless to negotiate their way around a total systems failure. She happened to be in Las Vegas during the week of Southwest’s outage, but was lucky enough to be flying on another airline.

“Pack at least one day of clothing and small amenities, plus all medications, in a carry-on, in the event luggage is checked and immediately not retrievable,” she says. Russell handled the situation correctly by re-booking his flight on another airline, she says. Southwest must refund a ticket when it cancels a flight.

The trick, says customer service expert Teri Yanovitch, is to look forward and not back. Southwest needs to figure out how to say it’s sorry without losing its shirt, and customers need a game plan should they get caught in a future systems failure.

“Southwest needs to explain the situation and how Southwest will prevent it from happening again,” she says. “As a customer, the best you can do when all critical IT systems are down is to keep calm, don’t take it out on the employee — it is not their fault — and consider your options for alternate transportation based on the situation.”

Research suggests that Southwest can make a full recovery, Yanovitch says. When a recovery is handled correctly, 96 percent of the customers will return. And when it’s not? In 2012, when United Airlines suffered its first meltdown, it was the world’s largest airline. Today, it’s No. 3.

Christopher Elliott’s latest book is “How to Be the World’s Smartest Traveler” (National Geographic). You can get real-time answers to any consumer question on his new forum, elliott.org/forum, or by emailing him at [email protected]

Disclosure: The information you read here is always objective. However, we sometimes receive compensation when you click links within our stories.

Read Next
30 Purchases That Will Start Saving You Money Immediately
30 Purchases That Will Start Saving You Money Immediately

Sometimes, you’ve got to spend to save. These canny deals are a good bet for savings.

11 Expenses That Quietly Drain Your Wallet
11 Expenses That Quietly Drain Your Wallet

It’s scandalously easy to overspend in these areas of your life.

3 Ways to Downsize Your Life to Save Money, Time and Stress
3 Ways to Downsize Your Life to Save Money, Time and Stress

Downsizing can cut the bloat in your finances — and give you more free time and less stress.

Marooned at Home? Earn Some Cash Playing on Your Computer
Marooned at Home? Earn Some Cash Playing on Your Computer

Earn cash by reading emails, taking surveys, playing games, shopping and signing up for offers through this website.

It’s Worth Paying More for These 7 Things
It’s Worth Paying More for These 7 Things

Sometimes, the difference in quality makes it worthwhile to open your wallet a little wider.

View this page without ads

Help us produce more money-saving articles and videos by subscribing to a membership.

Get Started

Most Popular
9 Things You’ll Never See at Costco Again
9 Things You’ll Never See at Costco Again

The warehouse store offers an enormous selection, but these products aren’t coming back.

11 Things Retirees Should Always Buy at Costco
11 Things Retirees Should Always Buy at Costco

This leader in bulk shopping is a great place to find discounts in the fixed-income years.

Over 50? The CDC Says You Need These 4 Vaccines
Over 50? The CDC Says You Need These 4 Vaccines

Fall is the time to schedule vaccines that can keep you healthy — and even save your life.

11 Senior Discounts for Anyone Age 55 or Older
11 Senior Discounts for Anyone Age 55 or Older

There is no need to wait until you’re 65 to take advantage of so-called “senior” discounts.

11 Household Items That Go Bad — or Become Dangerous
11 Household Items That Go Bad — or Become Dangerous

When you get the impulse to stockpile these everyday items, pay close attention to their expiration dates.

8 Things You Can Get for Free at Pharmacies
8 Things You Can Get for Free at Pharmacies

In this age of higher-priced drugs and complex health care systems, a trip to the pharmacy can spark worry. Freebies sure do help.

These Are the 4 Best Medicare Advantage Plans for 2020
These Are the 4 Best Medicare Advantage Plans for 2020

Medicare Advantage customers themselves rate these plans highest.

7 Ways to Boost Your Credit Score Fast
7 Ways to Boost Your Credit Score Fast

Your financial security might soon depend upon the strength of your credit score.

The 10 Most Commonly Stolen Vehicles in America
The 10 Most Commonly Stolen Vehicles in America

A new model parks atop the list of vehicles that thieves love to pilfer.

19 High-Paying Jobs You Can Get With a 2-Year Degree
19 High-Paying Jobs You Can Get With a 2-Year Degree

These jobs pay more than the typical job in the U.S. — and no bachelor’s degree is required.

5 Ways to Get Amazon Prime for Free
5 Ways to Get Amazon Prime for Free

Hesitant to drop $119 a year on an Amazon Prime membership? Here’s how to get it for free.

10 Reasons Why You Should Actually Retire at 62
10 Reasons Why You Should Actually Retire at 62

If you can, here are several good reasons to retire earlier than we’re told to.

3 Ways to Get Microsoft Office for Free
3 Ways to Get Microsoft Office for Free

With a little ingenuity, you can cut Office costs to zero.

26 States That Do Not Tax Social Security Income
26 States That Do Not Tax Social Security Income

These states won’t tax any of your Social Security income — and in some cases, other types of retirement income.

14 Things That Are ‘Free’ With Medicare
14 Things That Are ‘Free’ With Medicare

These services could save you money and help prevent costly health problems.

5 Keys to Making Your Car Last for 200,000 Miles
5 Keys to Making Your Car Last for 200,000 Miles

Pushing your car to 200,000 miles — and beyond — can save you piles of cash. Here’s how to get there.

5 Things That Make Life More Meaningful for Retirees
5 Things That Make Life More Meaningful for Retirees

Retirees agree: These are the things that give them purpose and fulfillment in their golden years.

15 Amazon Purchases That We Are Loving Right Now
15 Amazon Purchases That We Are Loving Right Now

These practical products make everyday life a little easier.

10 Things You Should Never Do With Bleach
10 Things You Should Never Do With Bleach

Does the pandemic have you reaching for bleach more than ever before? Learn the ins and outs of using this powerful disinfectant.

View More Articles

View this page without ads

Help us produce more money-saving articles and videos by subscribing to a membership.

Get Started

Add a Comment

Our Policy: We welcome relevant and respectful comments in order to foster healthy and informative discussions. All other comments may be removed. Comments with links are automatically held for moderation.