At a time when Apple’s iPhone and Google’s Android handsets are making life particularly difficult for the Blackberry, the last thing manufacturer Research in Motion needed was a service meltdown across Europe, the Middle East, Africa, India and parts of Latin America.
But that’s exactly what’s happened when a core switch fried at a data centre in Slough, west of London and the Blackberry network went … well, black, across large parts of the world.
We can leave to one side the question of how such a single point of failure was ever designed into the network in the first place, but for RIM, Sod Law’s timing was impeccable as always.
Today, the third day of degraded or no service for around 10 million users worldwide, RIM was holding it’s annual customer conference in London. To add insult to injury, pre-orders for the iPhone 4S, which will introduce a new service similar to Blackberry Messenger, are going gangbusters.
The apparent ease of using today’s mobile devices such as the Blackberry belie the underlying complexity needed to make them work. They require complex, clever infrastructure and gremlins can, and will, run amok from time to time. At such times, just like a heroin addict’s mood darkens when they’re unable to get a fix, customers unable to get their email or Facebook updates become similarly gloomy as they experience cold turkey.
It is a scientific fact that when you work with advanced technology, things will go wrong. But that’s not really the point. It’s about how you prepare for and manage major outages and communication with customers through the crisis that really matters.
The eye of any public storm is the time when the boys go one way and the men go another in the communications department. The consensus among Blackberry customers on this particular communications effort: FAIL! The customer response is unsurprising. Managing a very public crisis is never easy – I’ve managed more than my fair share over the years, so this isn’t rear view mirror commentary.
I’d wager that if the senior executives at Vodafone, O2, Orange or T-Mobile, the people that sell the Blackberry happen to be reviewing their Christmas card lists at the moment, RIM’s executives would almost certainly be blacklisted – their customer helpdesks went into meltdown as a direct result.
A detailed poke around the Blackberry homepage by mid morning on day three of the outage found no reference to the issue. There’s no specific statement there, no apology offered and no freefone telephone number provided for customers to call for updates.
Only when I looked at the Twitter feed did I get a sense that anything was wrong. When you face such a public problem, and you’ve got it wrong once already as RIM did, you need to make it easy for disappointed customers.
You need to communicate openly, not close your eyes and hope it goes away. This lack of acknowledgement also means no explanation that RIM’s engineers are working around the clock to restore the service. This might have calmed the customers’ frayed nerves a little.
Maybe RIM’s one step removed position from the end customers is part of the problem. They view the mobile network operators as their customers who in turn are responsible for the end users. After all, RIM, might argue, it’s the MNOs that get paid every month for the service.
But it would be negligent for any large company, let alone one so heavily dependant on technology supporting 70 million customers around the world, 10% of whom are based in the UK, not to have a documented, tested and ready to implement crisis communications plan. If RIM has such a thing in place, it was insufficient on this occasion. Customers were left in a vacuum.
Given the negative publicity the company received during the UK riots only a matter of weeks earlier, which included an appearance in front of the Commons Select Committee to explain it’s position on social responsibility, this should have been water tight. It should also have ben reviewed as a priority and updated if necessary.
This afternoon, Rory O’Neill, the company’s vice president of software and services, explained that they “… thought we had got to the root cause on Monday but we did not,” adding: “We are dealing with over 20PB of data every month so you can imagine the disruption we are trying to resolve.” Interesting and useful information to a telecoms engineer, but not a great message to the vast majority of Blackberry users who never imagine what that means.