FRAMINGHAM, 1 MARCH 2011 - Google (GOOG) says it is "very sorry" for a Gmail software bug that reset some 150,000 accounts and left their owners contemplating the prospect of having lost years worth of data. The outage affected only a fraction of 1% of Gmail users, but its severity was particularly noteworthy.
From a post last night on Google's Gmail blog: "I know what some of you are thinking: how could this happen if we have multiple copies of your data, in multiple data centers? Well, in some rare instances software bugs can affect several copies of the data. That's what happened here. Some copies of mail were deleted, and we've been hard at work over the last 30 hours getting it back for the people affected by this issue."
The issue left Google engineers scrambling to access backup tapes. More from that blog post: "To protect your information from these unusual bugs, we also back it up to tape. Since the tapes are offline, they're protected from such software bugs. But restoring data from them also takes longer than transferring your requests to another data center, which is why it's taken us hours to get the e-mail back instead of milliseconds."
In January, as part of an announcement tightening its service level agreement standards, Google reported that Gmail had a 99.984% uptime rate in 2010 for both business and consumer users.
Sign up for CIO Asia eNewsletters.