I read a post at Royal Pingdom the other day (via RWW) regarding Feedburner’s uptime, and it got me thinking about uptime in general: I feel total percentage of uptime is a misleading metric. For an analogy, if a broadcast meteorologist says there’s a 50% chance of rain in the next 24 hours, it should mean that there’s a 1 in 2 chance that rain will fall on me, which is probably how every listener interprets that information. I doubt very much (though I may be wrong) that they take in to account where in their broadcast area the rain will fall and adjust the probability for how many of their listeners will actually be rained upon. I live not far from the Pacific Ocean, and I’m sure lots of the forecasted rain falls out there on very few people. Although the level of meteorology expertise I expect only exists in Back To The Future Part 2, I think that personalized forecasts are nonetheless a realistic goal.
When it comes to Internet meteorology, which is what Pingdom does, a number like 99.94% uptime is missing critical information. I believe that 0.06% of the time that Pingdom tested Feedburner they got an error, but what does that mean to me? Did everyone who uses Feedburner have similar success, or were there 5,000 people who saw Feedburner down more like 2% of the time? Also, serving up RSS is more like serving up mail than serving up web pages; errors are more hidden and should have graceful recovery. RSS being down is quite different than broken images on an otherwise working site or full outage; it’s not a binary state in my opinion. Many sites have visitors that only stop by for a few minutes each day — downtime is less critical in that case. Furthermore, downtime is not necessarily systemic; two hours of downtime may be catastrophic, but if it only happens once every 5 years, then it’s hardly a trend.
I’m not sure there’s a metric that can easily describe the nuances of uptime better than the percentage, but I hope there is one. Here are some ideas. Bear in mind that these stats should be based on data such as the geographic distribution of the site’s audience and the average session length. In short, I suggest coloring Pingdom’s downtime stats with Compete.com’s traffic data to get a better picture of outages and spot trends.
- Total User Downtime: the number of hours of server downtime in a specific time period times the number of users affected. Short downtime that affects everyone would be balanced with long downtime on a subset of users.
- Daily Audience Percentage Affected: percentage of users affected at the time an incident. If there’s a rolling incident that only affects 5% of all users at a time, but gets everyone at some point in the day (as upgrades sometimes do) then the percentage of affected users would be high even though the incident would appear to be 5% downtime from a single user’s standpoint. On the other hand, 50% outage when 90% of the users are not using the site shows good planning.
- Last X Days Peak [TUD/DAPA]: given one of the above metrics, report the worst (peak) value in the last X number of days. This would mean that it takes a while for incidents to decay, which would be useful for people shopping for reliable hosting.
I am not an expert on these things, just a concerned web software developer who wants better data. If better thinking on uptime metrics has been done elsewhere by smarter people, please let me know with a link in the comments.
