Consumer Reports Hammers Microsoft For Surface Reliability


Consumer Reports recommended the Microsoft Surface to laptop and tablet buyers, or at least they did until earlier today. That was the point where they dropped the bombshell that in their survey of laptop customers one in four Microsoft Surface customers reported seeing problems with their device in the second year of ownership.

Pretty damning statistics.

Now personally I've owned three Surfaces of different vintage, going back as far as the original Surface Pro, and I've never had a problem with any of them. However its important not to confuse anecdotal and statistical information, especially in a high profile announcement like this.

Microsoft strongly disputed the figures in a diplomatically worded response and suggested its own data shows that the Surface was one of the most reliable devices on the market over the last few years.

Who to believe?

This is a high profile piece of news and if Microsoft has faith in its own statistics it needs to demand Consumer Reports produce evidence of how this data was collected. What level of rigour was undertaken to ensure that reports of problems came from subscribers who actually owned a Surface, what the sample size was and what sort of issues were reported. After all, they very nature of a subscriber to Consumer Reports - like the UK's Which? service - is that they are older and non-technical, therefore more likely to be befuddled by errors which don't relate to hardware at all. At least some of the issues are listed as being 'touch screens weren't responsive enough' which suggests a level of perception being applied in the results. Does that make the whole report suspect? 

Consumer Reports has already demonstrated weak process in its testing and reporting of performance issues with the Touch Bar MacBook last year. Issues which weren't real and a result of user error.

So whilst I wouldn't dismiss this report out of hand, I'd also not accept it without some corroborating evidence either.

0 comments: