How We Gather (and How to Interpret) Our Data
Our tables rating individual companies will be more valuable to you if you know how the data were gathered and how they should be interpreted.
Ratings from customers
We regularly survey area Checkbook and Consumer Reports subscribers for their feedback on services they have used. We sometimes also survey subscribers to various other websites and publications and survey a sampling of other area consumers.
On our survey on auto insurers, we asked consumers who had had a claim or claims against their auto insurance companies to rate their experiences “inferior,” “adequate,” or “superior” on several aspects of service. Our ratings tables show the percent of each company’s surveyed customers who provided a rating of “superior” (as opposed to “inferior” or “adequate”) for questions on our survey.
We have included in our ratings tables all of the insurance groups or companies for which we received at least 10 ratings on our customer surveys. If a company is not listed in our ratings tables, it simply means we did not receive at least 10 ratings for it; that fact has no negative or positive implications.
The insurer names we use are short-hand and may relate to a group of companies; the customer survey ratings are all that we received from customers who identified what we judged to be the company or group associated with this name.
Since some companies were rated by rather small numbers of raters, small differences between two firms in the percentage of raters who gave a particular rating (say, “superior”) should be ignored. The table below gives a rough guide to minimum differences you should look for in deciding on one firm over another.
When using these survey data, remember that the questions are to some degree subjective and that the differences among firms might be explained by differences in the personalities, backgrounds, critical standards, and other characteristics of the raters or by biases these raters might have.
Survey of Auto Body Shops
We surveyed area auto body shops and asked shop owners to rate the insurers “poor,” “fair,” “good,” “very good,” or “excellent” on “treating their customers (car owners) fairly.” Our ratings tables show shows the percent of surveyed shops that rated each company “good,” “very good,” or “excellent,” and the number of ratings each company received.
Our ratings tables report counts of private passenger auto insurance complaints filed with the Massachusetts Office of Consumer Affairs & Business Regulation during 2011, 2012, and 2013, the most recent years for which data were available when we checked. Our ratings tables also report a “complaint rate,” which takes into account the fact that companies that do much more business than others are likely to incur more complaints. It is calculated as the number of complaints per $10 million in private passenger auto insurance premiums written.
We collected these rates from the Massachusetts Office of Consumer Affairs & Business Regulation’s excellent auto insurance premium comparisons website. You can access the state’s comparisons, which include sample premiums for several more coverage amounts and locations than we report on, by visiting www.mass.gov/ocabr/insurance. Rates used in our comparisons were those reported on the state’s website in March 2016.
Timeliness of the Data
All of the data must be interpreted in view of timeliness. Our customer survey data are from surveys conducted from January 2009 to December 2017. Survey respondents were asked to report on experiences in the preceding year. Sample premiums were those available on the state’s website in March 2016.
We give checkmarks to companies that score highest on a scoring system that we devise for each service field. Our scoring systems weight the various data in our tables and text based on our subjective judgment of their importance. Since the scores are based entirely on information presented, you can apply your own subjective judgments, and decide whether you prefer companies we have not given checkmarks. Where we do not have important data on a company, we cannot give our checkmark.