Consumer Survey Limitations
We regularly survey area Consumers' Checkbook and Consumer Reports subscribers for their feedback on services they have used. We sometimes also survey subscribers to various other websites and publications and survey a sampling of other area consumers.
For many service types, we ask whether the consumer would "recommend" or "not recommend" the service provider and we indicate on this website the percent who recommended. Since many companies were rated by rather small numbers of raters, small differences between two firms in the percentage of raters who recommended should not be relied on although the information might be useful to consider along with other information in comments reported here or other information you gather from other sources. The table below gives a rough guide to minimum differences you should look for in deciding whether to give much importance to differences in the "recommend" percentages.
When using these survey data, remember that the survey responses are to some degree subjective and that the differences among companies might be explained by differences in the personalities, backgrounds, critical standards, and other characteristics of the raters or by biases these raters might have.
A rough guide for deciding whether the difference between two percentages is important | If one firm had this number of ratings: | And a second had this number of ratings: | Do not give much importance to the difference between percentages unless the difference is at least this many percentage points: | |
---|---|---|---|---|
Assuming the average of the two firms' percentages is 50 percent | 10 30 60 120 |
10 30 60 120 |
45 26 18 13 |
|
Assuming the average of the two firms' percentages is 80 percent | 10 30 60 120 |
10 30 60 120 |
36 21 15 10 |