Survey Methodology

Constructing an unbiased sample. When reviewing survey results, it's wise to remember one simple rule: A biased sample will produce biased results. It is the goal of every legitimate survey researcher to construct unbiased samples. While completely excluding all bias is almost impossible, several measures can be taken to reduce sampling error.

A sample should accurately reflect the target population. In this case, the target population was all attorneys in Wisconsin who were identified in the State Bar of Wisconsin membership database as being active, private practitioners. Due to time and budget constraints, it was necessary to choose a smaller, representative sample that would reflect the larger population of attorneys. To ensure appropriate geographical representation, the sample was stratified, or subgrouped, into six regions based on county. A small number of attorneys with "unknown" occupation were added to the samples within each region to account for the total number of active, private practitioners in the region, and reduce nonsampling error. The total sample numbered 3,175 attorneys.

Looking at the responses. A total of 955 questionnaires was returned by the June 30 cutoff date for a 30 percent response rate. Of this total, 932 were used for analysis. Those who identified themselves as corporate counsels, government employees, educators, and retired attorneys were excluded, as they did not meet the initial sampling criteria of active, private practitioners. The survey maintains a 95 percent confidence level, with an error of estimation of plus or minus 3.1 percent.

All survey researchers hope for a high response rate, although there is no "standard." A 30 percent response rate, while lower than desired, is typical of what the Bar has been seeing when surveying attorneys who have increasing demands on their time. The somewhat sensitive nature of the survey topic also may have contributed to the response rate. It is important to realize that 30 percent, in this case, reflects the input of nearly 1,000 private practitioners.

When evaluating response rates, the main question is, "How closely does the sample match the rest of the target population?" One way to get a handle on this is to compare the demographic attributes of survey respondents with those of the target population as a whole. Regardless of the sample size, the potential for nonresponse bias is a reality. So what's the best advice? Take survey results for what they are. Pay attention to survey methodology and review results with a critical eye.

Survey Executive Summary available. The 1998 Economics of Practice Survey Executive Summary is available free online. Or, to receive a copy by mail, please call the State Bar at (800) 728-7788 (there is a $3 fee to cover postage and handling).

Wisconsin Lawyer