| |
 |
Navigation |
Vol. 72, No. 11, November 1999 |
Survey Methodology
Constructing an unbiased sample. When reviewing survey results,
it's wise to remember one simple rule: A biased sample will produce biased
results. It is the goal of every legitimate survey researcher to construct
unbiased samples. While completely excluding all bias is almost impossible,
several measures can be taken to reduce sampling error.
A sample should accurately reflect the target population. In this case,
the target population was all attorneys in Wisconsin who were identified
in the State Bar of Wisconsin membership database as being active, private
practitioners. Due to time and budget constraints, it was necessary to choose
a smaller, representative sample that would reflect the larger population
of attorneys. To ensure appropriate geographical representation, the sample
was stratified, or subgrouped, into six regions based on county. A small
number of attorneys with "unknown" occupation were added to the
samples within each region to account for the total number of active, private
practitioners in the region, and reduce nonsampling error. The total sample
numbered 3,175 attorneys.
Looking at the responses. A total of 955 questionnaires was returned
by the June 30 cutoff date for a 30 percent response rate. Of this total,
932 were used for analysis. Those who identified themselves as corporate
counsels, government employees, educators, and retired attorneys were excluded,
as they did not meet the initial sampling criteria of active, private practitioners.
The survey maintains a 95 percent confidence level, with an error of estimation
of plus or minus 3.1 percent.
All survey researchers hope for a high response rate, although there
is no "standard." A 30 percent response rate, while lower than
desired, is typical of what the Bar has been seeing when surveying attorneys
who have increasing demands on their time. The somewhat sensitive nature
of the survey topic also may have contributed to the response rate. It is
important to realize that 30 percent, in this case, reflects the input of
nearly 1,000 private practitioners.
When evaluating response rates, the main question is, "How closely
does the sample match the rest of the target population?" One way to
get a handle on this is to compare the demographic attributes of survey
respondents with those of the target population as a whole. Regardless of
the sample size, the potential for nonresponse bias is a reality. So what's
the best advice? Take survey results for what they are. Pay attention to
survey methodology and review results with a critical eye.
Survey Executive Summary available. The 1998 Economics of Practice
Survey Executive Summary is available free
online. Or, to receive a copy by mail, please call the State Bar at
(800) 728-7788 (there is a $3 fee to cover postage and handling).
|
