Survey Methodology
Constructing an unbiased sample. When reviewing survey results, it's wise to remember one simple rule:
A biased sample will produce biased results. It is the goal of every
legitimate survey researcher to construct unbiased samples. While
completely excluding all bias is almost impossible, several measures can
be taken to reduce sampling error.
A sample should accurately reflect the target population. In this
case, the target population was all attorneys in Wisconsin who were
identified in the State Bar of Wisconsin membership database as having
active status. Due to time and budget constraints, it was necessary to
choose a smaller, representative sample that would reflect the larger
population of attorneys. To ensure appropriate geographical
representation, the sample was stratified, or subgrouped, into six
regions based on county. A small number of attorneys with "unknown"
occupation were added to the samples within each region to account for
the total number of active attorneys in the region, and reduce
nonsampling error. The total sample numbered 3,741 attorneys.
Looking at the responses. A total of 890 usable
questionnaires were returned for a 24 percent response rate. Respondents
represent a variety of practice types including private practice,
government agencies, the judiciary, in-house counsel, and those working
in legal aid or as law clerks. The survey maintains a 95 percent
confidence level, with an error of estimation of plus or minus 3.2
percent.
All survey researchers hope for a high response rate, although there
is no "standard." A 24 percent response rate, while lower than desired,
is typical of what the Bar has been seeing when surveying attorneys who
have increasing demands on their time. The somewhat sensitive nature of
the survey topic also may have contributed to the response rate. It is
important to realize that 24 percent, in this case, reflects the input
of nearly 900 Wisconsin attorneys.
When evaluating response rates, the main question is, "How closely
does the sample match the rest of the target population?" One way to get
a handle on this is to compare the demographic attributes of survey
respondents with those of the target population as a whole. Regardless
of the sample size, the potential for nonresponse bias is a reality. So
what's the best advice? Take survey results for what they are. Pay
attention to survey methodology and review results with a critical
eye.
2001 Survey Report available. The Economics of Law
Practice in Wisconsin - 2001 Survey Report is available for purchase.
The special member price of $19.95 includes the report and any
additional analysis specific to your individual practice setting or
assistance needed in interpreting the information presented. Nonmembers
may purchase the report for $59.95. To order the report contact the
State Bar at (800) 728-7788.
The
Economics of Practicing Law 
Wisconsin
Lawyer
