Step 3: Selecting Data Collection Methods

In this step, you determine the appropriate methods for conducting the social profile. Currently, social data does not exist at a watershed scale, although we hope to see mapping of social data on geographical information systems in the future. For now, you will need to rely on community data for larger towns in your watershed and on county level data. A county is the smallest unit of government in most U.S. states where all state, federal, and census data are reported. Most census data, including vital statistics, are available at the county and municipal level and sometimes even for villages and townships. In the case of watershed management, the analysis also will include data collected from individuals or households, formal groups, and the community. Any one or all of these units of analysis could be relevant to various aspects of the social profile.

Primary Data

Social profiles usually depend on both primary and secondary data. Primary data are generated and compiled by administering an original study, such as interviews, surveys, or focus groups. These types of data are designed to address a specific issue or information need that is not found in existing sources. Surveys are used to gather primary data about attitudes, beliefs, intentions, and behaviors. This method is unique in that it is the only information-gathering technique, other than talking to every single community member, that has the potential of representing all people in an area. In this respect a survey is a relatively inexpensive way to gather information from a large number of people in a short period of time.

A focus group consists of an interview with about a dozen people about a single topic. A moderator facilitates the focus group meeting and leads the discussion without influencing the responses. To be effective, the facilitator must be unbiased and trained in focus group techniques. Sometimes the discussion may be difficult to control and analyze, and the results cannot be extrapolated to the entire population. However, focus groups can be used successfully during the initial scoping phase to define issues of concern. They also are an effective method to substantiate and clarify results from mail surveys or telephone interviews. [Recommended sources for additional information]

Secondary Data

Secondary data come from information sources that already exist, such as statistical abstracts, state reports, historical studies, and other published literature. Secondary data are usually available at minimal cost and effort, and information covers a broad spectrum of subjects that might be difficult to collect directly. However, the data found on a topic may be overwhelming, not restrictive enough to apply to local communities, or the documents or studies may not be current. Secondary sources should be evaluated just as primary data are examined, and the information should be corroborated by using as many sources as feasible, given time and resources.

Triangulation Method for Cross-Checking Data

Recognizing the imperfections in each data collection method, social sciences research methodology recommends using a triangulation approach to cross-check gathered data (see accompanying diagram). Data validity is increased when you verify one set of data against data from another collection method. For example, the triangulation approach should be applied when conducting telephone or mail surveys. Because survey results usually are based on a sample of the population and responses sometimes can be skewed toward certain types of individuals, it is recommended that focus groups or interviews with key informants be conducted to corroborate and complement the survey findings. However, some data will be available only through one collection method. As long as one data source is not heavily relied upon, gathering from a mixed approach should ensure balanced results.

The data you collect can be recorded either quantitatively or qualitatively. Quantitative data consist of numerical scales that may be analyzed through the use of statistical techniques. Qualitative data are typically verbal or written descriptive accounts of an issue. Qualitative data are analyzed by looking for themes or reoccurring issues in the data. A researcher summarizes these themes and then may collaborate with the watershed planning committee to interpret the meaning of the themes or data.

Survey Methodology

The following information will help you conduct a mail survey in your watershed, but it is not intended to be a complete guide to survey methodology. It is essential that your group consult with your USDA NRCS state office, University Extension offices, local community college or university staff who are familiar with survey techniques.

While preparing the social profile for your watershed, you'll find that some data, especially information about citizen attitudes, does not exist in available sources. For this reason, your planning committee may select to use survey methods to gather additional information specific to your watershed. The most commonly used survey methods are person-to-person interviews, mail questionnaires, and telephone surveys. Most surveys are conducted on small groups of people, which can act as a sample of the total population. However, a survey can also be administered to everyone in a community, thus providing each person with an opportunity to express themselves. For watershed groups dealing with low participation rates at public meetings, administering a survey could serve as a means to define watershed problems and goals and to educate citizens about the watershed.

Although surveys are commonly used tools, they are difficult to develop and implement successfully. Poorly designed surveys may not identify underlying attitudes, may yield inaccurate results, and may also antagonize survey recipients. Therefore, it is essential that your group consult with agency, county extension, or university staff who can assist you with determining your sample size, compiling a mailing list, designing the questionnaire, and using techniques that promote a higher response rate. Surveys are often costly and time consuming, but if done well, they can be an effective method of collecting information. You also may wish to consult books on the topic. An excellent resource is How to Conduct Your Own Survey, by Priscilla A. Salant and Don A. Dillman. This book is written for people with no formal survey training and covers topics such as choosing a survey method, selecting a sample, writing good questions, questionnaire design, and analyzing and reporting results.

We provide sample cover letters and survey questions that have been designed for use by watershed groups. These questions have been used and tested in prior survey research; thus it is best not to significantly alter the individual questions. Following each question is an explanation of how your watershed committee can use the results. In most cases, it will not be necessary to use all of the provided questions in your survey. Depending on the circumstances in your watershed, your committee may select questions of interest and assemble a unique questionnaire for your watershed. However, an expert should be consulted to assist with the overall questionnaire design and the ordering and arrangement of questions. Although we are providing you with questions to use in your watershed survey, we wish to underscore the importance of familiarizing yourself with survey methodology before you begin. The questions alone are not sufficient for an accurate accounting of opinions. The proper methods must be employed to yield accurate responses to your survey.

The results of your mail survey will be most reliable when combined with another method (such as focus groups or personal or telephone interviews) that provides an interactive response and information from residents who did not respond to the mail survey. For a general landowner survey, a response rate between 40 and 80 percent is normal. However, for response rates under 70 percent, you will want to randomly contact a sampling of non-respondents from the mail survey to determine if they hold different opinions from those who initially responded to the survey. The non-respondents may hold different views that, if not accounted for, could lead to a bias in your survey results. Surveying of non-respondents could be done through telephone interviews that ask most of the same questions as the mail survey.

Along with your data collected from secondary sources, your survey data can serve as a baseline description of your watershed. Surveys can be re-administered later in time to measure changes in social parameters and attitudes. Keep in mind that you don't have to gather all the needed survey data with one survey, particularly if resources won't allow doing so. Multiple succinct surveys may ultimately be more effective than one comprehensive survey.

Create Your own Survey Now

Go to Top of Page

Go to Step 4