When creating surveys, questionnaires, or any type of
multi-answer testing mechanism, language use plays a key role in the usefulness
of the results it produces. The biggest flaw in the Writing Option major survey
is the language wasn’t as clear as it could have been. A lot of the questions
were either repetitive or contradictory. Questions like 1, 5, 12, and 14 all
relate to the revision process. Revision is a big factor in the success of a
paper, so documenting the typical steps a student would take when editing a
paper can provide useful and relevant information. However, the wording used in
this survey made the questions seem like they contradicted each other. Key
words like “typically,” and phrases like “spend most of my time” make it seem
like there is only one possible answer. “Typically,” and “most” are words used as
generalizations; “typically” is synonymous with other words like “usually” and “normally,”
which imply that an action is repeated enough that it becomes atypical when it
is not performed. The use of “typically” also implies that if one thing is done
“typically” or “usually,” then another action cannot be performed at the same
time and as often, because then the first action would not be “typical” anymore
and the two would conflict with each other. A person who “typically” studies
while listening to music cannot also “typically” study in silence. Likewise, if
a person is described as spending “most of their time” reading, they cannot
also be described as spending “most of
their time” with friends, because the actions contradict each other—does the
person read more, or go out more? So when this survey states that students typically spend most of their time working
on grammar, organization and arrangement, it feels like we, as students, have
to pick one as to not conflict with the others.
Surveys are useful for collecting general information. They can’t
collect very detailed information; the questions would be intricate for short
responses, and the answers given would require a lot more analysis. More
analysis of answers would take up more time than necessary and would be hard to
average together to get a general consensus. Surveys are best for quickly collecting
data from the group in interest in order to establish a general feel for what the
group thinks or feels.
Challenges include wording, specificity and bias. Wording,
as previously discussed, plays a large role in the interpretability of the
survey. Bad wording can result in skewed data, which is detrimental to the
outcome of the study. It is also difficult to obtain specific data, as the
questions must usually be concise so that more information can be gathered in
less time. Researchers run the risk of asking questions that are too broad.
This is what happened to the questions in the Writing Option survey. Because
the questions were too generalized, the purpose of the survey was lost. Researchers
must establish a focus—what are we trying to learn?—and stick to it. Questions
must always relate back to this so good information can be obtained.
Finally,
researchers need to avoid bias. Cullington’s essay is a good example of
question bias; the way in which a question is asked determines the answer that
is given. If a survey taker feels the researcher wants a specific answer, they
are more likely to answer in accordance to their expectations. This also skews
results and can lead to incorrect results. Researchers must make sure to be as
neutral as possible when asking their questions; they must try to avoid
revealing their stance on a topic and never lead the interviewee/survey taker
to believe they should answer with a specific answer.
