PREVIOUS   CONTENTS  NEXT

 

Page 73
Chapter 10
How Do I Evaluate Sources?
Nobody outside a baby carriage or a judge's chambers believes in an unprejudiced point of view.
LILLIAN HELLMAN
"All the news that fits we print" might be the unofficial motto of a free press. One of the great strengths of a free press is its ability to print anything that does not libel its subject. As far as researchers are concerned, however, that very freedom presents its own problems. A source that appears in print, in the media, or on-line is not necessarily valid. As a result, you must carefully evaluate every source you find before you use it. This means that you must read critically and carefully.
As you gather your sources, evaluate them carefully. Here are the three main criteria to use as you determine whether a source is valid for inclusion in your research paper:
1. Quality
2. Bias
3. Appropriateness
Let's look at each criterion in detail.

 

Page 74
Quality
As Spencer Tracey said about Katharine Hepburn in the movie Adam's Rib, "There's not much meat on her, but what there is is choice."
The same is true for movie stars as it is for research source materials: Quality counts. You want only the choice cuts for your research paper. If the material isn't of the highest quality, it won't support your thesis, convince your readers of your point, or stand up under your reader's scrutiny. In fact, it will have just the opposite effect. That's why it's important to evaluate the quality of every source before you decide to include it in your research paper.
The old maxim is true: You can't judge a book by its cover. You have to go deeper. Here's how to do it:
• Check the writer's qualifications. Is the writer or speaker really qualified to write on the subject? Is this someone you trust for a valid opinion? You can use the following simple checklist to evaluate the writer or speaker:
—— Is the person an expert or an eyewitness to the events described in the source?
—— What is the person's reputation?
Example:
You can check in biographical source books such as Contemporary Biography, Who's Who, and Who Was Who to validate a person's reputation. Anthologies often contain biographical information about the various contributors, too.
—— Does the person have the credentials to write on this subject?
Example:
Don't be fooled by degrees.A Ph.D. in chemistry doesn't give a scholar the credentials to write about biology, physics, or any other subject outside his or her field.
—— Is the author well known and respected in the field? How many other books or articles has the author published on the subject?

 

Page 75
—— Does the author have a bias or a personal agenda to advance? Check the author's credentials and reputation to see if you can discern bias; pay attention to tone as well.
• Evaluate the source itself. Here are some guidelines to use:
—— Was the source well reviewed?
Read some critical reviews in quality journals and newspapers to find out how the experts evaluated the book. If the book was not reviewed, it may not be on the front line of scholarship.
—— Who spoke in favor of the book?
Most books have endorsements (called ''blurbs'') penned by well known people in the field. These usually appear on the back cover of the dust jacket. See whether the endorsements were written by respected writers, scholars, and public figures. If not, the book may not be a solid source. A reliable blurb is not the final word, however; so be sure to verify completely, as described here.
—— Is the publisher reputable? Is it known for publishing reliable information?
Reputable sources include scholarly journals, university presses, and major publishers.
—— Is the source up-to-date? What is the publication date?
—— Is the source a first edition, revision, or reprint?
While the information in first editions is usually up-to-date, the book may be so new that it has not yet had time to be authenticated and replicated.
—— Is the source complete? Have certain facts been cut for their controversial nature or for space limitations? To make sure a source is complete, check

 

Page 76
it in more than one version—especially if it's a crucial source.
—— Does the author present sufficient evidence to support the thesis?
—— Does the author document his or her claims with the titles and authors of source materials? Are these sources credible?
—— Can the claims in the source be backed up in other sources?
Be especially suspicious of sources that claim to have the "secret" or "inside track." If you can't find the same information in other sources, the material doesn't hold up to scrutiny.
—— Is the source fair, or does it contain distorted information? The following section shows you how to evaluate sources for bias.
Bias
Every source is biased, because every source has a point of view. Bias is not necessarily bad, as long as you recognize it as such and take it into account as you evaluate and use the source.
Example:
An article on hunting published in Field and Stream is likely to have a very different slant from an article on the same subject published in Vegetarian Times.
Problems arise when the bias isn't recognized or acknowledged. Here are some problem areas to watch:
1. Bogus claims. A claim can be considered bogus, or false, when the speaker promises more than he or she can deliver.
Example:
The speaker may speak vaguely of "many important experiments" or "recent clinical studies" to prove a point. The point may indeed

 

Page 77
have value, but the studies the speaker cites as proof are too fuzzy to have merit.
Well educated people are rightly skeptical about promises from strangers.
Effective research sources use specific support, not just vague references to unidentified studies and sources. You can't evaluate "many important experiments" or "recent clinical studies" unless you know how they were undertaken, by whom, and where the results were published.
Also be on the lookout for sources that refer to "statistics that show... ." Statistics can be very useful in proving a point, but they can also be misleading—especially if you don't have the numbers to evaluate their validity. Ask yourself:
• Do the statistics raise any unanswered questions?
• Has the source of the statistics been revealed?
"Well known" information is another form of bogus claim. Be wary of sources that tell you that "Everybody knows that..." or ''It is a well known fact that ... ." If the fact is so "well known," why is the writer bothering to cite it as support? Very likely, it's the best support the writer can muster—which doesn't speak well for the validity of the source or writer.
2. Loaded terms. Suspect sources may use "loaded terms" to make their point. A term becomes loaded when it is asked to carry more emotional weight than its context can legitimately support. As a result, it becomes slanted or biased. These sources are often not reliable.
Words with strong connotations (emotional responses) often show bias.
Example:
A writer refers to the governor's "regime" rather than "administration." "Regime" is a loaded term because it is used to describe oppressive military dictatorships.
While loaded terms are most often used in political writing and speech, they can appear in any source. That's why it's important to read critically.

 

Page 78
3. Misrepresentation. This type of bias takes many forms. First, a writer or speaker can lie outright. Or a writer may be more subtle, inventing false data or "facts." In addition, dishonest writers often twist what their opponents have said. To misrepresent people this way, they use oversimplification. A complex argument can be reduced to ridicule in a slogan, or an important element of an argument can be skipped over.
How can you protect yourself from being misled by this type of bias? Here are some issues to consider as you evaluate a text for misrepresentation:
• Is someone quoted out of context?
• Are facts or statistics cited in a vacuum?
• Does the quotation reflect the overall content of the source or does it merely reflect a minor detail?
• Has key information been omitted?
To verify this, be sure to check two versions of the source. Remember, if something looks too good to be true, it probably is!
Appropriateness
Even if a source passes the first two tests and proves to be of high quality and free from bias, it may still not belong in your research paper. For a source to make the final cut, it has to fit with your audience, purpose, and tone. It must be appropriate to your paper. How can you decide if a source is suitable for inclusion in your research paper? Try these suggestions:
• Do you understand the material in the source?
• If the source is too technical for you to grasp fully, you might not use it correctly in your paper.
• Is the source written at a level appropriate to your readers?
• Does this source have the information you need?
• Does the source suit your purposes in this research paper?

 

Page 79
A Special Note on Evaluating Electronic Sources
Be especially leery of electronic sources that you'll find on the Internet. They can be difficult to authenticate and validate. Unlike most print resources such as magazines and journals that go through a filtering process (e.g., editing, peer review), information on the Web is mostly unfiltered. What does this mean for you? Using and citing information found over the Web is a little like swimming on a beach without a lifeguard.
Portable Vs. On-Line Sources
As a result, you have to use electronic sources with special care. To evaluate electronic sources, first see if the source is portable or on-line.
• Portable sources, such as CD-ROMs and encyclopedias, are like printed books: They have credited writers and publishers. In addition, they change only when a new version is issued. As a result, you can determine their value as you would a book.
• On-line sources, in contrast, may be published anonymously. This means you can't evaluate the writer or writers. Also, they can be updated and revised without notification. Further, they may vanish without warning. This makes it difficult to evaluate their reliability.
Header, Body, and Footer
Once you've determined that you are dealing with an on-line source, check the Web document for its three main elements: head, body and footer. Within each of these pieces, you should be able to determine the following vital elements for evaluating information:
1. Author or contact person (usually located in the footer)
As you evaluate the selection, ask yourself:
• Who is the author of the piece?

 

Page 80
• Is the author the original creator of the information?
• Does the author list his or her occupation, years of experience, position, or education?
• With this information or lack of it, do you feel this person is qualified to write on the given topic?
• Where does the on-line source come from? Knowing the source of the site can help you evaluate its purpose and potential bias.
Example:
A business has a different slant from a university. It's likely that the business wants to sell you a product or a service, while the university is probably seeking to disseminate knowledge.
You can often find clues to the origin of an on-line source in its address (URL, or Uniform Resource Locator). Look for the suffix to identify the source. Here are the common URL suffixes you'll encounter:
Suffix
Meaning
com
Commercial (business or company)
edu
Education (academic site)
gov
Government
int
International organization
mil
Military organization
net
Internet administration
org
Other organizations, including nonprofit, nonacademic, and nongovernmental groups
sci
Special knowledge news group

2. Link to local home page (usually located either in header or footer) and institution (usually located in either header or footer). As you evaluate the selection, ask yourself:
• What institution (company, government, university, etc.) or Internet provider supports this information?

 

Page 81
• If it is a commercial Internet provider, does the author appear to have any affiliation with a larger institution?
• If it is an institution, is it a national institution?
• Does the institution appear to filter the information appearing under its name?
• Does the author's affiliation with this particular institution appear to bias the information?
3. Date of creation or revision (usually located in footer).
• When was the information created or last updated?
4. Intended audience (determined by examining the body).
5. Purpose of the information, i.e., does it inform, explain, or persuade (determined by examining the body)?
Given all the information you determined from these clues, is this piece of information appropriate for your topic? If yes, explain your decision and express any reservations as you would with any other information.
In summary, all sources are not equally valid. Be sure to carefully and completely evaluate every source you find before you decide whether to use it in your research paper. Weak or inaccurate sources can seriously damage your credibility as a writer and thinker.
Now that you've made sure your sources are solid, it's time to learn how to document them. It's all covered in the next chapter.

PREVIOUS   CONTENTS  NEXT