Y'know, the basics of evaluation really aren't that hard. Honest. Specifically, when you are comparing groups, you must make sure you're not dealing with different populations. F'instance: you cannot -- although people do -- walk into a college classroom and ask "Who here is either a student or a professor" and extrapolate the resulting 100% to the population of the United States.As the good people at STATS found, that doesn't stop people from doing just that:
Media Miss Abstinence Study FlawsMarch 24 2005
Rebecca Goldin
Study fails to control for important confounding factors; confounds judgement on value of abstinence.
"Confound" is not a good thing.A recent study published in the Journal of Adolescent Health asserted that teens who pledge abstinence have the same risk for sexually transmitted disease (STDs) as their peers.
Most major news organizations reported this finding without taking a critical look at the study itself. USA Today headlined their article “Study: Abstinence pledges may trigger risky sexual behavior,” while the Washington Times led with “Virginity pledgers found no less susceptible to ills.” In contrast, U.S. Newswire claimed the opposite, that “the data clearly shows that virginity pledges are a helpful tool in curbing teen sexual activity and its dangerous consequences.”
So who is right?
The claim that teens who pledge abstinence have a similar rate of STD infection as non-pledging teens suggests that the two groups of people - pledgers and non-pledgers - are similar except for their pledge status. However, in this case, the authors of the study did not control for possibly important demographic differences.
Opening themselves to all kinds of possible problems:If promoters of abstinence pledging are targeting teens at high risk for STDs, then it is possible that the abstinence programs are having tremendous success. Perhaps those who pledge virginity would otherwise have had a higher rate of STDs than the general population. By pledging, they lower their risk to the average risk. While the study finds that the rate of STD transmission is similar in the two groups, those who pledged abstinence lowered their risk, from above average to average.
So abstinence pledgers could have a lower incidence of STDs.On the other hand, if those who pledge abstinence would otherwise be at lower risk than the general population, then it is possible that abstinence pledging has an adverse effect. Without the pledge, these teens would have a lower risk than average, but by pledging their risk actually increases. While counter-intuitive, this result is consistent with the possibility that those who pledge virginity then feel more inclined to experiment with oral and anal sex than they might have had they not pledged to stay away from vaginal sex.
Or pledgers might have a higher risk of STDs. There is another possible difference between pledgers and non-pledgers that could skew the results significantly. The study did not control for possible differences in sex education. Abstinence pledging is promoted largely through church-based programs. Perhaps these programs are reluctant to educate teens about protecting against STDs, should someone decide to break his or her promise (or participate in oral or anal sex). If those who pledge abstinence are generally less knowledgeable about how to use condoms than those who do not pledge, then they will be at an increased risk for STDs independent of their pledge. Perhaps an abstinence-pledging program with an informed population about how to minimize risk from sexual contact would be far more effective than a traditional sex-ed course. Or perhaps not.On the other hand, perhaps those who pledged abstinence are just as well-educated as those who do not pledge. This would suggest that the pledge itself gives a false sense of security that leads to additional risk taking, as much of the media suggests. Perhaps abstinence pledging limits the effectiveness of sex education. Or perhaps abstinence-pledging comes hand-in-hand with reduced education about STDs, as is often argued by those who oppose abstinence-only sex-education. In this case, it is reasonable to compare pledgers with non-pledgers without controlling for education level.
There are other possibly important confounding factors not addressed by the study, including: Which students were polled? The study did not poll home-schoolers or dropouts. Of the approximately 6,500 young teens first surveyed, only about 75 percent completed the second and third stages of the survey (occurring a few years subsequent).
In other words, pledging might do good or it might do bad. The point: you can't tell from this study. And that's because:In order to conclude that pledging has little to no effect on STDs, a study should control for
all (emphasis mine) factors that affect STD infection rates. However, the Journal of Adolescent Health study did not control for either a teen’s risk for sexually transmitted diseases, the type and level of sex education he or she received, or the school environment.
Because the two groups of teens are not necessarily comparable, the study’s results have no real meaning, either supporting the effectiveness of abstinence pledging or disputing its usefulness.
That's worth saying again: "the study's results have no real meaning".
