Posted on 7 Comments

What to do When Research Indicates Your Approach is Unlikely to Succeed: Part I of a Case Study on the Community-Based Abstinence Education Program RFP

The Community Based Abstinence Education Program (CBAE) from the Administration on Children, Youth and Families is a complicated, confusing, and poorly designed RFP based on suspect premises. That makes it an excellent case study in how to deal with a variety of grant writing problems that relate to research, RFP construction, and your responses.

CBAE is simple: you’re supposed to provide abstinence and only abstinence education to teenagers. That means no talk about condoms and birth control being options. In some ways, CBAE is a counterpoint to the Title X Family Planning funding, which chiefly goes to safe-sex education and materials rather than abstinence education. Its premise is equally simple: if you’re going to have sex, use condoms and birth control. Congress chooses to fund both.

Were I more audacious regarding CBAE proposals, I’d have used George Orwell’s 1984 as a template for the programs, since almost everyone in the novel conforms to the numbing will of an all-powerful state and many belong to the “Junior Anti-Sex League,” complete with scarlet sashes. I hope someone turned in a CBAE application proposing scarlet sashes for all participants.

More on point, however, page two of the RFP says:

Pursuant to Section 510(b)(2) of Title V of the Social Security Act, the term “abstinence education,” for purposes of this program means an educational or motivational program that: […]

(B) Teaches abstinence from sexual activity outside marriage as the expected standard for all school age children

Who is enforcing this “expected standard?” Society in general? A particular person in society? But it gets better:

(D) Teaches that a mutually faithful monogamous relationship in the context of marriage is the expected standard of human sexual activity;

This requirement ignores decades of anthropological research into indigenous societies as well as plenty of research into our own society, which Mary Roach described in Bonk, Alfred Kinsey described using imperfect methods in his famous but flawed research in the 50’s, and that Foucault described in his History of Sexuality. It also ignores the sexuality of other cultures and even our own, as discussed in books like Conceiving Sexuality: Approaches to Sex Research in a Postmodern World, or, better yet, Culture, Society and Sexuality: A Reader, which describes the way societies and others build a social model of sex. Through the CBAE program, Congress is building one such model by asserting it is true and using “expected standard” language, without saying who is the “expecting” person or what is the “expecting” body. It’s an example of what Roger Shuy calls in Bureaucratic Language in Government and Business a term that “seems to be evasive,” as when insiders “use language to camouflage their message deliberately, particularly when trying to avoid saying something unpleasant or uncomfortable.” In this case, the evasion is the person upholding the supposed standard.

Furthermore, the abstinence conclusion isn’t well supported by the research that does exist, including research from previous years of the program, which is at best inconclusive. A Government Accountability Office report (warning: .pdf file) says things like, “While the extent to which federally funded abstinence-until-marriage education materials are inaccurate is not known, in the course of their reviews OPA [Office of Population Affairs] and some states reported that they have found inaccuracies in abstinence-until-marriage education materials. For example, one state official described an instance in which abstinence-until-marriage materials incorrectly suggested that HIV can pass through condoms because the latex used in condoms is porous.”

The one comprehensive study that has been conducted by a nonpartisan firm is called “Impacts of Four Title V, Section 510 Abstinence Education Programs” by Mathematica Public Research, which was spun off from the guys who brought us the Mathematica software. The study was prepared for DHHS itself, and it says such encouraging things as, “Findings indicate that youth in the program group were no more likely than control group youth to have abstained from sex and, among those who reported having had sex, they had similar numbers of sexual partners and had initiated sex at the same mean age.” The programs it studied are based around the same methods that the CBAE demands organizations use, all of which boil down to inculcating a culture of fear of sex outside of marriage. The social stigma the program recommends is based around STDs and whether you’ll get into college (although an editorial in the L.A. Times argues otherwise), and, to a lesser extent, altering peer norms. Still, even in Puritan times this was not entirely effective, as Bundling by Henry Stiles explains. The practice meant sleeping in the same bed with one’s clothes on, as a solution to the problems of inadequate heat and space. But, as Jacques Barzun says in From Dawn To Decadence: 500 Years of Western Cultural Life, “Experience showed the difficulty of restraint and […] the rule was made absolute that pregnancy after bundling imposed marriage […] So frequent was this occurrence that the church records repeatedly show the abbreviation FBM—fornication before marriage.”

There are counter-studies that purport to show abstinence education as effective, like this one from a crew that, not surprisingly, is selling abstinence education materials. But it, like most others, has little bon mots amid its intimidating numbers and verbose language like, “In addition, the high attrition rate limits our ability to generalize the findings to a higher-risk population” (strangely enough, the .pdf file is set to disallow copying and pasting, perhaps to discourage irate bloggers like myself). But the study doesn’t list the attrition rate, making it impossible to tell how severe the problem is. In addition, even if it did, the population selected might also suffer from cherry picking problems of various kinds: that is to say, organizations are more likely to serve the participants who are most likely to be receptive to services and, concomitantly, less likely to do things like have early sex. This is an easy and tempting way to make a program look good: only let the kids in who are likely to benefit. And it’s a hard problem to tease out in studies.

So be wary of dueling studies: if you don’t read these carefully, it’s easy to accept their validity, and even if you do read them carefully, it’s easy to nitpick. This is why peer review is so helpful in science and also part of the reason evaluations are so difficult. Furthermore, many of the studies, including Heritage’s, come from biased sources, a problem Megan McArdle writes about extensively in a non-abstinence-related context. (See her follow-up here). Most of you justifiably haven’t followed the blizzard of links I put up earlier or read the books I cited for good reason: who has the time to sift through all this stuff? No one, and even pseudoscience combined with anecdote like this article in New York Magazine has an opinion (hint: be wary of anyone whose title has the word “evolutionary” in it).

Given this research, which is hard to miss once you begin searching for information about the efficacy of abstinence instruction, how is a grant writer to create a logic model that, as page 44 says, should list “[a]ssumptions (e.g., beliefs about how the program will work and is supporting resources. Assumptions should be based on research, best practices, and experience)”? (emphasis added).

Two words: ignore research. And by “ignore research,” I mean any research that doesn’t support the assumptions underlying the RFP. If you want to be funded, you simply have to pretend “Impacts of Four Title V, Section 510 Abstinence Education Programs” or the GAO study don’t exist, and your proposal should be consistent with what the RFP claims, even if it’s wrong. This is, I suspect, one of the hardest things for novice grant writers to accept, which is that you’re not trying to be right in the sense of the scientific method of discerning the natural world through experimentation. You’re trying to be right in the Willie Stark sense of playing the game for the money. No matter how tempting it is to cite accurate research that contradicts the program, don’t, unless it’s to knock the research.

Remember too that the grant writer is to some extent also a mythmaker, which is a subject Isaac will address more fully in a future post. The vital thing to consider is that the mythology you need to create isn’t always the same as the reality on the ground. As in politics, the way events are portrayed are often different than how they actually are. David Broder wrote an article on the subject of inventing political narratives, which occasionally match reality; your job as a grant writer is inventing grant narratives. We hope these match reality more often than not. Sometimes the myth doesn’t, as in this application, and when that happens, you’re obligated to conform to the RFP’s mythology, even if it isn’t your own.

The second part of this post continues here.

Leave a Reply

Your email address will not be published. Required fields are marked *