Tag Archives: Upward Bound

Upward Bound means more narrative confusion

The Upward Bound deadline passed, but the RFP lingers on in my mind like a foul meal.

The RFP was an extraordinary work of indirection, with 130-something pages of instructions supporting a 72-page narrative (counting “Competitive Priorities”). Upward Bound is one of the Department of Education’s “TRIO” programs—there used to be three: Upward Bound, Educational Opportunity Centers, and Student Supportive Services, but now there are five or six. Another TRIO program, Educational Opportunity Centers (“EOC”), was released last May, and that RFP is particularly close to my heart because I used its “Plan of operation” section to teach my University of Arizona students technical writing. The EOC RFP was also overly long and overly verbose, but its similarity to Upward Bound meant that looking at that proposal would help me with the new one.

It also included a trap, because the Department of Education made subtle but real changes between the way they phrased requirements from one program to the other. For example, under “Project Need” in EOC, the first two major headers said something like, “Low-incomes in the target area” and “High percentage of target area residents with education completion levels below the baccalaureate level.” The UB RFP says, “The income level of families in the target area is low” and “The education attainment level of adults in the target area is low.” So an applicant who applies for both EOC and UB can reuse data—but a straight copy-paste will result in the Department of Education knowing that you’ve done so. I wouldn’t be surprised if the Department of Education does this intentionally, like Van Halen and their legendary M&M Rider:

The rider’s “Munchies” section was where the group made its candy-with-a-caveat request: “M&M’s (WARNING: ABSOLUTELY NO BROWN ONES).” While the underlined rider entry has often been described as an example of rock excess, the outlandish demand of multimillionaires, the group has said the M&M provision was included to make sure that promoters had actually read its lengthy rider. If brown M&M’s were in the backstage candy bowl, Van Halen surmised that more important aspects of a performance–lighting, staging, security, ticketing–may have been botched by an inattentive promoter.

Van Halen uses brow M&M’s as a signal, and the Department of Education is using section headers the same way. If your section headers are identical to the EOC section heads, your proposal will be thrown out altogether, or at least have its points lowered.

There are other perils stashed in this RFP, too: its writers practically hide the location of the material you’re supposed to respond to. The RFP directs you to page 102, but the actual narrative requirement in the form of the “selection criteria” to which you’re supposed to respond starts on page 70 (of a 132-page RFP). And the narrative section lists “Objectives” on page 71, but you have to be cognizant enough to know that you have to copy the objectives listed on page 93.

Read and tread carefully when preparing to write a grant proposal.

EDIT: A former Department of Education reviewer wrote us to say:

I read with interest your article on Upward Bound grants in which you spoke of “traps” by the Department of Education. You clearly are experienced and are doing a great service for fledgling grants writers. However, I have served as a reader for several TRIO programs, and my experience is that the Department of Education NEVER puts traps in their RFPs. They work very hard to see that readers are fair and generally positive about the grant process. Of course, they also want consistency to keep down on appeals. The reason that I am writing is that you are doing your readers a disservice by making them think that there is a “magic phrase” that might result in acceptance for funding or rejection of a grant. The Department of Education wants writers to address the problems in a straightforward manner and teachers readers to reward clear writing.

Eat What You Kill: If You’re Not Hunting Grant Programs, You’re Not Eating

Mark Zuckerberg is supposedly only eating animals he kills this year. It’s a “personal challenge” for him, rather like not eating Big Macs for the rest of us.*

It’s easy to wonder how eating-what-you-kill as a metaphor might apply to the rest of Zuckerberg’s life, but since I’m not friends with him I can’t ask. Nonetheless, you can probably can imagine how this metaphor applies to grants: in times when you’re prospecting for grants, you should be a bear and bite any salmon, as Isaac wrote at the link. You can’t afford to be as picky as you might otherwise be.

In an idealized world, you’d probably get your preferred mix of grants, donations, and contracts, with a heavy emphasis on donations unencumbered by donor restrictions. I’d also like someone to randomly give me a million bucks, but that doesn’t seem real likely, and if someone did give me a million bucks, I’d worry that I was walking into a movie—specifically, a thriller with lots of murky motives that might leave the protagonist dead at the end. So I write proposals and teach undergraduates instead of waiting for magical money that, even if it did appear, would probably result in a scenario like the one described in Scott Smith’s A Simple Plan (hint: the title is ironic. Also, the book is quite good and highly recommended).

So if, like me, your organization doesn’t to exist in an idealized world, the desired blend of funding streams is not going to magically appear. Which means you should take what you can get. Until the recent financial crisis, for example, a lot of organizations could rely on capitated funds and contracts through cities, counties, and sometimes states, which provided steady, reliable sources of incomes to supplement donations and the occasional grant. Now a lot of organizations that once relied on such sources simply don’t have them. They can’t go to Safeway and pick up a nicely cut chicken.

They have to eat what they kill, like Zuckerberg, although Mark is doing so by choice and can send someone out to Whole Foods for a chicken or 12 anytime he feels like adopting a new philosophical stance. You can’t, and as a result the number and quality of grants that organizations apply for takes on greater urgency. One reason we like discussing Upward Bound so much is simple: it offers the possibility of five years of uninterrupted funding. That can carry an agency that might otherwise become skeletal through the lean times that continue.

Most organizations simply don’t have good alternatives to grants any more. They shouldn’t be worried about finding some existential balance between donations and grants; they should be taking whatever they can get. There aren’t a lot of choices; some organizations are trying to survive one Bratwurst at a time, as we described a few years ago, but fundraisers like washing cars and selling food are really tough, especially since you’ll naturally be up against professionals whose job is selling bun-wrapped meat or cleaning vehicles.

We’re not fools: we know that most nonprofits would rather have donors rain money on their head. I’d like to win the Lotto or an inheritance and be an artist full-time, but that’s not incredibly likely for me in the short term. For nonprofits, easy money from the Stimulus Bill is gone. You’re back to the basic stuff. You can debate this all day, but the proposals have to be written. You have to eat what you kill. If you’re not ready to wield the knife, you should hire somebody who will.


* I finished reading Steve Jobs by Walter Isaacson, and among many other fascinating tidbits Isaacson describes Jobs’s numerous nutritional oddities surrounding food and Jobs’s belief in the possibility food offers for transcendence. Jobs went through periodic dietary restrictions, like eating and drinking only fruit or fruit juices, being a vegan, and fasting. I wonder about the extent to which this impacted his illness and how, if at all, his unusual eating beliefs were tied to the extreme achievement in other aspects of his life.

FY ’12 Upward Bound Draft RFP Found with $305,289,000 for New Awards — A Nice Apparition for Halloween

Subscribers to our free weekly Email Grant Alerts and faithful blog readers know that I have been predicting for a few months that the FY ’12 RFP for the Upward Bound program would be soon be issued. It’s getting there, and we now have a copy of the complete Draft FY ’12 Upward Bound RFP.

For the last several weeks, we’ve known the draft FY ’12 Upward Bound NOFA was floating in the ether, although the Department of Education didn’t seem to want to post it publicly for some unknown reason. But, after 19 years in business, we’ve got our sources and finagled a copy of the draft RFP and related docs in advance of publication.

By the time your read this, you should be able to find an announcement about Upward Bound in the October 31 Federal Register. The draft essentially provides a 30 day comment period on the Department of Education’s plan for “reinstatement of a previously approved application for grants under the Upward Bound (UB) Project (1840-0550), which has expired.” This bit of federal Doublespeak means that there have been some legislative changes since the last Upward Bound RFP process in 2007. The Department of Education needs to go through a public comment period before issuing the RFP they’ve already produced—and it will probably be in more or less the same form as the version we have.

You’ve gotta love the timing of the Department of Education performing a little prestidigitation by releasing the phantom RFP on Halloween. Boo!

I’ve paged through the 114 single-spaced page Upward Bound RFP and it looks remarkably like every other TRIO Program RFP I’ve ever seen. But the best part in the RFP is that there will be $305,289,000 for new UB awards, with an average award of $330,000/year for five years. The Department of Education is still being coy about the deadline, but let’s do some math: the 30 day comment period starts on October 31, it’ll take about 15 days or so for the program officers to examine and reject comments, and about 15 days or so to set up the next FR publication. Thus, the FY ’12 RFP should be published between Christmas and January 15. These days, most Department of Education RFPs have 30 day deadlines, so expect the deadline to be late January to mid-February.

Upward Bound will be one of the best opportunities this year to grab a pretty big Department of Education grant salmon this year. Nonprofits and institutions of higher education (IHE, which means “college or university” in Edu-speak) are eligible applicants. Upward Bound is a great way of funding academic support programs for high school students to enable them to build the skills needed to graduate from high school and thrive in the postsecondary education milieu (free proposal phrase here).

Just don’t wait for the actual RFP to be issued. Find the draft RFP, read it, and, if you think you organization could run the program, go to work on planning the project. With over $300 million up for grabs, there should be at least 1,000 grants awarded. We’ve written many funded TRIO grants, including Upward Bound, and know that the funding decisions for these programs are often the stuff of strange tales. But if your organization doesn’t get moving and submit a great, technically correct proposal, you will miss out on a twice-a-decade opportunity. It’ll take that long for the next Upward Bound bus to roll by. Get on this one.

Grant-seeking dinosaurs look up: The bright light in the sky is an asteroid

For reasons not clear to me, I am on the (usually) happy-talk email distribution list of Dorothy Stoneman, doyenne of YouthBuild USA, the trade group for the 273 or so YouthBuild providers across America. I’ve never met Dorothy, who is generally affable in her emails and is a tireless advocate and change agent (note: the preceding phrase is a free example of proposolese for you to use at your discretion, although I prefer “tireless organizational change agent” to really get grant reviewers hot and steamy).

Since Congress began its on- and off-again assault on discretionary spending last fall, Dorothy’s periodic emails have become increasingly strident and alarmist, perhaps for good reason. Although we’ve written dozens of funded YouthBuild proposals over the years, including two in the most recent funding round, training at-risk youth and young adults for construction trade careers seems a little odd in the face of the collapse of the housing market. America has many problems at the moment, but a lack of affordable housing and skilled construction workers are not among them.

Dorothy’s June 5 call-to-arms communique says:

In the FY11 House budget in process before the November 2010 elections, the US Department of Labor (DOL) would have received $120M for the YouthBuild program. But after the elections, when the House changed hands, the House re-did the budget and funding for YouthBuild was eliminated.After a grueling negotiation, in which the Senate and the Administration supported YouthBuild, the final budget included $80 million for YouthBuild in DOL. This was a $47.5M cut from the funds appropriated in Fiscal Year 2010. (FY10). As a direct result, in May, one hundred twenty one (121) of DOL’s 226 previously-funded YouthBuild grantees lost their funding. Only 105 YouthBuild programs are funded by DOL for 2011-2013 [Shameless Plug: two of these courtesy of Seliger + Associates grant writing prowess!]. Hundreds of organizations applying for the first time were also turned down. Can you imagine the devastation to those 121 communities of losing this pathway out of poverty for the most disadvantaged young adults? Imagine the heartbreak of the adults* who have worked so hard and fought to bring these opportunities to the young people in their communities?

According to Dorothy, the end of the world is nigh for YouthBuild grantees and the at-risk young folk who presumably will end badly without YouthBuild training. I imagine Gene Wilder in Woody Allen’s hilarious “Everything You Wanted to Know About Sex” Drinking Woolite from a bottle after he loses his literally beloved sheep, Daisy.

Dorothy shouts about the deleterious impact of lowered YouthBuild funding on “America’s most disadvantaged communities” (another proposalese gem); she advises lovelorn YouthBuild grantees and supporters to call their congresspeople and senators, of course, to beg for more YouthBuild funding. Call me cynical, but I think the chance of increased funding for YouthBuild is about zero.

Dorothy’s breathless email immediately conjured up the classic cartoon image of a gaggle of brontosauruses (née brontosauri, much like “Winklevi?”) peacefully munching on treetops as that bright light in the sky gets bigger during the Cretaceous-Tertiary (K-T) extinction. All the lobbying by Brontosauri in the USA would not have helped 65 million years ago, and YouthBuld grantees are experiencing a K-T event whether or not they want to admit it. I know, because I can read the grant tea leaves and I’ve talked to about a half-dozen YouthBuild grantees in the last month.

Here’s what we recommend to our YouthBuild and other clients who provide job training:

* Apply for YouthBuild and whatever other job training grants are still available, as it takes government agencies a bit of time to turn the Titanic. Keep in mind, however, that it is fairly pointless to train people for jobs that don’t exist. For example, most Workforce Investment Act (WIA) programs, whether funded by the Department of Labor or a local Workforce Investment Board (WIB), offer grants via reimbursement only if the agency meets its performance targets. Since agencies conducting construction training are spectacularly unlikely to meet performance targets, there is little point in receiving such an award.

These days most training providers offer training for low-level health care and service sector jobs (e.g., push the cash register button with the picture of a cheeseburger when you sell a cheeseburger, etc.). Although YouthBuild is not directly performance-based, we’ve noticed that our YouthBuild clients are emphasizing their ability to get their graduates into community college, a DOL-accepted though rarely discussed YouthBuild program outcome, instead of placing them in a “career ladder job with living wage potential in the world of work” (another free proposalese phrase—there’s one in every box of GWC).

* While waiting around for job training grants, turn your YouthBuild program from a brontosaurus into a fleet-footed mammal by using the agency’s skills and track record to provide remedial education and related supportive services. Since every YouthBuild program has the same three components—adult basic education (ABE), job training and leadership development (YouthBuild-talk for wraparound supportive services)— kick out the middle leg, making the tricycle a bicycle, and go after programs like the Department of Education’s Upward Bound program, for which an RFP with about $350 million up for grabs should be issued in a couple of months. Or form a partnership with an LEA and take a flyer on the Department of Education’s Investing in Innovation (i3) Fund, which has an RFP with $172 million on the street as I write this.

With almost one-third of American youth failing to graduate from high school and many graduates unable to read or write, it’s fairly obvious that there are going to be many grant opportunities for remedial education over the next few years. Even the usually indefatigable Secretary of Education, Arne Duncan, is soon going to issue waivers to all public schools from the Bush-era No Child Left Behind requirement that all children pass proficiency tests. Without the waivers, 83% of American public schools would be labeled as “failing” next year (I am not making this up).

Or watch for the cascade of discretionary grant programs authorized and funded under the Affordable Care Act (“Obamacare,” or the Grant Writer’s Relief Act, as we call it around the office). The Affordable Care Act is larded with grant opportunities, many of which involve “outreach.” That’s another way of saying “walkin’ around money for clever mammalian nonprofits.”

In face of rapid grant climate change, it is not a good idea to either continue nibbling on tree tops or run around in pointless circles as the asteroid looms. Get busy, re-think your agency’s strengths, and unleash your inner grant writer. Let the Dorothy Stonemans of the world, who have a real vested interest in keeping YouthBuild (YouthBuild USA, for example, is not going to sell many licenses to use the word “YouthBuild” to nonprofits without the DOL program) and similar archaic funding programs going. Instead, widen your agency’s funding streams beyond job training. There are tons of grants available, even if you have to root around to find them, rather than blissfully picking off the last choice leaf on a withering treetop.


*Note to Would Be Grant Writers: Do not use this kind of overwrought lingo in your proposals. Whenever I read, “the heartbreak of _______________,” I think of the Heartbreak of Psoriasis TV commercials that ran when I was a teenager. I didn’t know what psoriasis was, but I did know that not having a date to my Junior Prom at age 15, not a skin condition, was genuine heartbreak. I did, however, get asked to the Sadie Hawkins Day Dance that year, so all was not lost.

Here They Come: RFPs Are Thundering Down the Plain, So Look out for the Carol M. White Physical Education Program (PEP), Upward Bound, Choice Neighborhoods, REACH CORE and More

In the somewhat interminable but occasionally engaging Dances with Wolves, Kevin Costner finally ingratiates himself with his Sioux neighbors by telling them that the “tatonka” (buffalo) are suddenly thundering nearby. Last February, I asked Where Have All the RFPs Gone? Well, the FY ’10 grant tatonka are finally here and the distant noise you hear is the sound of federal grant opportunities. Work fast, because this herd will have come and gone by the end of the federal fiscal year on September 30.

For example, the Department of Education finally issued RFPs for the Carol M. White Physical Education Program (PEP), the High School Graduation Initiative, Personnel Development To Improve Services and Results for Children With Disabilities, and the Fund for the Improvement of Secondary Education (FIPSE) last week.

In 2008, the FIPSE RFP was issued on March 21. This year, it was issued on June 14. Under normal circumstances, this could be chalked up to random variation in funders. This year, that’s much less likely because of the stimulus madness that continues to work through the federal system. The good news about FIPSE: in 2008 it had $2,584,000 for seven grants. This year it has $27,307,000 for 37 grants. This isn’t the only program that’s seen a massive money increase: Personnel Development To Improve Services and Results for Children With Disabilities has gone from $1,500,000 in total funding to $22,900,000.

We heard from a client recently (we wrote their funded Upward Bound proposal in the last funding round about four years ago) that RFPs for both Upward Bound and Talent Search will soon be issued by the Department of Education. It is unusual for RFPs for two “TRIO” programs to be issued in one fiscal year, but this is no usual year.

On the community development front, HUD has about 35 or so competitive grant programs, but only one or two NOFAs (HUD-speak for “RFP”) have been issued this year, which means there are more than 30 to go. Another client, for whom we wrote a funded Lead-Based Paint Hazard Control (LBPHC) Program proposal last year, was just at the grantee meeting. The HUD program officer told the group that all of the NOFAs are late this year (duh!) but would be issued with short turnarounds—just like the Department of Education RFPs listed above. Expect to hear HUD hooves in the distance for such old faves LBPHC, Healthy Homes, various Housing Choice Voucher—formerly called Section 8— programs and lots more soon. There will be a HUD NOFA stampede.

In a tease of goodies to come, HUD just released a “Pre-NOFA” for an entirely new competitive program, Choice Neighborhoods. This is not to be confused the Department of Education’s Promise Neighborhood Program, for which the RFP process concludes next week, even though both are new programs that can be used to fund more or less the same activities. Choice Neighborhoods will have $65,000,000 up for grabs once the HUD program officers can shovel the NOFA out the door, which should be within a few weeks. I’ve never seen a “Pre-NOFA” before, but once again this is an unusual year with strange portents in the grant world. I guess a Pre-NOFA is like getting one of those annoying “Save May 12, 2018 for Hershel Himmelfarb’s Bar Mitzvah” in the mail. This is HUD’s way of saying, “Stay tuned––MONEY COMING, MONEY COMING.”

I love the Promise Neighborhoods and Choice Neighborhoods programs because both offer planning and implementation grants, so grantees can keep the party going for years. Not to be outdone, HRSA also just issued an announcement for the wonderfully named Racial and Ethnic Approaches to Community Health for Communities Organized to Respond and Evaluate (REACH CORE) Program. REACH CORE grantees get two-year, $400,000 planning grants followed by multi-million dollar five-year implementation grants. Seven year grants! Now this is worth competing for.

Looks to me like it is a fine grant hunting season this summer. Get out your virtual Sharps 50 Caliber Buffalo Rifle in the form of a trusty iMac or MacBook out and start plinking. You’ll be exhausted, but you’ll have a week or two at the start of October before the FY ’11 RFPs start down the chute.

May 2010 Links: The Promise Neighborhoods Program, Federal Budgets, Upward Bound, Centers for Independent Living (CLI), the U.S. Institute of Peace (USIP), Restricting Fun Too Expensive, and more

* Federal programs never get delayed, unless they do. One of our clients received a letter from the Department of Education announcing that the Upward Bound program, which encourages at-risk youth to complete high school and go on to college, is being delayed until fiscal year 2012. This indicates that, as Isaac wrote in “Where Have All the RFPs Gone?,” the feds have gotten so backed up that they can’t spend all their money. To quote Isaac’s post:

Since federal agencies are running their regular programs while trying to spend additional Stimulus Bill funding and implementing entirely new programs, one imagines that our cadre of GS 10s and 11s, who are supposed to move the endless paperwork associated with shoveling federal funds out the door, simply have not gotten around to the FY ‘10 RFP processes.

Oops.

* Now that i3 madness is behind most of us, it’s time to see the other crazed, zombie-like offspring of the Department of Education. Alert reader and grant writer Shirley Nelson pointed me to the “Promise Neighborhoods Program,” which demands that one build “a complete continuum of cradle-through-college-to-career solutions (continuum of solutions) (as defined in this notice), which has both academic programs and family and community supports (both as defined in this notice), with a strong school or schools at the center.” I want those college solutions, whatever they are.

The RFP also says:

The continuum also must be based on the best available evidence including, where available, strong or moderate evidence (as defined in this notice), and include programs, policies, practices, services, systems, and supports that result in improving educational and developmental outcomes for children from cradle through college to career

In other words, the i3, quasi-evidence-based madness continues.

* The Department of Education’s Centers for Independent Living Program (CLI) program is particularly impressive because, as far as I can tell, nowhere in the 142-page application guidance does a definition of what “centers for independent living (CILs or centers)” means.

* I didn’t realize there was a U.S. Institute of Peace (USIP) until I saw the announcement of its Annual Grant Competition. I wonder what it’s like compared to the Department of Defense, which used to be called the Department of War.

* On healthcare nationally and in Massachusetts:

When Massachusetts rolled out its coverage program in 2007, many more people signed up for the new heavily subsidized insurance than was originally predicted by budget officials. Almost immediately, costs far exceeded what had been budgeted, forcing state officials to scramble to find cuts elsewhere in government and other sources of revenue.

After three years, no real progress has been made on rising costs. The program remains well over budget, with no end in sight. Further, state residents who now must buy state-sanctioned coverage are bristling at their rising premiums and the inability to find coverage which covers less and thus costs less.

State politicians are responding to the cost crisis the only way they know how: by promising to impose arbitrary caps on premiums and price controls for medical services. The governor and state regulators have disallowed 90 percent of the premium increases insurers –all of whom are not-for-profit–submitted for their enrollees for the upcoming plan year. The state says premium increases above eight percent are too high and unacceptable, though they themselves don’t have a plan to make health care more efficient in Massachusetts. They just want lower premiums. The insurers have responded by refusing to sell any coverage at the rates the state wants to impose.

* In essence, the country needs to figure out how to pay for the government that its citizens want. It’s a version — albeit a less extreme one — of the problem facing Greece right now.

* Wow: Records show that since 1992, only 10 Minnesota teachers fired for poor performance have challenged their dismissals all the way through that process.

* The nasty things local telcos do to prevent municipal fiber (and why this is so important).

* Politicians find that restricting fun is now too expensive (Note: this is not from The Onion).

* Electric Avenue: Learning to love a bike you don’t need to pedal.

* Academia isn’t broken. We are.

* The Shirky Principle: “Institutions will try to preserve the problem to which they are the solution.” — Clay Shirky. As Kevin Kelly says:

The Shirky Principle declares that complex solutions (like a company, or an industry) can become so dedicated to the problem they are the solution to, that often they inadvertently perpetuate the problem.

* The Department of Education appears to have invented a new word for the i3 RFP:

Growth may be measured by a variety of approaches, but any approach used must be statistically rigorous and based on student achievement data, and may also include other measures of student learning in order to increase the construct validity and generalizability of the information.

“Generalizability?” If that appeared in a student essay, I’d circle it.

* Why men don’t listen. Except they do, as this post into the pseudo science of gender brain differences shows.

* Megan McArdle has a characteristically astute essay on Lori Gottlieb’s book Marry Him!. As McArdle says, Gottlieb’s superficial thesis is that women are too picky in getting married. But “her real message she proves all too well, and I suspect that’s why it drives young women nuts, as in this Emily Gould essay I came across yesterday. It is the same thing overanxious mothers have been telling their daughters from time immemorial: your looks matter, and they are a wasting asset.”

I have no idea if this is true.

* Peak everything? Not really.

* The drive to make cities greener. And this is from the Wall Street Journal.

* One Man, Two Courts points out something that Isaac and few others seem to remember: the political party abortion flipflop. Until around 1977 or so, most Democratic politicians were mostly against abortion, while most Republicans supported it.

* In a Tough Economy, Old Limits on Welfare reads like a proposal. Except that the reporter forgets that there isn’t such a thing as “welfare;” he’s probably actually referring to TANF.

* Why humanity loves and needs cities.

* Why do colleges care about extracurricular activities? See my guess in the comments.

* For every doctor, there are five people performing health care administrative support. This may be part of our national problem, like the growth of administrators relative to professors in academia. (Hat tip Tyler Cowen.)

* Recommended: ManPacks.com. If you’re male, as I am, there’s a pretty good chance that you hate shopping for clothes and thus constantly have ratty socks, underwear, and t-shirts. ManPacks.com will send you two shirts with two pairs of underwear and socks every three months indefinitely. Once you set it up, you never have to think about the issue again, unless you move. And that set up takes maybe five minutes.

Brilliant.

* “China’s Youth Meet Microsoft,” an article, along with a rebuttal: “This article, IMHO, is written by someone who has no idea how things work just about anywhere that’s not the industrialized West, and is shocked and appalled that things aren’t as awesome as they are in the US of A.”

* A fascinating profile of Tyler Cowen, one of the proprietors of Marginal Revolution.

* “Describing himself as “terribly exhausted,” famed linguist and political dissident Noam Chomsky said Monday that he was taking a break from combating the hegemony of the American imperialist machine to try and take it easy for once.”

Studying Programs is Hard to Do: Why It’s Difficult to Write a Compelling Evaluation

Evaluation sections in proposals are both easy and hard to write, depending on your perspective, because of their estranged relationship with the real world. The problem boils down to this: it is fiendishly difficult and expensive to run evaluations that will genuinely demonstrate a program’s efficacy. Yet RFPs act as though the 5 – 20% most grant budgets usually reserved for evaluations should be sufficient to run a genuine evaluation process. Novice grant writers who understand statistics and the difficulties of teasing apart correlation and causation but also realize they need to tell a compelling story in order to have a chance at being funded are often stumped at this conundrum.

We’ve discussed the issue before. In Reading Difficult RFPs and Links for 3-23-08, we said:

* In a Giving Carnival post, we discussed why people give and firmly answered, “I don’t know.” Now the New York Times expends thousands of words in an entire issue devoted to giving and basically answers “we don’t know either.” An article on measuring outcomes is also worth reading, although the writer appeared not to have read our post on the inherent problems in evaluations.

That last link is to an entire post on one aspect of the problem. Now, The Chronicle of Higher Education reports (see a free link here) that the Department of Education has cancelled a study to track whether Upward Bound works.* A quote:

But the evaluation, which required grantees to recruit twice as many students to their program as normal and assign half of them to a control group, was unpopular from the start […] Critics, led by the Council for Opportunity in Education, a lobbying group for the federal TRIO programs for disadvantaged students, said it was unethical, even immoral, of the department to require programs to actively recruit students into programs and then deny them services.

“They are treating kids as widgets,” Arnold L. Mitchem, the council’s president, told The Chronicle last summer. “These are low-income, working-class children that have value, they’re not just numbers.”

He likened the study to the infamous Tuskegee syphilis experiments, in which the government withheld treatment from 399 black men in the late stages of syphilis so that scientists could study the ravages of the disease.

But Larry Oxendine, the former director of the TRIO programs who started the study, says he was simply trying to get the program focused on students it was created to serve. He conceived of the evaluation after a longitudinal study by Mathematica Policy Research Inc., a nonpartisan social-policy-research firm, found that most students who participated in Upward Bound were no more likely to attend college than students who did not. The only students who seemed to truly benefit from the program were those who had low expectations of attending college before they enrolled.

Notice, by the way, Mitchem’s ludicrous comparison of evaluating a program with the Tuskeegee experiment: one would divide a group into those who receive afterschool services that may or may not be effective with a control group that wouldn’t be able to receive services with equivalent funding levels anyway. The other cruelly denied basic medical care on the basis of race. The two examples are so different in magnitude and scope as to make him appear disingenuous.

Still, the point is that our friends at the Department of Education don’t have the guts or suction to make sure the program it’s spent billions of dollars on actually works. Yet RFPs constantly ask for information on how programs will be evaluated to ensure their effectiveness. The gold standard for doing this is to do exactly what the Department of Education wants: take a large group, randomly split it in two, give one services and one nothing, track both, and see if there’s a significance divergence between them. But doing so is incredibly expensive and difficult. These two factors lead to a distinction between what Isaac calls the “proposal world” and the “real world.”

In the proposal world, the grant writer states that data will be carefully tracked and maintained, participants followed long after the project ends, and continuous improvements made to ensure midcourse corrections in programs when necessary. You don’t necessarily need to say you’re going to have a control group, but you should be able to state the difference between process and outcome objectives, as Isaac writes about here. You should also say that you’re going to compare the group that receives services with the general population. If you’re going to provide the ever-popular afterschool program, you should say, for example, that you’ll compare the graduation rate of those who receive services with those who don’t, for example, as one of your outcome measures. This is a deceptive measure, however, because those who are cognizant enough to sign up for services probably also have other things going their way, which is sometimes known as the “opt-in problem:” those who are likely to present for services are likely to be those who need them the least. This, however, is the sort of problem you shouldn’t mention in your evaluation section because doing so will make you look bad, and the reviewers of applications aren’t likely to understand this issue anyway.

In the real world of grants implementation, evaluations, if they are done at all, usually bear little resemblance to the evaluation section of the proposal, leading to vague outcome analysis. Since agencies want to get funded again, it is rare that an evaluation study of grant-funded human services programs will say more less, “the money was wasted.” Rather, most real-world evaluations will say something like, “the program was a success, but we could sure use more money to maintain or expand it.” Hence, the reluctance of someone like Mr. Mitchem to see a rigorous evaluation of Upward Bound—better to keep funding the program with the assumption it probably doesn’t hurt kids and might actually help a few.

The funny thing about this evaluation hoopla is that even as one section of the government realizes the futility of its efforts to provide a real evaluation, another ramps up. The National Endowment for the Arts (NEA) is offering at least $250,000 for its Improving the Assessment of Student Learning in the Arts (warning: .pdf link) program. As subscribers learn, the program offers “[g]rants to collect and analyze information on current practices and trends in the assessment of K-12 student learning in the arts and to identify models that might be most effective in various learning environments.” Good luck: you’re going to run into the inherent problems of evaluations and the inherent problems of people like Mr. Mitchem. Between them, I doubt any effective evaluations will actually occur—which is the same thing that (doesn’t) happen in most grant programs.


* Upward Bound is one of several so-called “TRIO Programs” that seek to help low-income, minority and/or first generation students complete post-secondary education. It’s been around for about 30 years, and (shameless plug here) yes, Seliger + Associates has written a number of funded TRIO grants with stunningly complex evaluation sections.