Posted on Leave a comment

Another piece of the evaluation puzzle: Why do experiments make people unhappy?

The more time you spend around grants, grant writing, nonprofits, public agencies, and funders, the more apparent it becomes that the “evaluation” section of most proposals is only barely separate in genre from mythology and folktales, yet most grant RFPs include requests for evaluations that are, if not outright bogus, then at least improbable—they’re not going to happen in the real world. We’ve written quite a bit on this subject, for two reasons: one is my own intellectual curiosity, but the second is for clients who worry that funders want a real-deal, full-on, intellectually and epistemologically rigorous evaluation (hint: they don’t).

That’s the wind-up to “Why Do Experiments Make People Uneasy?“, Alex Tabarrok’s post on a paper about how “Meyer et al. show in a series of 16 tests that unease with experiments is replicable and general.” Tabarrok calls the paper “important and sad,” and I agree, but the paper also reveals an important (and previously implicit) point about evaluation proposal sections for nonprofit and public agencies: funders don’t care about real evaluations because a real evaluation will probably make the applicant, the funder, and the general public uneasy. Not only do they make people uneasy, but most people don’t even understand how a real evaluation works in a human-services organization, how to collect data, what a randomized controlled trial is, and so on.

There’s an analogous situation in medicine; I’ve spent a lot of time around doctors who are friends, and I’d love to tell some specific stories,* but I’ll say that while everyone is nominally in favor of “evidence-based medicine” as an abstract idea, most of those who superficially favor it don’t really understand what it means, how to do it, or how to make major changes based on evidence. It’s often an empty buzzword, like “best practices” or “patient-centered care.”

In many nonprofit and public agencies, evaluations and effectiveness are the same: everyone putatively believes in them, but almost no one understands them or wants real evaluations conducted. Plus, beyond that epistemic problem, even if evaluations are effective in a given circumstance (they’re usually not), they don’t necessarily transfer. If you’re curious about why, Experimental Conversations: Perspectives on Randomized Trials in Development Economics is a good place to start—and this is the book least likely to be read, out of all the books I’ve ever recommended here. Normal people like reading 50 Shades of Grey and The Name of the Rose, not Experimental Conversations.

In the meantime, some funders have gotten word about RCTs. For example, the Department of Justice’s (DOJ) Bureau of Justice Assistance’s (BJA) Second Chance Act RFPs have bonus points in them for RCTs. I’ll be astounded if more than a handful of applicants even attempt a real RCT—for one thing, there’s not enough money available to conduct a rigorous RCT, which typically requires paying the control group to follow up for long-term tracking. Whoever put the RCT in this RFP probably wasn’t thinking about that real-world issue.

It’s easy to imagine a world in which donors and funders demand real, true, and rigorous evaluations. But they don’t. Donors mostly want to feel warm fuzzies and the status that comes from being fawned over—and I approve those things too, by the way, as they make the world go round. Government funders mostly want to make congress feel good, while cultivating an aura of sanctity and kindness. The number of funders who will make nonprofit funding contingent on true evaluations is small, and the number willing to pay for true evaluations is smaller still. And that’s why we get the system we get. The mistake some nonprofits make is thinking that the evaluation sections of proposals are for real. They’re not. They’re almost pure proposal world.


* The stories are juicy and also not flattering to some of the residency and department heads involved.

Posted on 3 Comments

HUD Gets Back in the Job Training Biz: “Jobs Plus Pilot Program” NOFA Released

HUD just issued a NOFA (Notice of Funding Availability, which is HUD-speak for RFP) for the Jobs Plus Pilot Program. There’s $24 million up for grabs, with grants to $3 million, for Public Housing Authorities/Indian Housing Authorities (PHAs/IHAs). While the issuance of a new HUD NOFA is not usually all that interesting, this one is because it represents a shift in HUD’s priorities.

As I wrote last February, job training is one of the current favored project concepts in grant making. There are at least 47 federal job training programs, or possibly 48 including the newly minted Jobs Plus. You may not remember, though I do, that President Obama made a big fuss about job training in his most recent State of the Union address and vowed to unleash Vice President Biden to study federal job training initiatives in hopes of simplifying things.

Right.

That was the last I heard of this noble quest, and, as far as I can tell, the herd of federal job training programs continue to thunder across the plain. It’s job training business as usual, with the random new program tossed in for good measure.

This is not, however, what made me notice this notice.

At one time HUD had several competitive job training programs, including our old friend YouthBuild, which HUD managed for about 12 years. Suddenly, in the waning days of the reign of George Bush the Younger, Congress got the bright idea that maybe it isn’t such a good approach to have HUD, which is supposed to be involved in housing, fund job training programs. Not a bad reform, since HUD’s job training grant programs were not coordinated with other federal job training programs, particularly the ones operated by the Department of Labor. YouthBuild and other HUD job training programs were eventually transferred to DOL in a previous effort to “simplify things.” Now that eight years or so of DOL running former HUD job training programs have passed, it seems perfectly appropriate to make things more complex again by having HUD manage yet another job training program.

A cursory look at the Jobs Plus Pilot reveals that there’s not much new here, since it’s more or less a rehash of the “workfare” job training concept that emerged from the 1996 compromise Welfare Reform legislation negotiated by President Clinton and Speaker Gingrich. The basic idea was (and is) to tie public income supports, like TANF, to job training. This naturally works better when the economy is producing lots of entry-level jobs.

In the case of Jobs Plus, the target population is residents of the 250 or so remaining large public housing projects* that survived the lunacy of the now almost forgotten HOPE VI program that funded the demolition of thousands of public housing units across America. Even though we wrote some HOPE VI proposals, it always struck me as incredibly stupid to tear down the housing of last resort for the poorest Americans. The good news now is that, if your public housing development still stands, HUD is willing to toss you a job training bone. Of course, there’s nothing to prevent public housing residents from accessing the myriad of job training programs surrounding them. As a grant writer, however, I agree and have to ask, “why have 47 job training programs when 48 will do?”


* When writing a grant proposal about public housing, never use the term “housing project.” Instead, these are always referred to by the more PC “housing development.” Of course, I’m a geezer who grew up in the very poor North Minneapolis neighborhood adjacent to the huge Sumner Field Homes and associated public housing high rises.

I used to play at the Sumner Field park and kid and adults referred to this area as “the projects.” I’ve been to re-education camp since then and banished “projects” from my proposals. By the way, if you follow this link you’ll learn that a huge HOPE VI grant was used to destroy the entire Sumner Field Homes and associated buildings in 1998, displacing 97% of the over 3,300 poor residents in the name of the “new urbanism.” Not to worry: a much smaller mixed-use development replaced it, but there is no word on what happened to the thousands of residents who were tossed out.

Posted on 2 Comments

We’re Not Taking Sides: We’re Describing How Grant Programs, Like Those Related to Domestic Violence, Get Funded

In Isaac’s post about the NFL spurring new interest in domestic violence, he points out the likely public response to the issue: more grant money. He’s showing what is likely to happen, and he is tracing the formation of a new grant wave—as we have done before.

We want to clarify one point: we aren’t trying to minimize domestic violence as an issue. Our purpose in writing this blog is never to minimize or maximize issues. In one of our oldest posts, “What to do When Research Indicates Your Approach is Unlikely to Succeed: Part I of a Case Study on the Community-Based Abstinence Education Program RFP,” our goal was not to minimize or maximize teen sex education either: it was to describe real-world issues grant writers face. The job of the grant writer is first and foremost to tell the funder what they want to hear. A secondary job, however, is figuring out what project concepts and services are likely to be funded.

Depending on your perspective, the “right” issue may be highly fundable at a given moment, or the “wrong” issue might be. By definition, not every issue can be prominent at any given time—the word “prominent” does itself imply that an issue is necessarily and in some objective sense more important than another issue. It just means that some impetus or news or ideas have lifted it. If you’re a nonprofit, there is a limited amount that you can do to go against a particular funding tide.

Posted on Leave a comment

Don’t Trust What You Read in Grants.gov: Personnel Development To Improve Services and Results for Children With Disabilities

This week’s e-mail grant newsletter includes an RFP called “Personnel Development To Improve Services and Results for Children With Disabilities–Personnel Preparation in Special Education, Early Intervention, and Related Services.” Click the link and you’ll notice that there are nine grants available and $250,000 in “Estimated Total Program Funding;” those of us who can do simple math will think, “Gee, a little over $25,000 per grant: that’s not much money and consequently not a very interesting program.”*

But read the RFP itself and you’ll see there is actually $12,500,000 available, spread across 50 grants and four different sub-awards. The program suddenly got a lot more interesting and the maximum award amount is $250,000 per year, for up to five years. Now things have gotten really interesting. A lot of organizations that would pass on $25,000 now want to apply.

The topic of “don’t trust grants.gov” is becoming part of a continuing series, since these mistakes are shockingly common. If you see a program that looks appealing, always read the RFP. Not doing so could be a million-dollar mistake.


* Isaac is fond of quoting the Grandmaster Flash song, “White Lines,” which sits at an unusual intersection between rap and disco and contains the astute observation that “The money gets divided / The women get excited.” Note that the next couplet reminds us of the likely consequences: “Now I’m broke and it’s no joke / It’s hard as hell to fight it, don’t buy it!”

Without much money to be divided, no one gets excited.

Posted on 1 Comment

Sequestration Still Looms Over the Grant World: Two Months and Counting

I wrote about the potential impacts of the then-looming fiscal cliff a few weeks ago. At the last minute—actually, about a day after the last minute—Congress and President Obama awoke from their torpor and passed legislation, which what’s left of our media immediately hailed as “preventing the nation from going off the fiscal cliff.”

Well, not quite.

Lost in the fiscal cliff hubbub was the fact the fiscal cliff was actually a two-step fall. The first step downward—massive tax increases on almost all Americans—was indeed averted. The second part of the Cliff, sequestration, however, wasn’t addressed.

Rather, it was kicked down the road by about two months. As I pointed out in my earlier post, sequestration will impact the wonderful world of grants much more than tax increases. The former means significant reductions in discretionary grant funding, while the latter means more “money for nothing and chicks for free.”

Over the next few weeks, the battle lines over sequestration will form as the new Fiscal Cliff takes shape. This, too this will be a two-step Fiscal Cliff. Step one is our old pal sequestration, while the second part will be raising the federal debt ceiling. While the two steps are not inherently intertwined, the timing virtually ensures that the debates over both will be.

If you’re with a nonprofit that’s drifted back into somnolence because the Fiscal Cliff has been averted, shake yourself awake because the roller coaster is about to start again. In many ways,* coming to agreement on the largely non-raising of taxes was fairly easy for Congress and President Obama to agree on, compared to the coming battles over sequestration and raising the debt limit.

While understanding the byzantine politics of these two issues is above my pay grade, the eventual outcomes will likely include either decreases in federal spending on discretionary grant programs or, at a minimum, a reduction in the rate of increase. Either course will have significant impacts for grant seekers.


* Free proposal lead-in phrase in here.

Posted on 4 Comments

Sandy Hook School Shootings Tragedy Likely to Lead to New Grant Opportunities for School Security, After School and Mental Health Project Concepts

The recent tragic shootings at Sandy Hook Elementary in Newtown, CT will likely lead to new grant opportunities in school safety, safer facilities, and safety training, as well as mental health and at-risk youth services. It is a sad reality that shocking events often result in new grant programs and increased funding for existing programs, largely because politicians are chronically infected with “do something disease.” Even when the “something” will not necessarily change the dynamic leading to the problem, people feel better when something is being done and politicians are more than happy to oblige.

In the area of gun violence, we last saw this basic phenomenon following the Columbine School School Massacre in 1999. This was the first of what turned out to be a so-far unending series of similar school-based mass shootings. Most Americans were stunned by Columbine, particularly since it occurred in an upper-middle-class community with few obvious social problems facing youth.

Nonetheless, two teens decided to attack their peers because, as Dave Cullen describes in Columbine, one was a violent sociopath and the other was essentially in his thrall. Suddenly, it became clear that more or less all American youth were “at-risk” and the grant floodgates opened for nonprofits and schools interested in trying new approaches to reaching kids, even middle and upper middle class kids, with a variety of approaches.

The 21 Century Community Learning Centers (21st CCLC) program was one beneficiary of the national debate following Columbine. Funding for the 21st CCLC program, which was new at the time, was dramatically increased following Columbine. Even more importantly for many applicants, the Department of Education became very interested in funding 21st CCLC projects in relatively affluent areas, so much so that a special funding category for “suburban” schools was established.

While it later evolved to have an academic enrichment focus, the original idea behind the 21st CCLC program was essentially to keep kids in the school setting longer each day. The concept was to provide a safe place for them after school, while making sure they didn’t have enough unsupervised free time to build bombs and steal guns to bring back to school the next day. When the program was block granted to the states about seven years ago, a veneer of academic support was emphasized to both glam up the program and respond to the No Child Left Behind Act’s academic requirements.

We’ve written dozens of funded 21st CCLC grant proposals over the years, including one for a very affluent school district in Colorado not too far from Columbine. Yes, we shamelessly invoked Columbine in this proposal, as well as in other 21st CCLC and other at-risk youth proposals—particularly for projects in middle and middle upper class communities. We continue to do so, but now will add Sandy Hook as another example that even affluent kids face a daunting gantlet* of problems that make them at-risk and in need of wraparound supportive services.** This doesn’t diminish the enormity of the tragedy, but it does provide context and a salient example that reviewers will recognize.

In addition to 21st CCLC, many other federal and state grant programs were created after Columbine to fund such project concepts as hardened school security systems, disaster planning, school resource officers (cops that work in schools) and the like. When Congress returns in January, I expect to see new funding emerge for school safety and at-risk youth programs.

This is because there is a qualitative difference between Sandy Hook and other recent school shootings–in this case, not only were the victims from affluent families, but they were also mostly children. Citizens will demand action and about the only politically neutral and easy action governments can take is to expand funding for services that might help prevent other similar tragedies. “Politically neutral and easy” leaves aside the political minefield of more stringent gun control laws, a subject which is beyond the scope of this post and this blog.

If your nonprofit is concerned with these issues and has the capacity to make a local difference, use your holiday downtime to get your staff and board members together to brainstorm an innovative project concepts that might be relevant to upcoming grant opportunities. As Rahm Emanuel famously said, “You never want a serious crisis to go to waste.”***


* Free proposal phrase here. I know you think it should be daunting “gauntlet,” but that would make it a challenging glove. The word you’re looking for is gantlet. EDIT: Actually, as this commenter points out, either has become correct.

** Another free proposal phrase here.

*** It’s also possible that we’ll start to see changes in the mental health system, since so many shooters have been involved with the mental health system prior to committing their crimes. As Liza Long writes in Thinking the Unthinkable, “When I asked my son’s social worker about my options, he said that the only thing I could do was to get Michael charged with a crime.” That needs to change.