Posted on 2 Comments

Know Your Charettes! Especially Near the 21st Century Community Learning Centers (CCLC) Program

It’s not unusual for an RFP to ask how community feedback was incorporated into the design of a project, and a good answer is to have some form of group activity feedback meeting. Notice the last four words: “group activity feedback meeting.” What a vile phrase, even by proposal standards. Don’t use such a phrase—call them Community Meetings, Participant Committees, Group Action Committees, or something more recondite like charrette (“a final, intensive effort to finish a project, esp. an architectural design project, before a deadline”), which is my favorite term. I’ve even seen it used outside of proposals, which is especially amusing because when I first heard “charrette” years ago I suspected that Isaac had made it up.

He hadn’t, though, and the word is sufficiently but not overly esoteric to make it an interesting addition to a proposal, like capers to a recipe. The idea behind the word is often needed, whether before or after a project begins. For example, group meetings can “occur” before the project begins to show that the community is interested in the program the applicant seeks to implement. (I put scare quotes around occur because I suspect such meetings are written about somewhat more often than they are actually held.) Not surprisingly, most community groups find that free money from the government is a good thing, and Tim Harford discussed this tendency in The Logic of Life. One way to strength a proposal narrative is to include a description of the charrette that came to this conclusion.

The same thinking can apply to groups formed after the project begins. For example, a Project Advisory Committee (PAC) composed of participants, staff, collaborators, and other members of the community can help provide a continuous feedback loop to constantly improve. Once again, I’m sure more nonprofits write about PACs than actually run them, but the proposal world is not always identical to the real world, which is one reason I was so surprised to read about the design charrette I linked to in the first paragraph.

To give a specific example of where charrettes could see action, I’ll return to an RFP mentioned previously. I made fun of the California 21st Century Community Learning Centers (CCLC) – Elementary & Middle Schools program, but you could cite charrettes in section 1.c., which wants you to “[d]escribe how you assessed the needs and strengths of your community […]” and section 1.e., which wants you to “[b]riefly discuss the proposed process for conducting, and the results of, your needs assessment for family literacy services.”

In either case, an applicant could say that a community charrette found significant demand for the learning center proposed. Charrette members concluded would help make the community more livable, cure cancer, and perhaps save the world from global warming. I can’t imagine why a real group of community members would ever decide, for example, that the community doesn’t need more money, and ideally someone else’s money, but I guess it’s conceivable, like a reduction in government spending not forced by revenue problems or a time portal opening in my backyard. Back on point: the needs assessment can describe how the charrette brings together key stakeholders—e.g., business persons, nonprofit and faith-based organization representatives, community leaders, and the like—and then announce how much they want that money, meaning that the proposal springs from authentic desire, while the applicant is merely the body through which the charrette’s will is implemented.


On a historical note, the whole charrette in public service concept got started in earnest with the War on Poverty and the idea that the poor and disadvantage should have a say in how public and other money is spent. The goal was to create “workable plan committees” and the like for civic leaders and their constituents, and the key phrase was”” maximum feasible involvement”” of the poor. Daniel Moynihan‘s book, Maximum Feasible Misunderstanding, is actually a play on this term. As the review, published in 1970, says:

[…] Moynihan traces the development of the community action approach. From the gray areas program of the Ford Foundation, through the Mobilization for Youth in New York City, on to the President’s Committee on Juvenile Delinquency and Youth Crime, there emerged the ideas of the urgency for localized, community-based programs aimed at the involvement of youth in constructive, problem-solving activity. This stream of development was in the background of the community action orientation of the War on Poverty.

Posted on 6 Comments

Credentials for Grant Writers from the Grant Professionals Certification Institute—If I Only Had A Brain

A manager at the Community Redevelopment Agency of the City of Los Angeles, an agency we sometimes work for, recently sent me a link to the “Grant Professionals Certification Institute” (GPCI; link deliberately excluded), an organization that offers “credentials” for would-be grant writers. He wanted my reaction to the idea of grant writing credentials, which I gave him immediately: they’re (mostly) a waste of time. But I decided to take a look at this offer, since I’ve been writing proposals without a license for 35 years.

The GPCI was apparently formed just to offer credentials. The fee is $525 to take an “examination developed through rigorous national standards for professional credentials.” I have no idea what could be meant by “rigorous national standards” or who developed them, but a grant writer must have written this sentence because it is definitely proposalese: filled with vague citations to an unnamed authority and using many complex words where a few simple ones would convey the message.

Without going into the equally unintentionally funny “competencies and skills tested,” the best part is that a “writing exercise represents 20% of the examination score.” So, 80% of a test to prove one can write is not writing! This further confirms grant writers thought up the idea. You pay $525 to pass a test and get a certificate, presumably gilt-edged and suitable for framing.

Does this mean anything? My guess is not much to the recipient, but it’s a great deal for the people selling the credential, just like Tom Sawyer getting friends to whitewash a fence. Apparently, the GPCI never heard of such existing credentials as a baccalaureate, a Masters or a Ph.D. in English or journalism, so they decided to offer a degree of their own, but one with less rigor and no oversight. Let’s see—if they can get 1,000 people to sign up, that’s over $500K. Not a bad business proposition.

I’ve seen various versions of grant writing certifications over the years, along with endless self-help books, training seminars, and the like, but the bad news for those chasing such wills-o-the-wisp is that none will make someone a grant writer. For example, I recently received a notice from The Grantsmanship Center about a local training session, and the note proudly announced that “more than 100,000 people” have attended over the years. If their training was effective, thousands of qualified grant writers should roam the streets, but I don’t often run into them.

Despite the good intentions of some organizations promoting grant writing credentials and training, the only way to become good at grant writing is to write proposals—the more the better and the more varied the better. This is true of all kinds of writing. The challenge is, most people who want to be grant writers are not good writers to begin with or cannot write under deadline pressure. A five-day seminar or sitting for an exam is unlikely to solve this problem and make one a grant writer.

I’m constantly asked how to become a grant writer and I always give the same response: take English composition or journalism classes at a local college to sharpen your writing skills, find a nonprofit in need of help—not too tough to find—and start writing proposals. After a couple of tries, most people will give up, but a few will persevere and become proficient. It also helps if you can apprentice with a grant writer, which is one reason Jake is a good grant writer—he’s been marinating in grant writing for many years and whatever the “it” is, has soaked in.

If we ever decide to offer a grant writing credential, we would structure the exam like this: The supplicant will be locked in a windowless room with a computer, a glass of water, one meal and a complex federal RFP. The person will have four hours to complete the needs assessment. If it passes muster, they will get a bathroom break, more water and food and another four hours for the goals/objectives section and so on. At the end of the week, the person will either be dead or a grant writer, at which point we either make them a Department of Education Program Officer (if they’re dead) or give them a pat on the head and a Grant Writing Credential to impress their mothers (if they’ve passed).

The whole idea of grant writing credentials reminds me of Professor Marvel in The Wizard of Oz, who awards the Scarecrow a “diploma” to compensate for his lack of a brain. As Dorothy understands, you don’t need a diploma to prove you have a brain and you don’t need a two-bit credential to prove you are a grant writer.

EDIT 2015: Despite all the invective above, we are willing to offer grant writing training / seminars, though they won’t be much like most grant writing training and our primary mission is still grant writing—not grant-writing teaching. They also aren’t quite as draconian as the suggestions above. We charge $2,000 per day, or $5,000 per three-day workshop, and Jake teaches the classes.


UPDATE: Isaac responded to some of the commenters in this post. Jake wrote another post about credentials and certifications. If you aren’t altogether sick of the topic, you can also read this.

Posted on Leave a comment

RFP Absurdity and Responding to Narrative Questions

I’ve written about stylistically bad language from government RFPs, but more common than the outright bad is the silly, the coy, the euphemistic, and the ridiculous. Now comes a fine example: section 1.d. on page 30 of the California 21st Century Community Learning Centers (CCLC) – Elementary & Middle Schools narrative:

Explain how all organizations involved in your collaborative have experience or the promise of success in providing educational and related activities that will complement and enhance the academic performance, achievement, and positive youth development of students.

So you need either (1) experience or (2) the “promise of success.” In other words, your level of experience is irrelevant because you can have a lot or none. The RFP* could’ve just asked, “Are the organizations involved able to provide educational services and, if so, how?” RFPs, however, seldom use 13 easy-to-understand words when 36 words designed to obfuscate meaning are available.

The requirement quoted above is particularly egregious because it has only one answer. Is any applicant going to claim that their organizations don’t have the promise of success? Of course not! And what does “the promise of success” mean? To my mind, the answer is “nothing.” Orwell would be aghast at this and many other RFPs—in “Politics and the English Language” he finds examples where “The writer either has a meaning and cannot express it, or he inadvertently says something else, or he is almost indifferent as to whether his words mean anything or not.” I’ve not read a better concise description of RFPs.

Still, you’re writing a proposal and thus your output can and perhaps even should reflect the document that guides your input. Unlike most forms of writing, where brevity is beautiful, (Write Right**: “If I were limited to one rule of style, Omit Unnecessary Words would be the hands down winner”) grant applications encourage bad writing because you (a) need to fill space and (b) need to answer obfuscated questions fully and completely. The best way to do so is by parroting back variations on what the application writer expects, and the best way to avoid irritating a reviewer is by filling your proposal with muck and jargon.

This peculiar kind of poor writing is similar to the peculiar kind of speciousness Isaac discussed in Writing Needs Assessments: How to Make It Seem Like the End of the World. You write a narrative by sending back what you get in the RFP, and when you get garbage in, you usually reflect garbage out. Most RFPs are merely asking you variations on who, what, where, when, why, and how, while most proposals are merely variations on the answers to those questions. Remember that when you’re writing and consider which aspect you should be addressing in the response to each RFP question. The apparently difficult sentence I quoted above from the 21st CCLC can be simplified further to “Who’s going to carry out the program?” There. Nothing to fear. Novice grant writers are often intimidated by the jargon in RFPs, but that’s often just an artifact of bad writing rather than an indication of actual difficulty.

In Studio Executives, Starlets, and Funding, I wrote “Sometimes the funder will want agencies with long track records, sometimes new agencies.” Now I can say that sometimes funders want both, as long as you can somehow justify your experience or the virtue of not having any experience in a proposal. If you come across a narrative demand like the one above, play the RFP’s game. It’s the only way to win.


* Before I get irate e-mails from eagle-eyed readers, I’ll note that the 21st CCLC is a Request For Applications (RFA), but I just call them all RFPs for simplicity’s sake.** If I had to recommend just one book to aspiring writers, regardless of the kind of writing, it would be this one. It’s short, pithy, accurate, and will do more to improve most writers in less time than virtually any other book I know. If I had to recommend two, the second would be William Zinsser’s On Writing Well.

Posted on Leave a comment

Foundations and the Future: How Funder Incentives Affect Nonprofits, Grants, and Grant Writing

“New Voices Of Philanthropy”* is running an occasional series in which they invite bloggers involved in the nonprofit world to contribute; I’m tardy to this month’s question:

Will the Foundation of the Future only fund programs that benefit puppies and children? Will it be run by people that have attained the elusive PhD in Philanthropy? Will the Foundation of the Future actually be the donor advised fund of the future, since foundations are outlawed by Congress in 2016?

The short version: I think foundations in the future will be run much as they are in the present.

The longer version: most foundations seem to be run chiefly for the social prestige and well-being of the people running them. The primary evidence I can discern seems to fall into two categories, the first one stronger and more important than the other: foundations tend only to give away the minimum 5% of their assets every year, as required by law, and they tend to make the process through which nonprofits acquire funding unnecessarily arduous.

The New York Times has reported on a study done by a Barnard economics professor that found “[c]haritable foundations could give away 60 percent more money than they do now without eroding the total value of their assets” (emphasis added). His paper is available here (warning: .pdf link). The upshot is that foundations appear to hoarding money, and, as the study itself says:

[…] Congress intended to keep tax-favored foundations from becoming mere warehouses of wealth. To the extent that the foundation section operates as if though it were a non-endowment system, paying out new giving while allowing existing assets to compound in perpetuity, the foundation sector is in danger of appearing to be exactly what Congress wanted to prevent […] To the extent that individual foundation reduce payout to the legal minimum simply in order to increase their assets under management, they defeat the real social purpose of their privileged tax status[…]

In a similar vein, Akash Deep and Peter Frumkin wrote a paper, “The Foundation Payout Puzzle,” and found that “the average payout rate of this sample of foundations over time, as the policy regime has shifted slightly from a flat 6 percent, to the greater of total investment income or 5 percent, to a flat 5 percent” from 1972 to 1996. This, they later conclude, is bad because a dollar spent today would probably be more effective than a dollar spent tomorrow, assuming that the needs being addressed are important to the recipients.

Since they’re scholars, they give a long and detailed discussion about why foundations don’t increase their payouts. Since I’m a blogger, I’ll be short, mean, and accurate: foundations are, like most organizations, chiefly invested in their own interests and thus would rather propagate themselves into the future; Saul Alinski has argued that the only thing that matters in community organizing is identifying the “self-interest” of the those you are trying to organize, and in this case all that matters is the foundations. If they were purely motivated by the public good—which would seem the primary argument against my argument about foundations—they would presumably raise their payout rates.

Is there any way to counteract this dynamic and thus implicitly change the way foundations operate? It seems improbable. Occasionally a foundation may take a principled stand in a fashion similar to the way Harvard recently used its endowment to cut tuition, but the paper by Deep and Frumkin argues that the situation is getting worse, not better. While foundations are only required to give away five percent of their assets every year, the average American stock market indices have increased by, on average, about 11% per year since World War II. There is apparently some pressure to increase the payout rate, but I don’t think this will actually happen.

I’m skeptical because foundations themselves are unlikely to reduce their power and longevity by increasing payouts en masse, and Congress is equally unlikely to do so because the very rich who donate to congressional campaigns would immediately get every Congressman to whom they (the rich people) ever gave money to on the phone and demand that the payout rule be changed back to 5%. Why? Because the very rich tend to be men, and their wives tend to be the ones sitting on nonprofit boards, running foundations, donating to museums, and what not.

The minute Congress tries to alter this arrangement, the wives of whoever endowed the foundation are going to rise up in arms until the status quo is resolved. Isaac pointed this out to me one time when I read in the newspaper that Congress was threatening to cut the National Endowment for the Arts (NEA) funding: he said it would never happen because of just the situation I described, and he was right. The NEA is particularly unlikely to suffer deep cuts because it represents a very small but highly visible part of the government, and besides, it’s only a small part of discretionary social spending, which is dwarfed by mandatory spending, interest, and defense. This, incidentally, is why Alan Greenspan has been running around and talking about why Medicare—not the war in Iraq, or interest, or any number of other things—is the biggest long-term budget problem facing the U.S.

That was a long enough tangent, and the main point remains that since the same people who tend to fund foundations are also the ones who fund Congressional campaigns, it seems unlikely that Congress will tamper with foundations. So, foundations are unlikely to give more unless they want to. But the question of why do funders give remains.

Maimonides was a 12th century Rabbi who said there are eight levels of giving, with the top loosely being those who help anonymously and without expectation of reward and the bottom being those who give miserly or reluctantly and with the expectation of recognition. As you might have guessed, foundations tend to end up toward the bottom of Maimonides’ chain, meaning that they want to perpetuate themselves, put their names on things, and the like. This makes them highly unlikely to want to raise the payout rate and thus endanger their existence.

Now I’ll more fully discuss the second point: how difficult foundations make it to apply for money, as they seem uninterested in improving the grant making process for those requesting the grants. Questions are too often absurd and forms are poorly thought out (a great example of this will be discussed in a forthcoming post). In the fifteen years Seliger + Associates been in business, the number of times we’ve ever been called by funders asking how the process might be improved is zero.

Never. Not once. Proctor & Gamble, Microsoft, Boeing, and virtually every other large company or organization probably spends millions of dollars trying to figure out how to improve its products and services, but foundations do not appear to, or, if they do, they don’t ask the people who are involved in writing proposals. As a result, they raise the cost of acquiring funding and allow a proportionally lower amount to go to actual services, in a tangent phenomenon to what I discussed here.

Arguably, one could say that foundations make it difficult to receive money so the most interested and hence deserving nonprofits end up with funding. The application becomes a signaling device. There is some merit to the argument, but it also implies that foundations would cause nonprofits that are already successful not bother applying and simultaneously waste the time of foundations that do bother to apply by forcing them to play signaling games.

These perverse incentives coupled with the relative power of foundations compared with grant receivers, the vanity of being perceived as charitable, and the lack of discipline imposed on foundations will probably result in foundations of the future that look mostly like the ones of the present. Perhaps a few of them will buck the trend and spend significantly more than 5% per year, but this seems more likely to be the exception than the norm, especially after the initial funders die. After all, if you were running a foundation, would you be inclined to shut its doors and thus deprive yourself of management fees, free travel to study problems/applicant, or social prestige? Maybe you, the individual, would, but the plural you, who run foundations, wouldn’t.

And I haven’t even discussed how tax advantages work.

I don’t perceive much change in the foundation world, just as Isaac hasn’t seen much change in the overall world of grants in his 35 years of experience. In Charlie Wilson’s War (the movie version), Julia Roberts asks Tom Hanks why Congress says one thing and does another and he drolly replies, “Well, tradition, mostly.” The same could be said of the U.S. nonprofit world, and in 30 years I bet the problems and perils of foundation giving and many other aspects of grant writing will be the same they are today.


* As of 2018, it appears that Seliger + Associates has outlasted “New Voices of Philanthropy,” as the URL that used to be here deadends into a spam site. I guess sometimes the old voices endure longer than the new ones.

Posted on Leave a comment

Phoenix Programs

I noted earlier in Zombie Funding that programs can dwindle from a huge amount of available money to virtually nothing, but they can also rise from the ashes like a Phoenix. Isaac also commented on this phenomenon in Zombie Funding – Six Tana Leaves for Life, Nine for Motion.

Now I’ve seen a more recent example of the monster: last year the School-Based Student Drug-Testing Programs had $1,680,000 available for 12 awards; this year it’s got $12,750,000 for 85 awards, as the link demonstrates. I’m not sure why the program got an extra $11 million, but it leads to another important but perhaps not obvious point: rejection for an application one year doesn’t necessarily mean you shouldn’t apply the next, as changes in the program might make you more likely to be funded. The vast amount of extra money allocated to the School-Based Student Drug-Testing Programs could be a reaction to a large number of highly qualified applicants.

Or it could be random, but the additional money available still makes the School-Based Student Drug-Testing Programs more attractive to anyone who applied or thought about applying previously.

Posted on 1 Comment

The Goal of Writing Objectives is to Achieve Positive Outcomes (Say What?)

Writing the goals and objectives section of a grant proposal is usually a daunting task for the novice grant writer. Compounding the challenge is that almost every government or foundation Request for Proposal (RFP) requires some statement of goals and objectives, and if not required, should be included in most cases. So, here is a short course in how to get over this hurdle.

First, it is critical that one does not confuse goals, objectives, and methods. A goal is an overall statement of intent, such as, “The overarching goal of the LHEAP (Left Handed Enrichment Action Project) initiative is to improve educational outcomes for at-risk left-handed youth in southwest Dubuque.” Note: I don’t mean to keeping picking on Dubuque in my posts, it’s just that as a kid growing up in Minneapolis with a kosher butcher father who always seemed to be ordering meat from a packing plant in Dubuque, it remains an exotic locale in my mind. Sorry for the Proustian reverie (In Search of Lost Time) and back to the subject at hand.

In contrast to goals, objectives are specific, measurable and time-framed products or outcomes of activities proposed for funding through the grant. Objectives are often separated into “process objectives” (sometimes called “formative”) and “outcome objectives” (sometimes called “summative”). Process objectives could include such statements as, “LHEAP will serve a minimum of 100 targeted left-handed youth annually.” Outcome objectives could include such statements as, “A minimum 10% increase in scores on the standardized Iowa Test of Arcane Academic Knowledge will be achieved annually by left-handed students who participate in project activities for a minimum of 10 hours per week over the nine-month school year. Methods are ways of accomplishing objectives, such as conducting individual assessments, providing tutoring, mentoring youth with mentors, offering family literacy to parents/caregivers, etc. Keep methods out of the goals/objectives section and discuss them in the project description section.

The secret to writing effective goal/objective sections is to use the time-honored KISS method, which is to “keep it simple stupid.” At the risk of going Proustian again, I first heard this term in Air Force basic training and it fits perfectly to this aspect of grant writing (for a nice discussion of the KISS method and the virtues of simplicity in general see a post on Ed Sim’s Blog (BeyondVC). By keeping it simple, I mean try hard to state a minimum number of goals (one simply stated goal is ideal), because a separate set of process and outcome objectives is needed for each goal statement. If you have multiple goals, you end up with something like this:

Goal 1

Project Objective 1.1

Process Objective 1.2

Outcome Objective 1.1

Outcome Objective 1.2

Goal 2

Project Objective 2.1

Process Objective 2.2

Outcome Objective 2.1

Outcome Objective 2.2

And so on.

You can see that with four or five goal statements, the objectives will be repetitive and you will likely not only confuse yourself, but also the reader. Having multiple goals also unnecessarily complicates writing evaluation sections (I will soon write a post on how to draft evaluation sections, another novice proposal writer nightmare). So, unless the RFP requires multiple goals, keep it simple and try not to confuse goals, objectives and methods.

Posted on 6 Comments

The Perils of Perfectionism

In The Rest is Noise: Listening to the Twentieth Century, Alex Ross says:

Studio heads were confident that Stravinsky’s name would prove a box office draw; Louis B. Mayer reportedly agreed to give the composer a whooping $100,000, which would be well over a million dollars in today’s money. In a review of the composer’s Hollywood activities, Charles Joseph observes that in almost every case Stravinsky demanded too much time to finish the music and too much control over the finished product.

The same is true of journalism, where deadlines rule the day, and the same is true of grant writing, where perfect is the enemy of good—a necessary truism given the deadline-oriented nature of projects. Neither journalism nor grant writing are flawless arts, and as long as deadlines exist that isn’t going to change. Those who, like Stravinsky, want time to work should find another line of business, because additional time just isn’t going to be forthcoming.

We keep analogizing grant writing to movies because there’s a fair amount of similarity between the “get it done” attitude apparently necessary for movies, which are a kind of art, and grant writing, which is also a kind of art. In grant writing, working quickly is a large part of the art. Even if you do have more time than whatever the deadline imposes, the end result might not be any better. I’ll switch metaphors to Go, a board game in which two players take turns placing black and white stones.

The game scales in difficulty almost linearly and takes five minutes to learn and a lifetime to master—which isn’t where I actually want this metaphor to go, but it’s good to keep in mind nonetheless. The real point: Go is best learned by playing many games quickly, rather than agonizing over particular moves or situations. The game is faster, more fluid, and more fun, and you’ll acquire skill faster than you would otherwise. In the same way, grant writing is best learned by doing: you’re better off writing two proposals of reasonable quality a month rather than one proposal of slightly higher quality. If you continue the two-per month regimen, at the end of twelve months you’ll write two better proposals than the single one you would write if you only wrote one per month.

Later we’ll post more on the subject of how to write proposals under pressure if you’ve never written one before, but in the meantime you should remember that proposals are more like making movies than writing a novel or symphony. Don’t be Stravinsky by implicitly turning down $100,000 because you take too long to prepare: write fast, correct your mistakes, and move on—don’t linger, because you can’t win the race unless you enter. So if you are facing a proposal, the best way to start is with a sentence that attempts to answer whatever first question an RFP asks. Then write another sentence. When you pile enough sentences together, you have a proposal, but if you take too long, it’s not going to matter. Stravinsky was among the Twentieth Century’s most important composers, but he didn’t make much of a difference to Hollywood.

If you’re going to write proposals, you’re going to be in another version Hollywood, and you better meet those deadlines. Keep in mind that any proposal that is turned in late is automatically rejected, no matter how wonderfully crafted.


EDIT: I posted a follow-up article on Perfectionism Revisited.

Posted on Leave a comment

Bad Government English

I realize that I could collect examples of bad English from the Federal Register all day long and that doing so is as challenging as picking a fight with six-year-olds, but this sentence from the Department of Education’s Charter School Program stands out:

The purpose of the CSP is to increase national understanding of the charter school model and to expand the number of high-quality charter schools available to students across the Nation by providing financial assistance for the planning, program design, and initial implementation of charter schools, and to evaluate the effects of charter schools, including their effects on students, student academic achievement, staff, and parents.

Whew! For an extra challenge, diagram the sentence, with all its subordinate and nested clauses. Stanley Fish finds similar problems in an education report and sees bureaucrat speak as the problem:

In this case the bad writing takes two forms. First, there are the sentences made up of empty abstractions linked together in an awkward and strained syntax: “The goal is to magnetize lost talent and ensure that students thrive and progress, in order to create new generations of innovators who will enable New York State to continue as one of the world’s idea capitals.” And there are the sentences that actually say something, but in a prose so clotted and bureaucratic that it takes several readings to figure out what it is […]

A good description of many Requests for Proposals (RFPs)!

Posts such as this might become an occasional series, like the Bulwer-Lytton Fiction Contest, which is “a whimsical literary competition that challenges entrants to compose the opening sentence to the worst of all possible novels.” Sample loser/winner: “Detective Bart Lasiter was in his office studying the light from his one small window falling on his super burrito when the door swung open to reveal a woman whose body said you’ve had your last burrito for a while, whose face said angels did exist, and whose eyes said she could make you dig your own grave and lick the shovel clean.”

(And yes, the title of this post is intentional.)

Posted on 1 Comment

Self-Esteem—What is it good for? Absolutely Nothing

Roberta Stevens commented on “Writing Needs Assessments: How to Make it Seem Like the End of the World” by saying she was “having trouble finding statistics on low self esteem in girls ages 12-19.” This got me thinking about the pointlessness of “self-esteem” as a metric in grant proposals. A simple Google search for ‘“self-esteem” girls studies reports’ yielded a boatload of studies, but if you look closely at them, it is apparent that most are based on “self-reports,” which is another way of saying that researchers asked the little darlings how they feel.

When my youngest son was in middle school, he was subjected to endless navel gazing surveys and routinely reported confidentially that he had carried machine guns to school, smoked crack regularly and started having sex at age seven. In short, he thought it was fun to tweak the authority figures and my guess is that many other young people do too when confronted by earnest researchers asking probing questions.

Although such studies often reveal somewhat dubious alleged gender differences based on self-esteem, I have yet to see any self-esteem data that correlated with meaningful outcomes for young people. Perhaps this is obvious, since self-esteem is such a poor indicator of anything in the real world, given that Stalin appears to have had plenty of self-esteem, even if his moral compass was off target. Arguably our best President, Abraham Lincoln, was by most accounts wracked with self-doubt and low self-esteem, while more recent Presidents, Lyndon Johnson and Richard Nixon, both with questionable presidencies, did not seem short in the self-esteem department.

If I use self-esteem in a needs assessment for a supportive service program for teenage girls, I would find appropriately disturbing statistics (e.g., the pregnancy rate is two times the state rate, the drop out rate among teenage girls has increased by 20%, etc.) and “expert” quotes (“we’ve seen a rise in suicide ideation among our young women clients,” says Carmella, Kumquat, MSW, Mental Health Services Director) to paint a suitably depressing picture and then top it off with the ever popular statement such as, “Given these disappointing indicators, the organization knows anecdotally from its 200 years of experience in delivering youth services, that targeted young women exhibit extremely low self-esteem, which contributes to their challenges in achieving long-term self-sufficiency.” I know this is a nauseating sentence, but it is fairly typical of most grant proposals and is why proposals should never be read just after eating lunch.

So, to paraphrase Edwin Star, “Self-esteem, what is it good for? / Absolutely nothing.”

(In the context of gangs, Jake has also commented on suspect or twisted needs indicators .)


EDIT: A more recent post, Self-Efficacy—Oops, There Goes Another Rubber Tree Plant, takes up the issue of finding a metric more valuable than self-esteem for both grant writers and program participants.

Posted on 1 Comment

More on Charities

A previous post linked to a Wall Street Journal post on charities; now the paper released a full article (may not be accessible to non-subscribers) on the subject of how donors evaluate the usefulness of a program, arguing that donors are becoming more engaged in measurement. One thing missing: statistics showing this is actually part of a trend, rather than just a collection of anecdotes. The article is more descriptive of the practices around how to evaluate effectiveness and uses hedge words:

Wealthy people and foundations sometimes hire philanthropy consultants to help them gauge a charity’s effectiveness. But other donors who seek that kind of analysis usually have had to rely on guesswork or do it themselves, which makes it tough to figure out whether one approach to solving a problem is better than another.

“Sometimes” they hire consultants, other times they essentially use the hope and pray method. That’s not terribly different from how things have always been done. Most interesting, however, is a topic relevant to evaluations that we’ll comment on more later:

The problem is, it can be difficult — and expensive — to measure whether charitable programs are actually working, and most nonprofits aren’t willing to devote scarce resources to collecting such information.

Most federal programs have in effect chosen a tradeoff: they provide more money and almost no real auditing. This is because real auditing is expensive and generally not worthwhile unless a blogger or journalist takes a picture of an organization’s Executive Director in a shiny new Ferrari. To really figure out what an organization is doing with $500,000 or $1,000,000 would cost so much in compliance that it would come to represent an appreciable portion of the grant: thus, the hope and pray method becomes the de facto standard (more on that below).

The writers also are pressed for space or don’t fully grok nonprofit evaluations, because they write:

Philanthropy advisers suggest first asking nonprofits about their goals and strategies, and which indicators they use to monitor their own impact. Givers should see how the charity measures its results both in the short term — monthly or quarterly — and over a period of years.

Measuring results isn’t a bad idea if it can be done, but the reason such measurements often don’t occur is precisely because they’re hard. Even if they do occur, you’re asking the organization to set its own goal marker—which makes them easy to set at very, ahem, modest, levels. If you set them at higher levels, the measurement problems kick in.

If you’re going to decide whether an after school program for middle-schoolers is effective, you’ll have to get a cohort together, randomly divide them into those who receive services and those who don’t, and then follow them through much of their lives—in other words, you have to direct a longitudinal study, which is expensive and difficult. That way, you’ll know if the group who received services were more likely to graduate from high school, attend college, get jobs, and the like. But even if you divide the group in two, you can still have poisoned data because if you rely on those who present for services, you’re often getting the cream of the high-risk/low-resource crop. You have numerous other confounding factors like geography and culture and the like.

The research can be far more costly than the project, and as little as donors like not knowing whether their money is effective, they’re going to like it even less if you spend 50 — 80% of the project on evaluating it. This is why the situation donors say they want to change is likely to persist regardless of what is reported.


EDIT: We wrote another, longer post on evaluations here.