Category: Advice

Posted on by — 1 Comment

Self-Esteem—What is it good for? Absolutely Nothing

Roberta Stevens commented on “Writing Needs Assessments: How to Make it Seem Like the End of the World” by saying she was “having trouble finding statistics on low self esteem in girls ages 12-19.” This got me thinking about the pointlessness of “self-esteem” as a metric in grant proposals. A simple Google search for ‘“self-esteem” girls studies reports’ yielded a boatload of studies, but if you look closely at them, it is apparent that most are based on “self-reports,” which is another way of saying that researchers asked the little darlings how they feel.

When my youngest son was in middle school, he was subjected to endless navel gazing surveys and routinely reported confidentially that he had carried machine guns to school, smoked crack regularly and started having sex at age seven. In short, he thought it was fun to tweak the authority figures and my guess is that many other young people do too when confronted by earnest researchers asking probing questions.

Although such studies often reveal somewhat dubious alleged gender differences based on self-esteem, I have yet to see any self-esteem data that correlated with meaningful outcomes for young people. Perhaps this is obvious, since self-esteem is such a poor indicator of anything in the real world, given that Stalin appears to have had plenty of self-esteem, even if his moral compass was off target. Arguably our best President, Abraham Lincoln, was by most accounts wracked with self-doubt and low self-esteem, while more recent Presidents, Lyndon Johnson and Richard Nixon, both with questionable presidencies, did not seem short in the self-esteem department.

If I use self-esteem in a needs assessment for a supportive service program for teenage girls, I would find appropriately disturbing statistics (e.g., the pregnancy rate is two times the state rate, the drop out rate among teenage girls has increased by 20%, etc.) and “expert” quotes (“we’ve seen a rise in suicide ideation among our young women clients,” says Carmella, Kumquat, MSW, Mental Health Services Director) to paint a suitably depressing picture and then top it off with the ever popular statement such as, “Given these disappointing indicators, the organization knows anecdotally from its 200 years of experience in delivering youth services, that targeted young women exhibit extremely low self-esteem, which contributes to their challenges in achieving long-term self-sufficiency.” I know this is a nauseating sentence, but it is fairly typical of most grant proposals and is why proposals should never be read just after eating lunch.

So, to paraphrase Edwin Star, “Self-esteem, what is it good for? / Absolutely nothing.”

(In the context of gangs, Jake has also commented on suspect or twisted needs indicators .)

EDIT: A more recent post, Self-Efficacy—Oops, There Goes Another Rubber Tree Plant, takes up the issue of finding a metric more valuable than self-esteem for both grant writers and program participants.

Posted on by — 1 Comment

More on Charities

A previous post linked to a Wall Street Journal post on charities; now the paper released a full article (may not be accessible to non-subscribers) on the subject of how donors evaluate the usefulness of a program, arguing that donors are becoming more engaged in measurement. One thing missing: statistics showing this is actually part of a trend, rather than just a collection of anecdotes. The article is more descriptive of the practices around how to evaluate effectiveness and uses hedge words:

Wealthy people and foundations sometimes hire philanthropy consultants to help them gauge a charity’s effectiveness. But other donors who seek that kind of analysis usually have had to rely on guesswork or do it themselves, which makes it tough to figure out whether one approach to solving a problem is better than another.

“Sometimes” they hire consultants, other times they essentially use the hope and pray method. That’s not terribly different from how things have always been done. Most interesting, however, is a topic relevant to evaluations that we’ll comment on more later:

The problem is, it can be difficult — and expensive — to measure whether charitable programs are actually working, and most nonprofits aren’t willing to devote scarce resources to collecting such information.

Most federal programs have in effect chosen a tradeoff: they provide more money and almost no real auditing. This is because real auditing is expensive and generally not worthwhile unless a blogger or journalist takes a picture of an organization’s Executive Director in a shiny new Ferrari. To really figure out what an organization is doing with $500,000 or $1,000,000 would cost so much in compliance that it would come to represent an appreciable portion of the grant: thus, the hope and pray method becomes the de facto standard (more on that below).

The writers also are pressed for space or don’t fully grok nonprofit evaluations, because they write:

Philanthropy advisers suggest first asking nonprofits about their goals and strategies, and which indicators they use to monitor their own impact. Givers should see how the charity measures its results both in the short term — monthly or quarterly — and over a period of years.

Measuring results isn’t a bad idea if it can be done, but the reason such measurements often don’t occur is precisely because they’re hard. Even if they do occur, you’re asking the organization to set its own goal marker—which makes them easy to set at very, ahem, modest, levels. If you set them at higher levels, the measurement problems kick in.

If you’re going to decide whether an after school program for middle-schoolers is effective, you’ll have to get a cohort together, randomly divide them into those who receive services and those who don’t, and then follow them through much of their lives—in other words, you have to direct a longitudinal study, which is expensive and difficult. That way, you’ll know if the group who received services were more likely to graduate from high school, attend college, get jobs, and the like. But even if you divide the group in two, you can still have poisoned data because if you rely on those who present for services, you’re often getting the cream of the high-risk/low-resource crop. You have numerous other confounding factors like geography and culture and the like.

The research can be far more costly than the project, and as little as donors like not knowing whether their money is effective, they’re going to like it even less if you spend 50 — 80% of the project on evaluating it. This is why the situation donors say they want to change is likely to persist regardless of what is reported.

EDIT: We wrote another, longer post on evaluations here.

Posted on by — Leave a comment

Gangs, Again

Hot on the trail of yesterday’s post about L.A. gangs and statistics, the New York Times published “Los Angeles Combating Gangs Gone International.” It begins:

Two gangs that originated on the streets here have grown so large in El Salvador that there are two prisons in that country devoted exclusively to their members, one for each gang, according to officials who traveled there recently to meet with the local authorities.

That is just one measure of the way gangs in this city with the worst gang problem in the United States have bolstered their presence in Mexico and Central America, where they attract new members eager to come here.

Over the course of the article, you won’t notice any numbers regarding how many active gang members operate in L.A., and not until the fourth paragraph do you actually learn that its premise rests on 23 people being convicted of extra immigration crimes. In addition, the only two people quoted are, for lack of a better term, anti-gang professionals; evidently the reporter knows what Isaac said about narratives:

If you’re having difficulty building your argument with data, a good technique is to call local “experts” for quotes. For example, find and call the police unit responsible for gang suppression in your target area, then ask leading questions. Invariably, the officer will tell horror stories about rampant gang activity. Just ask if you can quote her and she will almost always agree. It’s always fun to include the names of some local gangs in your proposal for a dash of reader titillation.

I’ve written about questionable New York Times articles on my personal blog, and there is something vaguely rotten about “Los Angeles Combating Gangs Gone International.”

You also don’t get the names of the gangs, but one is almost certainly “Mara Salva Trucha” or MS-13 (sometimes spell “Salvatrucha”), which is famous enough or has good enough PR to merit a Wikipedia page, although you shouldn’t necessarily trust what you read there. Still, given all the publicity MS-13 has received, you can always cite Bloods, Crips, and MS-13 affiliates as making up your local gang problem.

Posted on by — Leave a comment

On Gangs and Proposals

As Isaac wrote, it almost never hurts to claim gang activity in a proposed service area (“[. . . f]ind and call the police unit responsible for gang suppression in your target area, then ask leading questions. Invariably, the officer will tell horror stories about rampant gang activity.”). Now, by way of Freakonomics, I found an L.A. Weekly article on the subject:

Eventually, James started talking. He told me he’d started gangbanging when he was 12. “I got shot when I was 15, and that’s when it got bad,” he said softly. “I got extreme after I got shot.” James started teaching youngsters from Nickerson how to gangbang. Using rival gangbangers for practice, he taught his students how to hunt and kill. “You teach a person how not to take losses, how to be gladiators, run them down, gun them down,” he explained.

James wasn’t remorseful, but he was far from proud. In truth, he seemed numb; his life of crime and death hung about him in a static haze. There is a personal demilitarized zone in the advanced lives of former hardcore gang members, should they survive their 20s, where they live as neither soldier nor citizen. James said he struggles to keep a gun out of his own hands every day, but that in January he was tempted to join the battle with the Grape Street Crips after a young Bounty Hunter he knew was killed.

Steven Levitt of the Freakonomics Blog says he can’t agree with the article’s conclusion about increasing violence:

Landesman argues that the gang problem is worse than it has ever been, and that gang violence hasn’t dropped the way other crimes have. A quick glance at the homicide rates among young black males over the last 15 years shows that this statement just can’t be accurate. The biggest declines in homicide have been among young black males, both in absolute and percentage terms.

What Levitt doesn’t address is the possibility that gangs are getting worse in Los Angeles, which is the area covered by the article, even if the situation is improving nationwide. Judging from articles like “The Story of a Snitch” in The Atlantic, which covers Baltimore, I wouldn’t be surprised if gang problems are changing rather than abating. Or maybe programs like G.R.E.A.T. are having some effect, as The National Institute of Justice (NIJ) argues. This seems improbable but possible, and I don’t know of anyone who has examined the methodology of the NIJ’s study.

The preceding paragraph again goes back to Writing Needs Assessments: How to Make It Seem Like the End of the World by demonstrating how statistics can be attacked and why you shouldn’t necessarily trust conclusions found in the popular press or elsewhere. They can be changed and challenged in an actual proposal. If you’re writing one, you usually have an incentive to make the area or situation appear as bad as possible so you can remedy problems through your project.

Consequently, when writing about a target area, you’re better off claiming gang activity, since most reviewers aren’t going to be aware of gang trends nationwide, and even if they are, you can announce that gang activity in your area is rising, as the L.A. Weekly reviewer did. You could also cite Levitt, as I do, and then attack his reasoning.

As with most things about grant writing, there are some local aspects Levitt doesn’t know about, and in this case it’s something everyone in South Central does: Nickerson Gardens, a public housing development, is among the worst areas around. Here’s one example of its history: “The Nickerson Gardens is considered by many as the most violent, drug infested, crime riddled neighborhood in the country.” It’s a very scary place and, ironically, there’s a street running through called “Success,” which reminds me of Soviet propaganda about worker achievement and happiness.

Success Ave

(Click to here to see the full image.)

When I talked to Isaac about this article, he immediately said that he bet that the Housing Authority of the City of Los Angeles (HACLA), which owns Nickerson, hasn’t tried to use the HUD HOPE VI program to revitalize Nickerson. He was right, and HOPE VI also hasn’t been attempted at the other three public housing projects in Watts: Jordan Downs, Imperial Courts, and Avalon Gardens. Nickerson was also right in the middle of the civil disturbances in Watts in 1965 and the Rodney King incident in 1992, and 15 years later it remains a place of hopelessness and gang activity. Despite the street name, there is little success in Nickerson.

This is the kind of revisionist information you could include in a needs assessment about the area. The stories embedded would add flavor and help counteract what Levitt writes, so even if the reviewer happens to know a lot about gang activity, which is improbable, he or she would still award you points for need.

EDIT: This post covers similar territory.

Posted on by — Leave a comment

Writing Needs Assessments: How to Make It Seem Like the End of the World

Almost every grant proposal requires some form of needs assessment. More or less, the sentiment one must get across it that “It’s the end of the world as we know it and I feel fine,” as REM says. Essentially, the object is to make problems look overwhelming, but solvable with just a dollop of grant funds. So, how does a grant writer do this?

Start by making the end appear nigh, which requires a needs assessment. Look at the Census data available at American Fact Finder, which has a variety of geographic choices (e.g. county, city, zip code, census tract, etc.). It is almost always best to match the project target area with a census data geographic area to make assembling data easier, regardless of whether the census area perfectly matches the area you want to serve. Try not to make the target area, “the Westside of Dubuque,” unless that happens to conform with four census tracts. Most geographic areas have 2000 Census data, as well as estimates for 2005. Pick the date that is to your advantage, and being to your advantage means making the situation look worse. For example, if incomes have been trending downward and unemployment upward due to plant closings, the 2005 data may be better. Announce that, if current trends continue, Dubuque may be abandoned completely in 2010 because there are too few jobs, but the situation can be improved with the requested grant.

Once you have your target area, find useful socioeconomic indicators like ethnic breakdown, median family income, age cohort percentages, percent of people living below poverty, percent with disabilities, etc. Only include data that supports your case. A winning grant proposal is not like a thesis, so you are under no obligation to use all available data. Also, it is critical that you provide some data on a larger area for comparison purposes, so your readers understand the relative problems. This can be the city, the county, state or even national data—pick whichever makes your situation look worst, meaning with the greatest discrepancy between the target area and the larger sample. It doesn’t really matter which geographic level you compare to, as long as you can say something to effect of, “The target area median family income is just 2/3 that of Los Angeles County.” Depending on the target population, it may be advantageous to compare data for a particular ethnic group to all residents. For example, if the target area includes a significant African American population with lower incomes, you can set up tables showing African American indicators versus white indicators for the same geographic area, in essence comparing the target area to itself. American Fact Finder has a handy tool on the left button bar for “Fact Sheet for a Race, Ethnic or Ancestry Group” that makes this easy to do.

FactFinder

(Click here to see the full image.)

You can also use census data to obfuscate the actual reality in the target area. For example, in many Southern California cities there are high percentages of Asian Americans, who in some communities have higher-than-average incomes. This can be used for statements such as, “over two in five residents is a person of color.” For better or worse, most grant reviewers will usually associate persons of color with lower incomes and higher risk factors whether this is true or not. Grant reviewers seldom have a deep background in statistics and they probably don’t even know statistics for journalists let alone real statistics. Even if they do, everything starts to become a haze after reading a dozen federal proposals that can be onerously long, so most reviewers are apt to begin looking more for conclusions than data not long into the process. Do you somehow fulfill the checkbox that asks whether educational attainment is lower in the target area than the nation? If so, give ’em five points and move on.

Other good sources of data include state and local departments of education. Some states and school districts have better data engines than others. For example, the California Department of Education has a great site, DataQuest, but other states’s data system are, as Borat would say, “not so much”. If a good data engine/warehouse is not available, find the school/district reports cards mandated by the federal “No Child Left Behind” legislation. Many districts try to hide these reports, as they are often unflattering after you get past the mission/vision statement platitudes, but if you dig hard enough you will find them. If necessary, call the statistics unit at the district or state and force the reports out of them.

Once you have data, only use what helps the argument. So, if test scores for certain grades are low relative to the county or state, use those, not all test scores. If you want to use dropout data, use the four-year derived rate, not the single year, which will be much lower. In some states, such as Illinois, drop out data is wildly understated, due to the way the state treats students who are no longer in school, so if you have to use it, underscore this fact. Health data, including disease incidence, mortality, etc., can usually be found at state and local health department web sites, while crime and gang data are typically found at police department web sites.

If you’re having difficulty building your argument with data, a good technique is to call local “experts” for quotes. For example, find and call the police unit responsible for gang suppression in your target area, then ask leading questions. Invariably, the officer will tell horror stories about rampant gang activity. Just ask if you can quote her and she will almost always agree. It’s always fun to include the names of some local gangs in your proposal for a dash of reader titillation. This is particularly important if the reader is on proposal 35 out of 40 and just wants to go find the hotel bar. You can also find the name of any large social service provider or city official in the target area (other than the one for whom you are working) and ask them about local problems with the target population. For example, if you seek information about at-risk youth services and you talk to the local Boys and Girls Club executive director or city parks director, this person will almost always say that new problems are erupting every minute while their funding is declining.

This gives you the opportunity to write something like, according to Conrad Cuttlebone, YMCA Director, “there are many more latch-key kids in the community since the Hindenburg Dirigible Factory closed, and we’re seeing many more cases of domestic violence, while at the same time the county cut our funding by 50%.” When all else fails, you can simply write, “although specific target area level data is not available, the agency knows anecdotally that teen pregnancy is on the rise, mirroring national trends.” Of course, you can do this even if the local area doesn’t match national trends, as most reviewers don’t have the vaguest idea about national trends for anything.

In other words, while it is not a good idea to make up data, it’s perfectly fair to exaggerate problems through obfuscation and specious analysis. You’re generally rewarded for such effects: the worse the target area, the more likely you are to get points, and the more likely you are to be funded.

The gentle art of writing needs assessments really comes down to painting word pictures that combine cherry-picked data with opinions and anecdotes strung together to meet the expectations of reviewers, who assume something terrible must be going on in your community, or you would be doing almost anything other than writing a grant proposal, such as watching my favorite college football team, the KU Jayhawks, trounce Virginia Tech in the upcoming Orange Bowl. Rock! Chalk! Jayhawk! KU!

Posted on by — 2 Comments

Studio Executives, Starlets, and Funding: Part II

In Part I of “Studio Executives, Starlets, and Funding,” I began by responding to a commenter who said:

<blockquote>I cannot shake the observation that to get a grant you must tell people with the money what they want to hear […] But there seems to be no objective criteria by which these grants are awarded […]</blockquote>

Today I’ll mostly deal with the second part.

You’ll sometimes find objective criteria and sometimes not. The former usually corresponds to government grants and the latter more often to foundation/corporate giving sources. The thing is, you can’t generalize the myriad of funders in the grant universe. Most federal grants will have points and criteria, and the more diligently you work to fulfill those criteria the better off you’ll be. I just looked at our email grant newsletter and found the “Smaller Learning Communities Program.”

Follow that link to “V. Application Review Information,” where you’ll find a list of criteria with point values. The agencies with the most points get funding. With most Federal, state, and local government grants, some number of proposals will be funded; it’s not like the Pulitzer Prize in Fiction, which doesn’t give awards if the judges consider no novel or story collection worthy.

If the funder says they’re offering $500,000 for after school supportive services, and you want to provide prison rehabilitation and drug abuse reentry services to adult offenders, you’ve got the wrong funding source. Sometimes there’s a middle ground—if you want to provide bilingual education, you might apply to my hypothetical program by saying you’re going to deliver after school supportive services in two languages to at-risk youth. Now you’ve told the funding source what they want to hear and you’ve done what you want, so barring other considerations you’re at least in the running to be funded.

Contrast that with most foundation sources: they often give broad guidelines, you submit proposals, wait, and hope for a phone call or letter. You seldom get point lists, but you’ll often get guidelines. If they say they only fund in Cupertino, San Francisco, and Northern California, and you’re in San Diego, you’ve at the very least discovered that the funder is unlikely to be interested in you. If you want to raise your probability of being funded, look for funders who are interested in San Diego.

But it’s their money, it’s a free country, and as long as a foundation distributes 5% of its assets to something vaguely charitable each year they can do what they want, including violating their own guidelines. More often than not, the best you’ll be able to do is submit a complete and technically accurate proposal written to the best of your ability with whatever attachments and paperwork they ask for. If they want a two-page letter, don’t send them an eight-page proposal, and the same if the reverse is true. I keep repeating the phrase “complete and technically accurate proposal” for a reason: people who do it are more likely to get funded than not, and you can rage against the opacity of funding sources if you want to, but probably won’t get you funded. Note how “probably” creeps into my post.

Do whatever you can to increase the probability of your application being reviewed favorably. Chances are, if you follow the snarky golden rule (“He who has the gold makes the rules”), you’re more likely to be funded. After all, he who has the gold makes the rules. But even if you follow the rules, you might not be funded. That’s just how it goes, and if you accept that and keep applying and working at what you’re doing, you’ll eventually succeed.

But nobody knows anything, including me. Good luck.

(COMING ATTRACTION: Make sure you apply to programs you actually want to run! See it in theaters and on this website soon!)

Posted on by — 1 Comment

Studio executives, starlets, and funding

In William Goldman’s hilarious Adventures in the Screen Trade*, he wrote, “Nobody knows anything.” Nobody knows how much money a movie will make or which movies should be made or what audiences want. Goldman cites movies a studio thought were a sure thing and flopped, and movies every studio but one rejected, only to see it do such astronomical business that every executive in the studio could buy a Beverly Hills pad and live like Ari Gold in Entourage.

Any smart studio head will they say, “I did it once with Star Wars, so why can’t I do it 50 more times with Space Battles and Sun Fight and so on until I exhaust my thesaurus?” This is grossly unrealistic because it assumes studio executives own a thesaurus, but the point remains that if they could reliably pick billion-dollar movies, they wouldn’t have to spend so much time agonizing over scripts from those ungrateful striking writers, putting projects together, trying to get Russell Crowe and his $20 million and 20% deal on board, and doing the 750,000 other things required for a movie to have what they consider a shot at the big time. In other words, they want to find an algorithm for finding a sure thing, so instead of all of that studio executives could skip straight to the mansion-and-floating-in-the-pool-drinking-mai-tais part of their lives, which is why studio executives became who they are. If they’d gotten in for another reason, they’d probably be picketing the Universal lot.

As a result of this effort at picking winners, studio executives have spent enormous amounts of time and money on screen tests, re-cuts, edits, and probably much I don’t know about, all in an effort to discover whether if their movies will succeed. Despite all this effort, most movies still bomb.

Nobody knows anything. If they did, movies wouldn’t bomb so often.

Many of you might be thinking, “I’m at a site about grant writing, right? And if so, why am I reading about Hollywood?”

It actually has a lot to do with grant writing. A commenter to our fourth post writes: “I cannot shake the observation that to get a grant you must tell people with the money what they want to hear […] But there seems to be no objective criteria by which these grants are awarded […]” Telling funders what they want to hear is a fine observation because you should follow whatever guidelines they provide.

The golden rule cliche says, “Do unto others as you would have them do unto you.” The almost-as-old, snarky version goes, “He who has the gold makes the rules.” If you want to make the rules about who gets funded, you have to lead a federal agency or start a software company, make more money than some countries’ GDP, and endow a foundation. Assuming that Bill Gates isn’t reading this, you’ll have to follow the rules of whoever has the gold, and if you don’t want to follow them, you’re less likely to get funded. You should follow the funder’s instructions if you want to be funded, just as studio executives know that if they want their movies have a better chance of success they should hire Kate Winslet.

You can maximize your probability of being funded by submitting technically correct proposals written and prepared to the best of your ability. This will improve your odds of being funded, which in part includes making sure that you follow whatever submission guidelines are available.

To be sure, no one can be certain whether a given proposal will be funded. Almost anyone who makes guarantees about funding is probably doing something unethical, exaggerating their connections, or simply lying. It’s also difficult to gauge the likelihood of a particular proposal being funded because you probably won’t know many other factors: the other applicants to a program, the mood of the reviewers, who the reviewers are, how many other applicants there are, whether the funder has hidden priorities, the phase of the moon, etc. Like studio executives picking movies, you won’t always know what the audience wants, despite what the audience may say.

Sometimes the funder will want agencies with long track records, sometimes new agencies. Sometimes funders will have geographic preferences—if you’re working for an organization in the Northeast, for example, and the Northeast has brutal competitors while practically no one from the Southwest bothers to apply, and if the funder wants to fund programs evenly in the U.S., then it’s possible that an otherwise good agency with an otherwise good program won’t be funded in the Northeast. Notice the preceding sentence’s length and how many caveats it contains, and that’s only for a single hypothetical factor. Multiply that by fifty or a hundred or infinite factors, right up to whether a reviewer knows where the proposed evaluator went to college and doesn’t think much of that college, which happens to be his college’s rival, and you’ll start to realize why no one can guarantee funding no matter how great an applicant and application might be.

Still, as an organization applies, it should pay some attention to whether its proposals are consistently rejected. As the Seliger + Associates FAQ states:

Over time, you should achieve a 25% – 50% success rate. If less than 25% of your proposals are being funded, you’re probably doing something wrong (e.g., incomplete application packages, ineligible applicant, etc.). If more that 50% of your grants are being funded, you probably are not stretching the envelop far enough by trying to get grants to extend your agency’s service capabilities.

So, to go back to the commenter, who writes about how there seems to be no objective criteria by which these grants are awarded, I’ll give a short answer now: it depends on what you’re applying for. This post, however, has been extended long enough, so Part II will deal with that and come next Monday. Any successful post ought to have a sequel, action figures, and lunch boxes.

Part II is now live.

*An essential book for anyone who wants to better understand the movies they watch, and, implicitly, why so many are abysmal.