Comparing score tables for two SBIR Phase I grants provides a grantsmanship lesson not to forget: The advice in your Summary Statement isn’t necessarily everything you need to know to revise your grant and win
You probably know every NIH SBIR grant application is reviewed by three people who use a 9-point scale to grade Significance, Investigator(s), Innovation, Approach and Environment, where 1 is the best score and 9 the worst. These scores are contained in the Summary Statement NIH sends containing their critiques of your grant.
Based on those five scores, each reviewer also gives the grant a sixth score (also 1 through 9) representing its overall technical and scientific merit. This is the Overall Impact score. I’ll write about Overall Impact scores another time.
The SBIR scoring system at work
To give you a feel for the scoring system, below are scores received by two SBIR Phase I grant applications. Neither was funded, but the one on the left came very close and probably will win on its next try. It was written by a client of mine with long experience in SBIR grant writing. The other was written by a first-timer (not a client) who isn’t a scientist and didn’t understand what successful SBIR grant applications look like.
Comments on the near miss
The biggest lesson from the high scores on the left is how tough competition for NIH SBIR grants has become. Years ago when there were so many fewer competitors, this Phase I grant probably would have won without needing to be revised and resubmitted.
In the table I marked the 3s for Innovation as my main concern because innovation is an especially subjective thing to judge, so much in the eye of the beholder. So we’re rewriting the innovation section to dispel any doubt the Innovation scores should be higher.
Nor are we stopping there. We’re following the advice for improvement in the Summary Statement and looking beyond the statement for every improvement we can make so the resubmitted application scores above payline.
Comments on the crash
I got the idea for this article when I realized the person who sent me these scores couldn’t interpret them. Encouraged by the wonderful scores for Significance, they didn’t understand most of the rest of their scores were disastrous. They were being told this grant stinks but didn’t know it. To resubmit and win, this grant has to be rethought top to bottom.
Why the quality of advice in your Summary Statement may depend on your scores
Moreover, I explained to these folks in deep trouble that I thought it unlikely their Summary Statement would tell them everything they needed to fix.
To understand why this is so, suppose you are a reviewer for one of the big institutes at NIH like NCI or NIAID, where several thousand SBIR applications get reviewed each year. Let’s say you’re responsible for reviewing 10 SBIR Phase I grants for the next grant review meeting, and you’ve just started reading the deep doo-doo grant. Within 10 minutes you know it hasn’t the slightest prayer of being funded.
Now, with all the other grants you have to review, not to mention demands on your time for everything else you’ve got going on in your life, are you really going to spend hours explaining to the authors of this piece of garbage every single thing they need to fix?
Probably not, I think. More likely you’ll offer some broad and general advice for their Summary Statement and leave it at that. Why explain more when it’s doubtful they could fix the grant no matter how good your instructions.
Now suppose the next grant you review is the near miss, the one above on the left. Just in reading Specific Aims, you sense this is a quality piece of work. Then as you read Significance, then Innovation, then Approach, your conviction grows that they’ve thought of just about everything. By the time you finish reading you know they have an excellent chance to make this work. You only see a few minor problems, which shouldn’t take much time to explain.
For the near miss, then, isn’t it more likely the critique you’ll write for the Summary Statement will explain exactly what you think they need to fix?
This is why the best grants are likely to get the most helpful Summary Statements.
Summary Statements don’t have to be comprehensive
Regardless of how good your scores are, never forget that reviewers have no obligation to tell you everything they think you should fix in order to win. All they have to do to satisfy NIH is tell you enough to justify their opinion. It is always possible your grant has an important weakness they didn’t mention — which is up to you to find on your own and fix it if you are to win the next time around.