The NSF reviewing process
The typical NSF panel I've seen is composed of about 1/4-1/3
people who have worked in the area at one point in time, but may not
be active/publishing, about 1/4-1/3 people who are in related fields,
and the rest actively working in at least the broad area. You probably
have about a 30% chance of having someone who really knows your
specific area read your proposal. You'll probably get one person who
at least keeps up on the research in your area, even if they don't
work in it themselves, and one person who knows very little about your
Each proposal is (usually) read by 3 people. Sometimes it will be
read by more for one or more reasons:
- Someone on the panel decided to read it, just because it looked
interesting from the title/abstract, even though it wasn't assigned to
- Someone on the panel (on the day of the meeting) volunteers to
read it. This can happen because of a conflict of opinion amongst the
assigned reviewers, or because, while listening to the discussion,
they decided they might have an opinion.
- If your proposal crosses disciplines, it will typically have an extra reviewer or two from another area.
Each reviewer is suppose to read the proposals before the meeting,
and enter comments. A typical review load is 5-15 proposals, and,
while some reviewers try to read them all the night before (!) usually
they'll be read over a month or so.
The reviewers don't see each other's comments (typically) until
the day of the meeting.
The day of the meeting, each proposal is discussed in turn, and
the proposals categorised as Really Good (will be funded), Good (we
like them, but won't be funded unless money falls out of the sky), and
Reject. (This varies a bit - usually as the proposals are discussed
they're roughly ordered. ) If your proposal ended up in the Really
Good category (and typically only about 1/10 do) then you will be
funded, barring any unusual circumstances. These proposals come back
with straight Excellences and the occasional Very Good. Of the
proposals in the Good category, the top one or two might be funded.
The program chair listens in on the discussion, but basically
doesn't contribute. He/she has the final say on what gets funded. I
think they generally follow the advice of the panel, but as I've never
checked to see if the proposals we ranked highly were funded, I can't
tell you if this is indeed the case. I have heard of at least one
occasion where the program chair killed an excellently ranked
proposal because "it wasn't in an area I was interested in" (he was
retiring that year...). But (hopefully!) this sort of thing is rare.
Note: Some small proposals (especially from under-represented
schools/states) may get funded even though they weren't in the top
1/10 (but they better be ranked well). This is because there's other
pots of money to draw from in this case. And (I think) a small
proposal might just slip in to fill in the chinks in the budget.
How this affects your proposal review.
- The reviewer will remember, at best, your research area, a basic
idea of your proposed research plan, and any unusual collaborative
efforts. But that's only if you make a concerted effort to clearly lay
out those points in the first two pages.
- They will definately remember if:
- The writing was bad/confusing/full of errors.
- You didn't do what you said you were going to do in the first two pages.
- You missed out someone else's research you should have been aware of.
- You should do your best to make sure they remember:
- That you did an excellent job describing the current state of research in your area, and what the open problems are.
- That you clearly laid out a list of problems and potential solutions.
- Moreover, you included a summary of those problems in an easy-to-find place so the reviewer can find it again when needed.
- You thought about the broader impact and education components.
- (If applicable) That you have thought out how to evaluate your research, and/or apply it to real problems.
How the composition of the panel affects your ranking
If someone on the panel works in your area, this can be either a
Good thing, or a Bad thing. But it will definitely affect your
ranking, because the remaining panelists will defer to the person who
"knows what they're talking about".
On the plus side: If you really do have a good idea, but maybe you
haven't presented it as clearly as you might, or argued why this is an
important research topic, a knowledgeable reviewer can help.
On the down side: If the reviewer doesn't think much of your idea,
they can kill it. The reviewer's arguments will be one of the
following: "I/someone tried this x years ago and it failed", or they
may present technical reasons. To combat this, I suggest the following:
- If you've got demonstrable results, include them and, more
importantly, make sure they're easy to find/stand out from the body of
- Make sure your "why this research is important" and "why this
approach will succeed" sections are clearly understandable to someone
who is (presumably) reasonably intelligent, but not versed in your
- Include enough detail to convince someone in your area that you
know this material cold, and that your solutions will work. The real
reason for this section is so that the semi-knowledgeable reviewer (who
likes your proposal) can point to that section and say "this looks
really convincing - explain to me why this isn't convincing you".
People outside of your area will primarily judge your proposal on
the first two pages, the outreach section, and on how well the
proposal is written (does it flow, are the non-technical arguments
convincing). I have seen a proposal that was very well-written, but
somewhat slim on technical merit, get funded because the three
reviewers were not knowledgeable in the area. And, in general, a
well-written proposal will get higher marks from less-knowledgeable
people simply because that's the only thing they have to judge the
Of course, this means that if your proposal is dense and lacks
compelling, easily understood arguments, it will get lower marks. If
you're working in an area where the problems are not so obvious (e.g.,
if you work with robots, you know it is stunningly difficult to get a
robot to roll down a corridor, but the average computer science
researcher doesn't think this is a hard problem because they walk down
corridors all the time, so how hard can it be...) then you're going to
have to spend some time convincing the reviewer that this is actually
a problem, and the current solutions are not good enough.
Why proposals aren't funded
The following are the most common rejections I've seen. They come
in pairs, more or less, with a good proposal balancing between the two.
- It's already been done.
- This usually happens with a proposal where the PI is new to the
field, and just plain missed an entire body of research (possibly
because it came under a different name in another field). Usually the
review will be a kindly one, and simply say "go look at this work".
- It also happens (sadly enough) to new proposal writers because
they didn't do a literature search. Please, do your homework.
- Sometimes this arises because of a mis-understanding of the
proposed research, and a mis-remembering of what's been done. The best
way to combat this is to make sure you've touched on all related and
are explicit about what you're doing that they didn't.
- Not enough detail/vagueness
- This usually arises because the PI just plain hasn't figured out
how they're going to proceed. If the reviewers liked the problem area
and the approach, then this proposal is a very good candidate for
thinking/working on some more, and re-submitting.
- You may argue that, if you knew how to do something, you wouldn't
be asking for money to do it.. Don't confuse not knowing how to do
something with not enough detail. For example, I don't know what roads
to take to get from St. Louis to New Orleans, but I do know enough to
pop-up map quest and get driving directions, and that there's probably
a north-south freeway that parallels the Mississippi, and that it will
take a day or two to get there.
- There's not enough research/nothing new
- This is slightly different than it's already been done. I usually
see this with research ideas that have reached the point where most of
the things left to do are fairly straight-forward, or just require an
application of existing, known techniques. Yes, engineering/follow
through is important, but it's not what NSF funds.
- There's usually no fixing this sort of proposal. It may, however,
make a great start/first third of your next proposal.
- Too ambitious
- I know of several cases where the research in a proposal that got
rejected as "too ambitious" was mostly completed in the following
year... I think the way to avoid this review is to include at least a
few pages of fairly short-term goals that are clearly achievable. Then
you can be pretty ambitious in the rest of the proposal.
- Lack of evaluation/no application
- This happens if the reviewer thinks you're solving a non-existent
problem, or if your proposed solution won't be any better than
- If your work is just too far from being usable (yet) in a
real-world application, at least come up with compelling test cases
and evaluation criteria.
- Poorly written
- There are services you can hire that will help you clean up your
writing. If you aren't a native speaker, or you don't write well, use
them. Most universities have a writing center as well.
- Write the proposal early, let it sit, and go back to it.
- Don't cobble together bits and pieces from papers and expect it to flow.
- If you're working with multiple PIs, designate someone as the
coordinator and make sure everyone's reading everyone else's part.
How to make a reviewer unhappy
Don't spell check. Change formatting halfway through the
paper. Use really small text. Don't check your figure numbers and
captions. Use equations without defining the variables. Bounce around
from subject to subject. Claim you're going to do x then actually
describe y. List all of your achievements and describe all of your
current research and spend one page describing new work. Define 20
equations without any accompanying text. Make it really difficult to
dig out the broader impact, intellectual merit, and proposed work. Ask
for tons of money for lots of students to do very little work. Assume your reviewer knows your field intimately, and jump straight to the details. Cut and paste three existing proposals from different people into one proposal, and add a summary page that "glues" the result together.