SBIR/STTR Grant Writing Best Practices: A Reviewer s Perspective. Willard M. Freeman, Ph.D. Dept. Pharmacology Penn State College of Medicine

Transcription

1 SBIR/STTR Grant Writing Best Practices: A Reviewer s Perspective Willard M. Freeman, Ph.D. Dept. Pharmacology Penn State College of Medicine

2 Your Goal Investment ($100,000s of dollars) With no ownership stake Everything you do in the proposal process is centered on this goal.

3 About NIH SBIR/STTR Grant Review Multi-stage process Study sections provide a peer-review based evaluation and ranking of research proposals. The f-word (funding) is not used at a study section Programs/Institutes make funding recommendations based on peer-review, programmatic goals, and budgets. Final funding decisions are made by NIH councils.

4 Key members of the peer-review process Scientific Review Administrator/Officer (SRA/O) Center for Scientific Review (CSR) employee Generally a PhD level person Oversees peer-review Your contact with them is generally limited to providing an update letter Program Officer (PO) NIH employee Often has a home base at a single institute Your contact with them can be more extensive both in the presubmission, post-review, and (hopefully) post-award phase. Study section reviewers No contact with study section reviewers in regards to your proposal - Ever! Know your audience

5 Who is on an SBIR/STTR study section? More representation from industry on SBIR/STTR panels than standard study sections

6 SBIR/STTR study sections Centered around a general area Neuropharmacology Respiratory Sciences Cardiovascular Sciences Will see a range of applications Phase I, Phase II, Fast-track, STTR Multiple institutes Meet three times a year (each cycle) Reviewers are experts in the area but not all of them are going to be experts in your disease. You have to briefly educate them on the disease and the unmet medical need. Each meeting has about applications

7 Grant Application Let me tell you a story

8 The review process

9 The review process Your application gets assigned to three reviewers to score. a. Primary, Secondary, Tertiary b. Sometimes additional phone or mail reviewers for very specific things

10 The review process 1-2 Months before the study section meets reviewers get a CD Each reviewer gets 4-7 grants to review

11 The review process And they promptly forget about it About 2-3 weeks before the meeting they remember, hopefully, to start reading applications. The reviewer looks at the specific aims page, CVs, and the company. Often the reviewer will give a fast read through of the whole proposal.

12 The review process At this point there will be a general impression of good, ok, bad. Make sure you are in the first two. After taking some notes as they read through all the proposals they will often forget about all this and work on that paper they need to finish. Then they get an from the SRA reminding them that written reviews are due in a week. SRA Reviewer They go back and finish the written review, scoring each section and giving an overall impact.

13 The review process If it is a research area tangential to their work they may read a couple of papers (a Nature Reviews of the subject and one of the principal papers from the research team). Based on the preliminary scores an order of review is made. Best preliminary score to worst preliminary score. Only the top 40-60% are scheduled to be discussed by the group. Don t be the sad part of a nature film.

14 The review process Everyone heads to the study section meeting

15

16 Review Criteria Significance. Does the proposed project have commercial potential to lead to a marketable product, process or service? Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field? 2. Investigator(s). Are the PD/PIs, collaborators, and other researchers well suited to the project? If Early Stage Investigators or New Investigators, or in the early stages of independent careers, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? If the project is collaborative or multi-pd/pi, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project? 3. Innovation. Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed? 4. Approach. Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed? If the project involves clinical research, are the plans for 1) protection of human subjects from research risks, and 2) inclusion of minorities and members of both sexes/ genders, as well as the inclusion of children, justified in terms of the scientific goals and research strategy proposed? 5. Environment. Will the scientific environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements?

17 Review Criteria Significance. Will this lead to a product that addresses an important problem and changes methods, technologies, or treatments? 2. Investigator(s). Do they have appropriate experience and training, with a demonstrated record of accomplishments and leadership? 3. Innovation. Will successful completion shift current research or clinical practice? 4. Approach. Are the studies well-reasoned and appropriate to accomplish the specific aims and establish feasibility of the product? 5. Environment. Are the resources available adequate for the project proposed?

18 Review Criteria Phase II (Type 2 R42 and Type 2 R44 applications). How well did the applicant demonstrate progress toward meeting the Phase I objectives, demonstrating feasibility, and providing a solid foundation for the proposed Phase II activity? Did the applicant submit a concise Commercialization Plan that adequately addresses the areas described in the SBIR/STTR Guide for Reviewers? To what extent was the applicant able to obtain letters of interest, additional funding commitments, and/or resources from the private sector or non-sbir/sttr funding sources that would enhance the likelihood for commercialization? Does the project carry a high degree of commercial potential, as described in the Commercialization Plan? Fast Track (Type 1 R42 and Type 1 R44 applications). Does the Phase I application specify clear, appropriate measurable goals (criteria for success, milestones) that should be achieved prior to initiating Phase II? Are there clear criteria by which success will be evaluated? Did the applicant submit a concise Commercialization Plan that adequately addresses the described in the Guide for Reviewers? To what extent was the applicant able to obtain letters of interest, additional funding commitments, and/or resources from the private sector or non-sbir/sttr funding sources that would enhance the likelihood for commercialization? Does the project carry a high degree of commercial potential, as described in the Commercialization Plan?

19 Overall impact score = average of all reviewers x =perfect, 90=atrocious

20 Reviewer Stereotypes Dr. PhD Scientifically meticulous Wants some mechanistic insight Knows the literature inside and out Knows what projects have failed in the past Dr. Chairman Big picture Wants to be excited by the research Process focused over methodological details Good scientist but doesn t read Nature every week

21 Reviewer Stereotypes Dr. Fusspot Grumpy Attention to detail above practicality Negative by nature, don t give them ammunition Dr. MD How would I use this in the clinic? Mechanism, schmecinism, we don t know the mechanism of many of the drugs I prescribe. Will this be better than what I am already doing?

22 And now the dry part

23 Major review points Successful grants generate enthusiasm for the project Can be high risk but clear measurements for success and failure The best proposals have Good science Defined product goal Scenario for use Clear experimental plan Delineated outcomes

24 Common SBIR/STTR Pitfalls Budgets Rules have changed to increase SBIR/STTR budgets. Phase I increased to $150,000. You can exceed this and many projects do. Get advice from your PO. NIH maximum for salaries. Especially for investigators with an academic appointment. People months versus percent efforts for investigators splitting effort between university and company jobs Going to rent new space, outsource a service, or buy an instrument, include a quote Check with PO about pass through. There are often rules requiring a % of work (as defined by $s) to be performed at the company. People Collaborations Dr. Big Cheese is going to help us. Get a letter of support, not just pro forma but includes the title of the application and what they are specifically going to do. Write it for the collaborator if need be. Choice of PI Does it look like a backdoor to provide R01-like funding? A junior person provides separation but they lack a track record. Co-PIs, What if you disagree?

25 Common SBIR/STTR Pitfalls Animals Animal welfare can now be included in scientific review. Must address the five points. You are still responsible for animal protection even if it is being done by a collaborator or a private company. Pain and discomfort, procedures not just that a vet will look at the animals. Human Subjects Who are the patients and what are their relevant characteristics? Every possible detail Gender Must explain if only one gender. Even for a pilot study. Race Because you don t have a specific race in your recruitment area is not a rationale. Children (children are under 21 not 18) year old participation must be described if to be included.

26 Common SBIR/STTR Pitfalls Intellectual property After you invent this what is to prevent someone else for making knock-offs? Seriously, after you invent this what is to prevent someone else for making knock-offs? Strategy for use patents. What makes your use patent likely to survive a challenge? The university owns the patent. Tell us your plan. Approach Specific milestones. Not just that something will get done but that it will have X>10, Y<1 as measured by technique Q. Fruitcake text. Not crazy fruitcake but dense as a fruitcake. Make sure your figures come out crisp and legible once converted to a pdf

27 Resubmissions So you didn t get funded the first time. Really read the reviews Discuss with your PO Respond/answer their critiques You will get some of the same reviewers. The first thing they will do is look at the previous review and see if the issues have been resolved.

28 Unsolicited Advice Get other people to read your grant before you submit it. Buy them lunch, beg Have them be brutal Take their advice, most of the time If they didn t get a point neither will the reviewers No smoke and mirrors They ve seen it before and won t be fooled

29 Unsolicited Advice SBIRs and STTRs get treated pretty similarly But with different funding rates Phase II have a high bar Fast-track (combined Phase I and II) applications have a very high bar

30 Chemoinformatic approach to reposition existing drugs to new diseases Excellent group, all the right Nature and Cell level publications Good idea to shorten development timelines by using drug with existing tox, PK/PD data Case Studies - Failure Achilles heal Why is this going to work when lots of other projects like this have failed?

31 Case Studies - Failure Genetic test to determine risk for serious psychiatric disorder Good prelim data, strong group Novel combination of genetic data with other patient variables Achilles heal How does this fit into clinical practice? There is no treatment for this disorder. You are going to tell people that they are likely to develop a disease for which there is no treatment.

32 Blood biomarker for depression Some prelim data, strong patient population. All the right technologies and approaches. Case Studies - Failure Achilles heal How does this improve clinical care? If someone is depressed can t this be assessed by standard means?

33 Case Studies - Failure Protein biomarker for CNS degeneration Excellent set of patient samples The best technology and collaborators in the world Unmet medical need Achilles heal Company was headquartered in a house and all of the work was to be done at other sites.

34 Case Studies - Success Pharmacodynamic marker of CNS drug action No existing methods Highly novel technology Very clear development plan Excellent people with best equipment Even with a small market, only Phase I- III trials, meets a clear need in drug development

35 Biomarker for treatment decision making in ER setting Wrong treatment is potentially fatal Very clear clinical scenario Development plan centered around very short time frame available in clinic High risk, high return Case Studies - Success

36 Final Recommendations Start early Get feedback from NIH on your questions Internal review Dot every i and cross every t Use existing knowledge sbir_policy.htm Preferred Provider grant writing assistance

37 Good Hunting!

38 The real review process 1. Your application gets assigned to three reviewers to score. a. Primary, Secondary, Tertiary b. Sometimes additional phone or mail reviewers for very specific things 2. They get your application as a pdf on a CD about 1-2 months before the review meeting. 3. Each reviewer gets 4-7 applications to read. 4. They forget about it. 5. About 2-3 weeks before the meeting they remember (hopefully) to start reading applications.

39 The real review process 6. The reviewer looks at the specific aims page, CVs, and the company. a. If your company is already in existence a nice website can be a small help 7. Often the reviewer will give a fast read through of the whole proposal. a. At this point there will be a general impression of good, ok, bad. Make sure you are in the first two. 8. After taking some notes as they read through all the proposals they will often forget about all this and work on that paper they need to finish. 9. They get an from the SRA reminding them that written reviews are due in a week. 10. They go back and finish the written review, scoring each section and giving an overall impact. a. If it is a research area tangential to their work they may read a couple of papers (a Nature Reviews of the subject and one of the principal papers from the research team). 11. A few days later the SRA will reviewers on applications with variant scores (e.g. 3, 5, and 7). a. Reviewers can read the comments from the other reviewers and see if they want to amend their score. b. This is done to make the discussed/not discussed line clearer. 12. A few days before the study section meeting reviewers can read the critiques by the other reviewers. Good reviewers will take a look at the other reviews and see where agreements and disagreements may lie.

40 The real review process 13. Based on the preliminary scores an order of review is made. Best preliminary score to worst preliminary score. 14. Only the top 40-60% are scheduled to be discussed by the group a. Applications can be raised from the dead but there must be a compelling belief that you can convince the committee that the application should receive a competitive score. 15. Your application is scheduled for 15-25min of discussion. 16. Each of the written reviewers give their preliminary impact score. 17. Each written reviewer then gives a synopsis of their review for each of the review criteria a. Significance, Investigators, Innovation, Approach, Environment 18. Other members of the study section chime in. 19. Animal and human subject protections are addressed. 20. Written reviewers give their amended overall scores. 21. Every member of the study section gives a score based on the scoring range of the written reviewers. a. For example, written reviewers give a: 3, 3, and 4. The range is 3-4. The choice is then to score it as a 3 or 4. Voting outside the range is possible. 22. Budget issues are raised, if applicable. 23. Off to the funding agency. Voodoo sacrifices are appropriate at this point.