Evaluation worksheet
The following is the evaluation worksheet our team and NIC members use to evaluate Open Call applications.
In line with our policy, please disclose here if you have any financial interest or fiduciary relationship with the applicant or organisation(s) the applicant is affiliated with – this includes, but is not limited to, the receipt or holding of honoraria; educational grants; membership; employment; consultancies; stock ownership or other equity interests; and expert testimony or patent-licensing arrangements.
Personal or professional relationships with the applicant or organisations the applicant is affiliated with which are non-financial and non-fiduciary in nature do not need to be disclosed. If you are unsure of the nature of your relationship with an application or organisation the applicant is affiliated with, please disclose it here.
If no such relationship exists please leave the field blank or write “not applicable.”
Required checkbox.
Note: Reviewers respond to each question using a scoring system on a 1-5 scale, with 1 being "poor" or "very weak," 5 being "excellent" or "very strong," and 0 being "non-applicable." All of the below are required. In general, those with higher aggregate scores will progress farther in the workflow.
Considerations: Does the application identify and acknowledge what the challenges will be? Does the application state how it is done today and the limitations? Is it clear and specific who could benefit and what the impact will be if the project is successful? Does the application cite an actual use case? Does the application state how much it will cost and how long will it take?
Considerations: Are the goals and objectives easy to find and read? Are the goals realistically met by the proposed objectives and tasks?
Considerations: Does the application propose methods appropriate to the goals and objectives? Does the application effectively apply its stated methods? Does the application suggest modified procedures in response to changing circumstances? Will the work fill a potential need or function that is currently unfilled, be reinventing the wheel or creating a solution in search of a problem? Is the proposed solution viable in the real world?
Considerations: Does the effort demonstrate a high degree of usability/accessibility? Does the application demonstrate external demand (i.e., demand originated from existing people with harms vs. potential, hypothetical users, or would-be patrons)? Does the effort's focus appropriately affect either a small number of high value or at-risk users, or a more general large numbers of users?
Considerations: Does the application clearly state the effort's technology, research + investigation, policy change, and/or public campaigns outcomes? Are they articulated technically and succinctly? Does the application explain what is new in the approach and why it will succeed? Does the project buy tactical breathing space for existing problems or push towards convergence by changing the playing field? Does the project identify any expected or past hurdles in achieving its technical goals?
Considerations: Does the application identify or recognise potential incentives to an adversary? Does the proposed work increase or decrease known attack surfaces? Does it create new surfaces? Does the proposal consider potential illicit uses? Does it discuss how an adversary might use the solution to further their own goals? Does the application identify potential unintended consequences? Does the application identify and understand the proposed work's asymmetry? Given an attacker and defender of the effort, does the application explain which role is more advantageous (effort, cost, time, etc.) and why? Does the application explain medium and long‐term strategies from both the attacker and defender point of view? Does the application discuss how one could defeat the effort and identify its deficiencies, or does it presume there are none?
Considerations: What is the plan for future development/implementation? Does the application state if the project has other funding sources? Does the project currently receive any Reset, Luminate, or Omidiyar Group support? Does the application identify any cost sharing or matching for the proposed effort? Will the effort be able to support itself by the requested funding, community sources, or other in-kind or indirect support? Does the project have a diversified funding/support stream (i.e., how dependent would the project be on Reset)?
Considerations: Does or should the effort support a collaborative open source community? How will any created intellectual property be licensed/how is it licensed? Will it be freely available to the public? Will there be usable/extensible API's or easy to manipulate text formats? How does the effort facilitate inter-project collaboration, including: talking with other efforts doing similar things and identifying potential points of overlap; acted/planned modularization code to enable others to reuse? Do the application deliverables assist other Reset projects/goals beyond this particular effort? Do the application deliverables assist other Internet freedom projects/goals beyond this particular project?
Considerations: Is the budget realistic and commensurate with both the project needs and its time frame? Is the project going to require funds immediately upon proposal approval?
Considerations: Does the application clearly identify the team member(s) and their work experience? Does the team posses the skills uniquely qualifying them to complete the proposed scope of work? Does the team have a history of successful work related to the current initiative? Is there a core team (leadership, developers, etc.) dedicated to this project? How directly connected or embedded is the team with communities we care most about?
Considerations: Does the project articulate a measurable set of evaluation criteria and milestone metrics against results? Are the metrics both quantitative and qualitative? How difficult will it be to conduct an assessment of success or failure?
Considerations: Is the proposed project and its trajectory in-line with our Goals, objectives, & areas of work? Is the proposed project and its trajectory in-line with our Principles & values?
Note: Only the first question is a required "yes" or "no." All others questions are voluntary, allowing for any general thoughts you may have after reviewing a proposal. The optional text fields provide space to include comments and questions for areas we do not specifically ask for a rating. Comments and questions posted here will be aggregated back to the the rest of the review board and the applicant.
(Do you recommend supporting this proposal’s statement of work for the requested amount, Yes or No?)
(Here you can share any specific items that made you really excited or that you see as positive about this application.)
(Here you can share any specific items that leave you feeling uneasy or that you feel warrant concern regarding this application.)
(Here you can share any specific items that warrant dire warnings, a notice of very serious dangers, and/or potential disasters.)
(Here you can provide any additional comments not captured in or suitable for the above sections.)
- Private: Visible only to staff.
- Reviewers and Staff: Visible to other reviewers and staff.
Last modified 2yr ago