Back to all updates

almost 11 years ago

Why judge apps by criteria that include how soon they are entered? Doesn’t this deter a better submission that takes longer to develop?

We understand and agree that there are trade-offs when setting selection criteria. We’ve run several contests and we are determined to make both the rules and our thinking as clear as possible.  

There are two stages to this contest and a separate selection criteria for each.
The Finalists are selected first.  For this stage, we chose an objective set of criteria (detailed in the Rules section) that asks contestants to meet a sufficiently high bar as fast as possible.

Each Finalist is then given a cash prize and two 3-day periods for beta tests of their app.  During this period, Finalists are expected to use their cash prize to offset the cost of making improvements and enhancements based on the actual user experiences of the beta test.
The Finalists are selected this way because the alternative would result in a process that is less likely to get the best outcome and would introduce greater uncertainty under a lengthier contest.  

The Winner is selected second.  For this stage, we will use subjective criteria and a larger group of judges. This is when the best app will be chosen.
We could have elected to be more prescriptive about our requirements, increasing the likelihood that waiting longer would generate the winning app.   We chose not to because we wanted to establish minimum requirements and give the developer community as much creative freedom as possible.

If we had included additional selection criteria, we would have essentially designed the “best” app.   And it would have taken longer for the community to respond.  We made a deliberate choice NOT to overly prescribe what the solution to this problem could be, beyond certain essential capabilities.

Another benefit of this approach is that we will know soon who the finalists are, and do not have to wait until the end of the submission period in July in order to assess entries.  Our judges (Jon Bon Jovi, Aneesh Chopra, Peter Levin, Todd Park, and others) know that they need to think carefully and evaluate five entries.  That predictability is important to them, and to us.

We take concerns about selection criteria seriously.  When we designed this challenge, we tried to balance objective criteria with maximum creative freedom. We want our community to build the best answer based on the solid platform of clear and concise criteria.