Technology

Google DeepMind Paid $200,000 to People Who Could Build Better AI Tests

Martin HollowayPublished 2w ago4 min readBased on 4 sources
Reading level
Google DeepMind Paid $200,000 to People Who Could Build Better AI Tests

Google DeepMind's Kaggle-hosted hackathon "Measuring AGI — Cognition and Values" has concluded, with final standings across five categories and a combined $200,000 prize pool. The competition was announced in a March 2026 blog post titled "Measuring Progress Towards AGI: A Cognitive Framework" on blog.google. Participants were asked to build tests and write reports assessing different mental abilities of AI systems. Source: Google DeepMind Blog

This hackathon worked differently from most online coding competitions. Instead of trying to get the best score on a test that already existed, participants had to create their own test. They used a free, open-source software tool called the kaggle-benchmarks SDK (available on GitHub under an Apache 2.0 license). Each submission included a public link to the participant's test so judges could evaluate how well the test was designed, not just the results it produced. The tests stayed private during the competition and were automatically published for everyone to see after the deadline. Any private data files attached to a public report were also made public at that time. Source: Kaggle Competition Overview

Each submission was a written report capped at 1,500 words, with a required title, subtitle, and detailed analysis. A cover image was mandatory. Participants had to choose one of five categories: Learning, Metacognition, Attention, Executive Function, or Social Cognition. Optional elements included a Media Gallery and a Public Notebook. Drafts that were not submitted by the deadline were not considered. Each team could submit only one entry, following Kaggle's hackathon rules. Source: Kaggle Competition Rules

The $200,000 prize pool was divided among 14 winners. Four $25,000 grand prizes went to the best submissions across all categories. Ten category prizes of $10,000 each were awarded as two per category across the five areas, with no one winning twice. Winning entries were placed under a CC0 license, meaning they entered the public domain and anyone can use them freely. Source: Kaggle Competition Rules

Teams could have up to five members, and teams could merge if the team leader initiated it, as long as submission limits were respected. Because running AI model evaluations requires significant computing power, participants received extra Kaggle computing credits of $50 per day and $500 per month. Source: Kaggle Competition Overview

The final private leaderboard was calculated using about 70% of the test data, with the rest reserved for the public leaderboard. The competition is now complete. The leaderboard page is titled "Measuring Progress Toward AGI - Cognitive Abilities." Source: Kaggle Leaderboard

The competition sponsor is Google LLC, headquartered at 1600 Amphitheatre Parkway, Mountain View, California. Source: Kaggle Competition Rules

The broader context here is worth understanding. The five categories come from cognitive science, the study of how minds work. Metacognition means a system's ability to think about its own thinking. Executive function covers planning, switching between tasks, and holding back impulses. Attention, social cognition, and learning round out the set. This is not the first attempt to measure progress toward artificial general intelligence, but it is one of the more carefully organized. By asking participants to build the test itself rather than just perform well on an existing one, the competition flipped the usual dynamic. The tests people created are reusable tools that other researchers can use later. Combined with the public domain licensing and automatic publication of all tests after the deadline, the competition was designed to produce a collection of freely available AI evaluation tools.

Whether these community-built tests actually get used in future research is an open question. The free, open-source software tool used to build them makes it easy for anyone to continue the work outside the competition. The five categories also provide a structure that other organizations could adopt. Whether DeepMind's five areas become the standard way to think about measuring AI, or the field settles on a different set of categories, is something this competition may help inform but cannot answer on its own.

For anyone who works on evaluating AI systems, the now-public tests and the software tool are available to use right away. The competition's approach — build a test, write about it, release it — is a model other organizations could copy when sponsoring similar work.