OpenAI Launches GeneBench-Pro to Measure AI Scientific Reasoning
The benchmark targets the gap between AI that can follow workflows and AI that can make the high-level judgments real scientific research requires, with current models solving less than a third of problems.
Key Facts
- OpenAI released GeneBench-Pro, a computational biology benchmark that tests whether AI can judge if data is noise or significant, handle ambiguity, and perform iterative analysis.
- OpenAI's top model, GPT-5.6 Sol, scored 31.5% at the highest reasoning setting on GeneBench-Pro.
- Human experts estimate each GeneBench-Pro problem takes 20 to 40 hours to solve.
- At the lowest reasoning level, GPT-5.6 Sol's pass rate on GeneBench-Pro stayed in single digits, showing a sixfold increase with more computation.
- OpenAI stated that if progress continues at the current pace, GeneBench-Pro could be saturated by the end of the year.
Reporting from 1 source: GIGAZINE.
OpenAI announced GeneBench-Pro, a computational biology benchmark that tests whether AI can judge if data is noise or significant, handle ambiguity, and perform iterative analysis. Its top model, GPT-5.6 Sol, scored 31.5% at the highest reasoning setting, while human experts estimate each problem takes 20 to 40 hours. OpenAI says partial automation could already generate scientific value.
OpenAI released GeneBench-Pro, a benchmark designed to test whether AI can do more than recall facts or follow predefined steps. Each problem presents unorganized datasets and experimental backgrounds, requiring the AI to explore data, select analysis methods, and iterate toward an answer. OpenAI notes that scientific data rarely comes with a manual, and researchers must judge whether a pattern reflects a real phenomenon or is noise.
When tested, OpenAI's most powerful model GPT-5.6 Sol achieved a pass rate of 31.5% at the top reasoning setting. At the lowest reasoning level, the pass rate stayed in single digits, showing a sixfold increase with more computation. OpenAI stated that if progress continues at the current pace, the benchmark could be saturated by the end of the year.
Synthesized by Yomimono from the 1 cited source below, including Japanese-language reporting where cited, then editorially reviewed before publishing.