OpenAI’s GPT-4o AI model has achieved the highest score in Solidity programming rankings, surpassing o1-preview, o1-mini, and other competitors. This is according to research data from BrainDAO.
The top five are:
- GPT-4o — 80.05 points;
- o1-preview — 77.61;
- o1-mini — 75.08;
- Claude 3.5 — 74.88;
- Grok-2 — 74.64.
The SolidityBench rankings were compiled using two benchmarks, NaïveJudge and HumanEval, designed to assess and rank AI models’ proficiency in generating smart contract code.
Neural network code was evaluated based on “strict criteria” such as implementing key functions, handling edge cases, error management, correct syntax usage, overall structure, and maintainability. Gas efficiency and storage management were also analyzed.
The scoring range is from 0 to 100, providing a comprehensive assessment of functionality, security, and efficiency.
Previously, Meta developed the AI tool CodeCompose for generating program code.
In September, Google released two updated versions of the Gemini 1.5 models, which showed improvements in programming language capabilities.