🧪 MD-EvalBench Leaderboard
The First Comprehensive Benchmark for LLMs in Molecular Dynamics
336
MD-KnowledgeEval Questions
368
LAMMPS-SyntaxEval Questions
566
LAMMPS-CodeGenEval Tasks
1,270
Total Evaluation Samples
MD-EvalBench is the first comprehensive benchmark for evaluating Large Language Models in the Molecular Dynamics (MD) domain, proposed in the paper "MDAgent2: Large Language Model for Code Generation and Knowledge Q&A in Molecular Dynamics".
The benchmark consists of three evaluation datasets:
- MD-KnowledgeEval (336 questions): Theoretical knowledge assessment covering interatomic potentials, integration algorithms, equilibrium conditions, and statistical ensembles.
- LAMMPS-SyntaxEval (368 questions): Command and syntax understanding assessment for LAMMPS scripting.
- LAMMPS-CodeGenEval (566 tasks): Automatic code generation quality assessment for executable LAMMPS scripts.
Models are evaluated on both Question Answering (knowledge + syntax) and Code Generation (execution success + human scoring) capabilities.
To access the evaluation datasets, code, and submission guidelines, please visit our GitHub repository.
Knowledge & Syntax QA Performance
Performance comparison on MD-KnowledgeEval and LAMMPS-SyntaxEval. Δ vs 8B indicates the absolute improvement over the Qwen3-8B baseline.
| # | Model | Size | Overall Avg | MD-KnowledgeEval | LAMMPS-SyntaxEval | Δ vs 8B |
|---|---|---|---|---|---|---|
| 🥇 | Qwen3-max Closed | Large | 82.49 | 86.57 | 78.40 | +11.99 |
| 🥈 | Qwen3-32b Open | 32B | 77.34 | 81.94 | 72.74 | +6.84 |
| 🥉 | MD-Instruct-8B Open | 8B | 74.67 | 76.89 | 72.45 | +4.17 |
| 4 | Qwen-flash Closed | Large | 73.47 | 78.64 | 68.30 | +2.97 |
| 5 | Qwen3-14b Open | 14B | 72.91 | 77.90 | 67.92 | +2.41 |
| 6 | Qwen3-8b Open | 8B | 70.50 | 75.15 | 65.84 | 0.00 |