Simple Arithmetic

A benchmark to evaluate a model's ability to perform basic arithmetic: addition, subtraction, multiplication, and division.

Run Benchmark
Questions

40

Leaderboard Entries

27

Best Score

100/100

Leaderboard
Rank Model Score Run Date Actions
Gemma 4 E4B IT (LMStudio)
3200 MB
100
620.5ms median
 2026-04-02 20:57:37 View Details
2 GPT-5.4 nano
100
691.5ms median
 2026-03-17 19:15:50 View Details
3 GPT-5.4 mini
100
719.5ms median
cost warning
2026-03-17 18:52:55 View Details
4 Qwen3.5 4B (LMStudio)
3400 MB
100
838.5ms median
 2026-03-03 21:01:17 View Details
5 Qwen3.5 2B (LMStudio)
2700 MB
100
444.0ms median
 2026-03-03 19:20:30 View Details
6 Phi-4 (LMStudio)
9100 MB
100
1112.0ms median
 2026-03-03 00:50:02 View Details
7 Claude Haiku 4.5
100
602.0ms median
 2026-02-28 19:43:52 View Details
8 Qwen3 VL 8B (LMStudio)
5000 MB
100
2982.5ms median
 2026-02-28 18:36:38 View Details
9 Qwen3 4B (LMStudio)
2800 MB
100
410.5ms median
 2026-02-28 18:12:40 View Details
10 Qwen3 1.7B (LMStudio)
1100 MB
100
202.0ms median
 2026-02-28 18:04:53 View Details
11 OLMo 3 7B (LMStudio)
4300 MB
100
433.0ms median
 2026-02-28 17:36:22 View Details
12 Ministral 8B (LMStudio)
4900 MB
100
635.5ms median
 2026-02-28 17:22:01 View Details
13 Llama 3.2 1B (LMStudio)
1300 MB
100
187.0ms median
 2026-02-28 17:12:39 View Details
14 Llama 3 8B (LMStudio)
4900 MB
100
563.0ms median
 2026-02-28 16:45:31 View Details
15 Granite 3.2 8B (LMStudio)
4900 MB
100
706.0ms median
 2026-02-28 16:17:33 View Details
16 Gemma 3 12B (LMStudio)
8100 MB
100
1489.0ms median
 2026-02-28 15:50:59 View Details
17 Gemma 2 9B (LMStudio)
5800 MB
100
663.5ms median
 2026-02-28 14:59:40 View Details
18 GPT-5 mini
100
1066.5ms median
 2026-02-26 01:17:15 View Details
19 GPT-5 nano
100
881.5ms median
 2026-02-26 00:49:30 View Details
20 Qwen3.5 9B (LMStudio)
6600 MB
98
1231.5ms median
 2026-03-03 19:54:27 View Details
21 Gemma 2B (LMStudio)
1500 MB
98
310.5ms median
 2026-02-28 15:34:27 View Details
22 Gemma 2 2B (LMStudio)
1500 MB
98
649.0ms median
 2026-02-28 03:21:50 View Details
23 Gemma 4 12B (LMStudio)
7560 MB
95
1511.0ms median
 2026-06-03 18:46:24 View Details
24 SmolLM2 1.7B (LMStudio)
1100 MB
95
196.0ms median
 2026-02-28 19:18:55 View Details
25 Llama 3.1 8B (LMStudio)
4900 MB
90
607.0ms median
 2026-02-28 17:00:54 View Details
26 Llama 2 7B (LMStudio)
4900 MB
57
493.0ms median
 2026-02-28 03:04:20 View Details
27 Phi-3.5 Mini (LMStudio)
2500 MB
10
355.0ms median
 2026-02-28 17:48:52 View Details
Questions

Question
What is 16 ÷ 8?
Question payload
{
  "question_text": "What is 16 ÷ 8?",
  "answer_type": "numeric",
  "correct_answer": 2.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 47 - 7?
Question payload
{
  "question_text": "What is 47 - 7?",
  "answer_type": "numeric",
  "correct_answer": 40.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "subtraction"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 12 ÷ 12?
Question payload
{
  "question_text": "What is 12 ÷ 12?",
  "answer_type": "numeric",
  "correct_answer": 1.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 3 × 7?
Question payload
{
  "question_text": "What is 3 × 7?",
  "answer_type": "numeric",
  "correct_answer": 21.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 46 + 13?
Question payload
{
  "question_text": "What is 46 + 13?",
  "answer_type": "numeric",
  "correct_answer": 59.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "addition"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 11 × 11?
Question payload
{
  "question_text": "What is 11 × 11?",
  "answer_type": "numeric",
  "correct_answer": 121.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 10 + 50?
Question payload
{
  "question_text": "What is 10 + 50?",
  "answer_type": "numeric",
  "correct_answer": 60.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "addition"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 12 × 4?
Question payload
{
  "question_text": "What is 12 × 4?",
  "answer_type": "numeric",
  "correct_answer": 48.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 64 ÷ 4?
Question payload
{
  "question_text": "What is 64 ÷ 4?",
  "answer_type": "numeric",
  "correct_answer": 16.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 2 × 8?
Question payload
{
  "question_text": "What is 2 × 8?",
  "answer_type": "numeric",
  "correct_answer": 16.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 6 × 2?
Question payload
{
  "question_text": "What is 6 × 2?",
  "answer_type": "numeric",
  "correct_answer": 12.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 4 × 4?
Question payload
{
  "question_text": "What is 4 × 4?",
  "answer_type": "numeric",
  "correct_answer": 16.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 39 - 23?
Question payload
{
  "question_text": "What is 39 - 23?",
  "answer_type": "numeric",
  "correct_answer": 16.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "subtraction"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 10 × 6?
Question payload
{
  "question_text": "What is 10 × 6?",
  "answer_type": "numeric",
  "correct_answer": 60.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 41 - 38?
Question payload
{
  "question_text": "What is 41 - 38?",
  "answer_type": "numeric",
  "correct_answer": 3.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "subtraction"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 12 ÷ 12?
Question payload
{
  "question_text": "What is 12 ÷ 12?",
  "answer_type": "numeric",
  "correct_answer": 1.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 23 - 8?
Question payload
{
  "question_text": "What is 23 - 8?",
  "answer_type": "numeric",
  "correct_answer": 15.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "subtraction"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 23 + 18?
Question payload
{
  "question_text": "What is 23 + 18?",
  "answer_type": "numeric",
  "correct_answer": 41.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "addition"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 11 - 11?
Question payload
{
  "question_text": "What is 11 - 11?",
  "answer_type": "numeric",
  "correct_answer": 0.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "subtraction"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 7 × 7?
Question payload
{
  "question_text": "What is 7 × 7?",
  "answer_type": "numeric",
  "correct_answer": 49.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 6 ÷ 2?
Question payload
{
  "question_text": "What is 6 ÷ 2?",
  "answer_type": "numeric",
  "correct_answer": 3.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 90 + 9?
Question payload
{
  "question_text": "What is 90 + 9?",
  "answer_type": "numeric",
  "correct_answer": 99.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "addition"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 4 × 10?
Question payload
{
  "question_text": "What is 4 × 10?",
  "answer_type": "numeric",
  "correct_answer": 40.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 6 × 6?
Question payload
{
  "question_text": "What is 6 × 6?",
  "answer_type": "numeric",
  "correct_answer": 36.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 49 + 6?
Question payload
{
  "question_text": "What is 49 + 6?",
  "answer_type": "numeric",
  "correct_answer": 55.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "addition"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 9 ÷ 9?
Question payload
{
  "question_text": "What is 9 ÷ 9?",
  "answer_type": "numeric",
  "correct_answer": 1.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 12 × 4?
Question payload
{
  "question_text": "What is 12 × 4?",
  "answer_type": "numeric",
  "correct_answer": 48.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 95 ÷ 5?
Question payload
{
  "question_text": "What is 95 ÷ 5?",
  "answer_type": "numeric",
  "correct_answer": 19.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 3 × 4?
Question payload
{
  "question_text": "What is 3 × 4?",
  "answer_type": "numeric",
  "correct_answer": 12.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 73 + 76?
Question payload
{
  "question_text": "What is 73 + 76?",
  "answer_type": "numeric",
  "correct_answer": 149.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "addition"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 95 ÷ 5?
Question payload
{
  "question_text": "What is 95 ÷ 5?",
  "answer_type": "numeric",
  "correct_answer": 19.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 60 ÷ 4?
Question payload
{
  "question_text": "What is 60 ÷ 4?",
  "answer_type": "numeric",
  "correct_answer": 15.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 63 - 48?
Question payload
{
  "question_text": "What is 63 - 48?",
  "answer_type": "numeric",
  "correct_answer": 15.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "subtraction"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 32 ÷ 8?
Question payload
{
  "question_text": "What is 32 ÷ 8?",
  "answer_type": "numeric",
  "correct_answer": 4.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 8 × 6?
Question payload
{
  "question_text": "What is 8 × 6?",
  "answer_type": "numeric",
  "correct_answer": 48.0,
  "category": "simple_arithmetic",
  "difficulty": "easy",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 12 × 6?
Question payload
{
  "question_text": "What is 12 × 6?",
  "answer_type": "numeric",
  "correct_answer": 72.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "multiplication"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 121 ÷ 11?
Question payload
{
  "question_text": "What is 121 ÷ 11?",
  "answer_type": "numeric",
  "correct_answer": 11.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 168 ÷ 12?
Question payload
{
  "question_text": "What is 168 ÷ 12?",
  "answer_type": "numeric",
  "correct_answer": 14.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 61 + 32?
Question payload
{
  "question_text": "What is 61 + 32?",
  "answer_type": "numeric",
  "correct_answer": 93.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "addition"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
What is 28 ÷ 2?
Question payload
{
  "question_text": "What is 28 ÷ 2?",
  "answer_type": "numeric",
  "correct_answer": 14.0,
  "category": "simple_arithmetic",
  "difficulty": "medium",
  "tags": [
    "arithmetic",
    "division"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}