Book Author Match

A benchmark to evaluate matching famous books to their correct authors.

Run Benchmark
Questions

18

Leaderboard Entries

23

Best Score

100/100

Leaderboard
Rank Model Score Run Date Actions
Gemma 4 E4B IT (LMStudio)
3200 MB
100
2414.0ms median
 2026-04-02 21:21:36 View Details
2 GPT-5.4 nano
100
915.0ms median
 2026-03-17 19:26:33 View Details
3 GPT-5.4 mini
100
934.5ms median
 2026-03-17 19:04:52 View Details
4 Claude Haiku 4.5
100
1690.5ms median
 2026-03-03 06:02:12 View Details
5 Gemma 3 12B (LMStudio)
8100 MB
100
9186.0ms median
 2026-03-02 23:36:19 View Details
6 Gemma 2 9B (LMStudio)
5800 MB
100
2996.5ms median
 2026-03-02 23:07:32 View Details
7 GPT-5 mini
100
2239.5ms median
 2026-03-02 18:32:38 View Details
8 Phi-4 (LMStudio)
9100 MB
94
14947.5ms median
 2026-03-03 01:15:26 View Details
9 Ministral 8B (LMStudio)
4900 MB
94
2826.0ms median
 2026-03-03 00:20:30 View Details
10 Granite 3.2 8B (LMStudio)
4900 MB
94
5832.0ms median
 2026-03-02 23:50:13 View Details
11 OLMo 3 7B (LMStudio)
4300 MB
94
4290.5ms median
 2026-03-02 20:26:52 View Details
12 GPT-5 nano
94
1431.5ms median
 2026-03-02 18:36:57 View Details
13 Llama 3 8B (LMStudio)
4900 MB
89
3735.5ms median
 2026-03-03 00:10:11 View Details
14 Llama 3.1 8B (LMStudio)
4900 MB
89
4019.5ms median
 2026-03-02 20:39:54 View Details
15 Phi-3.5 Mini (LMStudio)
2500 MB
83
2236.5ms median
 2026-03-03 00:33:25 View Details
16 Qwen3.5 2B (LMStudio)
2700 MB
72
2935.0ms median
 2026-03-03 19:29:55 View Details
17 Gemma 4 12B (LMStudio)
7560 MB
67
4109.5ms median
 2026-06-03 19:36:16 View Details
18 Qwen3.5 4B (LMStudio)
3400 MB
61
6148.0ms median
 2026-03-03 21:39:37 View Details
19 Llama 3.2 1B (LMStudio)
1300 MB
44
1357.5ms median
 2026-03-03 00:15:09 View Details
20 Llama 2 7B (LMStudio)
4900 MB
33
2283.0ms median
 2026-03-03 00:01:00 View Details
21 SmolLM2 1.7B (LMStudio)
1100 MB
22
694.0ms median
 2026-03-03 05:51:51 View Details
22 Gemma 2B (LMStudio)
1500 MB
11
717.0ms median
 2026-03-02 23:13:05 View Details
23 Gemma 2 2B (LMStudio)
1500 MB
11
2014.5ms median
 2026-03-02 23:00:08 View Details
Questions

Question
Who wrote 'The Great Gatsby'?
Question payload
{
  "question_text": "Who wrote 'The Great Gatsby'?",
  "answer_type": "multiple_choice",
  "correct_answer": "F. Scott Fitzgerald",
  "category": "literature",
  "difficulty": "easy",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "John Steinbeck",
    "Sinclair Lewis",
    "Ernest Hemingway",
    "F. Scott Fitzgerald"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'To Kill a Mockingbird'?
Question payload
{
  "question_text": "Who wrote 'To Kill a Mockingbird'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Harper Lee",
  "category": "literature",
  "difficulty": "easy",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Truman Capote",
    "E.B. White",
    "Harper Lee",
    "Carson McCullers"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'Pride and Prejudice'?
Question payload
{
  "question_text": "Who wrote 'Pride and Prejudice'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Jane Austen",
  "category": "literature",
  "difficulty": "easy",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Jane Austen",
    "Mary Shelley",
    "Charlotte Brontë",
    "Elizabeth Gaskell"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'Moby-Dick'?
Question payload
{
  "question_text": "Who wrote 'Moby-Dick'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Herman Melville",
  "category": "literature",
  "difficulty": "medium",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Nathaniel Hawthorne",
    "H. Melvilline",
    "Herman Melvilleburg",
    "Herman Melville"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'The Brothers Karamazov'?
Question payload
{
  "question_text": "Who wrote 'The Brothers Karamazov'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Fyodor Dostoevsky",
  "category": "literature",
  "difficulty": "hard",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Fyodor Dostoevsky",
    "Leo Tolstoy",
    "Nikolai Gogol",
    "Ivan Turgenev"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'Ulysses'?
Question payload
{
  "question_text": "Who wrote 'Ulysses'?",
  "answer_type": "multiple_choice",
  "correct_answer": "James Joyce",
  "category": "literature",
  "difficulty": "hard",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "James Joyce",
    "Samuel Beckett",
    "Virginia Woolf",
    "T. S. Eliot"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'Beloved'?
Question payload
{
  "question_text": "Who wrote 'Beloved'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Toni Morrison",
  "category": "literature",
  "difficulty": "medium",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Alice Walker",
    "Toni Morrison",
    "Gloria Naylor",
    "Toni Cade Bambara"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'Brave New World'?
Question payload
{
  "question_text": "Who wrote 'Brave New World'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Aldous Huxley",
  "category": "literature",
  "difficulty": "easy",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Aldous Lee",
    "George Orwell",
    "Ray Bradbury",
    "Aldous Huxley"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'The Catcher in the Rye'?
Question payload
{
  "question_text": "Who wrote 'The Catcher in the Rye'?",
  "answer_type": "multiple_choice",
  "correct_answer": "J.D. Salinger",
  "category": "literature",
  "difficulty": "easy",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Jerome D. Salinger (pen name variant)",
    "D. J. Saling",
    "J. D. Salter",
    "J.D. Salinger"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'The Hobbit'?
Question payload
{
  "question_text": "Who wrote 'The Hobbit'?",
  "answer_type": "multiple_choice",
  "correct_answer": "J.R.R. Tolkien",
  "category": "literature",
  "difficulty": "easy",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "C.S. Lewis",
    "J.R.R. Tolkien",
    "E.R. Eddison",
    "G.K. Chesterton"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'The Name of the Rose'?
Question payload
{
  "question_text": "Who wrote 'The Name of the Rose'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Umberto Eco",
  "category": "literature",
  "difficulty": "hard",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Umberto Verdi",
    "Giovanni Ecochi",
    "Alessandro Moretti",
    "Umberto Eco"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'Crime and Punishment'?
Question payload
{
  "question_text": "Who wrote 'Crime and Punishment'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Fyodor Dostoevsky",
  "category": "literature",
  "difficulty": "medium",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Fyodor Dostoevsky",
    "Leo Tolstoy",
    "Ivan Turgenev",
    "Nikolai Gogol"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'One Hundred Years of Solitude'?
Question payload
{
  "question_text": "Who wrote 'One Hundred Years of Solitude'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Gabriel García Márquez",
  "category": "literature",
  "difficulty": "medium",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Isabel Allende",
    "Jorge Luis Borges",
    "Gabriel García Márquez",
    "Mario Vargas Llosa"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'The Trial'?
Question payload
{
  "question_text": "Who wrote 'The Trial'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Franz Kafka",
  "category": "literature",
  "difficulty": "medium",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Max Brod",
    "Thomas Mann",
    "Robert Musil",
    "Franz Kafka"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'The Stranger'?
Question payload
{
  "question_text": "Who wrote 'The Stranger'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Albert Camus",
  "category": "literature",
  "difficulty": "medium",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "André Gide",
    "Albert Camus",
    "Jean-Paul Sartre",
    "Raymond Queneau"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote '1984'?
Question payload
{
  "question_text": "Who wrote '1984'?",
  "answer_type": "multiple_choice",
  "correct_answer": "George Orwell",
  "category": "literature",
  "difficulty": "easy",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Ray Bradbury",
    "Anthony Burgess",
    "George Orwell",
    "Aldous Huxley"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'Invisible Man'?
Question payload
{
  "question_text": "Who wrote 'Invisible Man'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Ralph Ellison",
  "category": "literature",
  "difficulty": "medium",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "James Baldwin",
    "Richard Wright",
    "Ralph Ells",
    "Ralph Ellison"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}

Question
Who wrote 'Frankenstein'?
Question payload
{
  "question_text": "Who wrote 'Frankenstein'?",
  "answer_type": "multiple_choice",
  "correct_answer": "Mary Shelley",
  "category": "literature",
  "difficulty": "easy",
  "tags": [
    "knowledge",
    "books",
    "authors"
  ],
  "choices": [
    "Percy Bysshe Shelley",
    "Claire Clairmont",
    "Mary Shelley",
    "John Polidori"
  ],
  "evaluation_criteria": {
    "exact_match": true,
    "case_sensitive": false,
    "contains": false,
    "required_fields": [],
    "tolerance": 0.0,
    "alternatives": []
  }
}