Run Details #670

Model

Gemma 2B (LMStudio)

Score

11

Correct (≥100)

2/18

Incorrect (<100)

16

Median Time

717.0ms

Tokens Used

2703

Cost

685µ$

Benchmark: 0153_book_author_match
Run Date: 2026-03-02 23:13:05
Questions (18)

Question: Who wrote 'The Great Gatsby'?
Expected 
F. Scott Fitzgerald
Provided 
{
  "reasoning": "The Great Gatsby was written by F. Scott Fitzgerald.",
  "answer": "F. Scott Fitzgerald"
}

Question: Who wrote 'To Kill a Mockingbird'?
Expected 
Harper Lee
Provided 
{
  "reasoning": "Harper Lee wrote 'To Kill a Mockingbird' in 1960.",
  "answer": "3. Harper Lee"
}

Question: Who wrote 'The Name of the Rose'?
Expected 
Umberto Eco
Provided 
{
  "reasoning": "The Name of the Rose was written by Emily Dickinson",
  "answer": "2"
}

Question: Who wrote 'Crime and Punishment'?
Expected 
Fyodor Dostoevsky
Provided 
{
  "reasoning": "The novel 'Crime and Punishment' by Fyodor Dostoevsky was published in 1866.",
  "answer": "1. Fyodor Dostoevsky"
}

Question: Who wrote 'One Hundred Years of Solitude'?
Expected 
Gabriel García Márquez
Provided 
{
  "reasoning": "The book 'One Hundred Years of Solitude' was written by Gabriel García Márquez.",
  "answer": "3"
}

Question: Who wrote 'The Trial'?
Expected 
Franz Kafka
Provided 
{
  "reasoning": "The Trial was written by Max Brod.",
  "answer": "1. Max Brod"
}

Question: Who wrote 'The Stranger'?
Expected 
Albert Camus
Provided 
{
  "reasoning": "The Stranger was written by Albert Camus.",
  "answer": "2"
}

Question: Who wrote '1984'?
Expected 
George Orwell
Provided 
{
  "reasoning": "George Orwell wrote '1984'.",
  "answer": "3"
}

Question: Who wrote 'Invisible Man'?
Expected 
Ralph Ellison
Provided 
{
  "reasoning": "The novel 'Invisible Man' was written by James Baldwin.",
  "answer": "1. James Baldwin"
}

Question: Who wrote 'Frankenstein'?
Expected 
Mary Shelley
Provided 
{
  "reasoning": "The correct answer is 3. Mary Shelley",
  "answer": "Mary Shelley"
}

Question: Who wrote 'Pride and Prejudice'?
Expected 
Jane Austen
Provided 
{
  "reasoning": "Jane Austen wrote 'Pride and Prejudice'",
  "answer": "1. Jane Austen"
}

Question: Who wrote 'Moby-Dick'?
Expected 
Herman Melville
Provided 
{
  "reasoning": "The correct answer is 1. Nathaniel Hawthorne",
  "answer": "Nathaniel Hawthorne"
}

Question: Who wrote 'The Brothers Karamazov'?
Expected 
Fyodor Dostoevsky
Provided 
{
  "reasoning": "The novel 'The Brothers Karamazov' was written by Fyodor Dostoevsky.",
  "answer": "1. Fyodor Dostoevsky"
}

Question: Who wrote 'Ulysses'?
Expected 
James Joyce
Provided 
{
  "reasoning": "James Joyce wrote 'Ulysses'",
  "answer": "1. James Joyce"
}

Question: Who wrote 'Beloved'?
Expected 
Toni Morrison
Provided 
{
  "reasoning": "Alice Walker wrote 'Beloved'.",
  "answer": "1. Alice Walker"
}

Question: Who wrote 'Brave New World'?
Expected 
Aldous Huxley
Provided 
{
  "reasoning": "The correct option is 2. George Orwell",
  "answer": "George Orwell"
}

Question: Who wrote 'The Catcher in the Rye'?
Expected 
J.D. Salinger
Provided 
{
  "reasoning": "The correct answer is 1. Jerome D. Salinger (pen name variant)",
  "answer": "Jerome D. Salinger"
}

Question: Who wrote 'The Hobbit'?
Expected 
J.R.R. Tolkien
Provided 
{
  "reasoning": "J.R.R. Tolkien wrote 'The Hobbit'.",
  "answer": "2"
}