Benchmark Dashboard

Updated September 15, 2026 at 06:25
Benchmark All models Claude Haiku 4.5
GPT-5 mini
GPT-5 nano
GPT-5.4 mini
GPT-5.4 nano
GPT-5.6 Luna
Gemma 2 2B (LMStudio)
Gemma 2 9B (LMStudio)
Gemma 2B (LMStudio)
Gemma 3 12B (LMStudio)
Gemma 4 12B (LMStudio)
Gemma 4 E4B IT (LMStudio)
Granite 3.2 8B (LMStudio)
Llama 2 7B (LMStudio)
Llama 3 8B (LMStudio)
Llama 3.1 8B (LMStudio)
Llama 3.2 1B (LMStudio)
Ministral 8B (LMStudio)
OLMo 3 7B (LMStudio)
Phi-3.5 Mini (LMStudio)
Phi-4 (LMStudio)
Qwen3 1.7B (LMStudio)
Qwen3 4B (LMStudio)
Qwen3 VL 8B (LMStudio)
Qwen3.5 2B (LMStudio)
Qwen3.5 4B (LMStudio)
Qwen3.5 9B (LMStudio)
SmolLM2 1.7B (LMStudio)
Word Length 0011_word_length T1 ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠
Letter Count 0012_letter_count T1 ⚠ ⚠ ⚠ ⚠ ⚠
Vowel Count 0013_vowel_count T1 ⚠ ⚠ ⚠ ⚠ ⚠
Syllable Count 0014_syllable_count T1 ⚠ ⚠ ⚠ ⚠ ⚠
Spell Check 0015_spell_check T1 ⚠ ⚠ ⚠ ⚠ ⚠
Antonym Identification 0016_antonym T1 ⚠ ⚠ ⚠ ⚠ ⚠
Multilingual Synonym Generation 0017_synonyms T1 ⚠ ⚠ ⚠ ⚠ ⚠
Pinyin Letter Count 0018_pinyin_letters T1 ⚠ ⚠ ⚠ ⚠ ⚠
Simple Arithmetic 0021_simple_arithmetic T1 ⚠ ⚠ ⚠ ⚠ ⚠
Unit Conversion 0022_unit_conversion T1 ⚠ ⚠ ⚠ ⚠ ⚠
Math Word Problems 0023_word_problems T1 ⚠ ⚠ ⚠ ⚠ ⚠
Fractions and Percentages 0024_percentage_math T1 ⚠ ⚠ ⚠ ⚠
Algebra 0025_algebra T1 ⚠ ⚠ ⚠ ⚠ ⚠
Time Arithmetic 0026_time_arithmetic T1 ⚠ ⚠ ⚠ ⚠
Geometry 0027_geometry T1 ⚠ ⚠ ⚠ ⚠
Definitions 0031_definitions T2 – ⚠ ⚠ ⚠ ⚠ ⚠ – – – –
Part of Speech 0032_part_of_speech T2 ⚠ ⚠ ⚠ ⚠ – – –
English Plural Generation 0033_plural T2 ⚠ ⚠ ⚠ ⚠ – – –
Word to IPA 0061_word_to_ipa T3 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
Sentence Decomposition 0062_sentence_decomposition T3 – N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
Translation en_fr 0101_translation_en_fr T1 ⚠ ⚠ ⚠ ⚠ ⚠
Translation en_es 0102_translation_en_es T1 ⚠ ⚠ ⚠ ⚠
Translation en_de 0103_translation_en_de T1 ⚠ ⚠ ⚠ ⚠
Translation fr_es 0104_translation_fr_es T1 ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ⚠
Translation en_zh 0105_translation_en_zh T1 ⚠ ⚠ ⚠ ⚠
Translation en_ja 0106_translation_en_ja T1 ⚠ ⚠ ⚠ ⚠
Translation fr_ko 0107_translation_fr_ko T1 ⚠ ⚠ ⚠ ⚠
Translation it_lt 0108_translation_it_lt T1 ⚠ ⚠ ⚠ ⚠
Translation ja_lt 0109_translation_ja_lt T1 ⚠ ⚠ ⚠ ⚠
Verb Forms 0121_verb_forms T3 – N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
Lemma Identification 0122_lemma T3 – N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
Validate Lemma Form (lokys) 0130_validate_lemma_form T3 – N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
Validate Definition (lokys) 0131_validate_definition T3 – N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
Validate Translation (voras) 0132_validate_translation T3 – N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
Validate Bulk IPA/Phonetic (bebras) 0141_validate_pronunciation_bulk T3 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
Geography Knowledge 0151_geography T2 ⚠ ⚠ ⚠ ⚠ – – –
Syllogism Validity 0152_syllogism_validity T2 ⚠ ⚠ ⚠ ⚠ – – – ⚠ –
Book Author Match 0153_book_author_match T2 ⚠ ⚠ ⚠ ⚠ – – – –
Food Category Classification 0154_food_category_classification T2 ⚠ ⚠ ⚠ ⚠ ⚠ – – ⚠ –
Historical Event Year 0155_historical_event_year T2 ⚠ ⚠ ⚠ ⚠ ⚠ – – –
Python Hello World Function 0301_python_hello_world T2 ⚠ ⚠ ⚠ N/A – – –
Python GCD With Validation 0302_python_gcd T2 ⚠ ⚠ ⚠ ⚠ – – – –
Python Letter Count in String 0303_python_letter_count T2 – ⚠ ⚠ ⚠ ⚠ – – – –
Python Minimum Coin Change 0304_python_coin_change T2 – ⚠ ⚠ ⚠ ⚠ ⚠ – – – –
Python Prime Factorization 0305_python_prime_factorization T2 – ⚠ ⚠ ⚠ ⚠ – – – –