Uncategorized

An Emotional Intelligence Benchmark for Large Language Models



Download a PDF of the paper titled EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models, by Samuel J. Paech

Download PDF
HTML (experimental)

Abstract:We introduce EQ-Bench, a novel benchmark designed to evaluate aspects of emotional intelligence in Large Language Models (LLMs). We assess the ability of LLMs to understand complex emotions and social interactions by asking them to predict the intensity of emotional states of characters in a dialogue. The benchmark is able to discriminate effectively between a wide range of models. We find that EQ-Bench correlates strongly with comprehensive multi-domain benchmarks like MMLU (Hendrycks et al., 2020) (r=0.97), indicating that we may be capturing similar aspects of broad intelligence. Our benchmark produces highly repeatable results using a set of 60 English-language questions. We also provide open-source code for an automated benchmarking pipeline at this https URL and a leaderboard at this https URL

Submission history

From: Samuel Paech [view email]
[v1]
Mon, 11 Dec 2023 10:35:32 UTC (405 KB)
[v2]
Wed, 3 Jan 2024 12:20:35 UTC (405 KB)



Source link

Leave a Reply

Your email address will not be published. Required fields are marked *