This is the official GitHub repository for LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation.
LLM-as-an-Interviewer is an evaluation framework that assesses the capabilities of LLMs through an interview-style process. In this approach, the LLM acting as the interviewer evaluates other LLMs by providing feedback and asking follow-up questions, enabling a more comprehensive assessment of their capabilities.
Our framework includes a flexible pipeline that can be easily adapted to various tasks by incorporating a customized evaluation rubric.
Access the code used to replicate the results discussed in our paper. github
Explore the framework implementation. github
The framework is also available as a Python package on PyPI. Install it using:
pip install interview-eval