1.0 KiB
1.0 KiB
Keypoint Evaluator
This application evaluates a RAG (Retrieval-Augmented Generation) system based on the keypoint methodology from the RAGEval paper.
How to use
To run the evaluation, execute the following command from the root directory of the project:
python -m keypoint_eval.main --evaluation-name <EVALUATION_NAME> --matriz-eval <PATH_TO_EVALUATION_MATRIX_FILE>
Arguments
--evaluation-name: The name of the evaluation.--matriz-eval: The path to the evaluation matrix file.
The application will read the evaluation matrix from the specified file and will generate a CSV and a JSON file with the evaluation results.
Input File Structure
The input file can be a CSV, Excel, or JSON file.
The file must contain the following columns:
input: The user's question.expected_output: The ground truth or expected answer.category(optional): The category of the question.
If the input column is not found, the application will look for columns containing "pregunta" or "question".