30 lines
1.0 KiB
Markdown
30 lines
1.0 KiB
Markdown
# Keypoint Evaluator
|
|
|
|
This application evaluates a RAG (Retrieval-Augmented Generation) system based on the keypoint methodology from the RAGEval paper.
|
|
|
|
## How to use
|
|
|
|
To run the evaluation, execute the following command from the root directory of the project:
|
|
|
|
```bash
|
|
python -m keypoint_eval.main --evaluation-name <EVALUATION_NAME> --matriz-eval <PATH_TO_EVALUATION_MATRIX_FILE>
|
|
```
|
|
|
|
### Arguments
|
|
|
|
* `--evaluation-name`: The name of the evaluation.
|
|
* `--matriz-eval`: The path to the evaluation matrix file.
|
|
|
|
The application will read the evaluation matrix from the specified file and will generate a CSV and a JSON file with the evaluation results.
|
|
|
|
## Input File Structure
|
|
|
|
The input file can be a CSV, Excel, or JSON file.
|
|
|
|
The file must contain the following columns:
|
|
|
|
* `input`: The user's question.
|
|
* `expected_output`: The ground truth or expected answer.
|
|
* `category` (optional): The category of the question.
|
|
|
|
If the `input` column is not found, the application will look for columns containing "pregunta" or "question". |