Skip to content

Commit

Permalink
Merge pull request #25 from stanford-oval/Yucheng-Jiang-patch-add-pro…
Browse files Browse the repository at this point in the history
…metheus-eval-prompt

Create eval_prometheus_no_ref.prompt
  • Loading branch information
Yucheng-Jiang committed Apr 22, 2024
2 parents fdb111f + 3e08ca2 commit 195aa99
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions eval/eval_prometheus_no_ref.prompt
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
###Task Description:
An instruction (might include an Input inside it), a response to evaluate, a reference answer that gets a score of 5, and a score rubric representing a evaluation criteria are given.
1. Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general.
2. After writing a feedback, write a score that is an integer between 1 and 5. You should refer to the score rubric.
3. The output format should look as follows: \"Feedback: (write a feedback for criteria) [RESULT] (an integer number between 1 and 5)\".
4. Please do not generate any other opening, closing, and explanations.

###The instruction to evaluate:
{instruction}

###Response to evaluate:
{response}

###Score Rubrics:
[{criteria_description}]
Score 1: {score1_description}
Score 2: {score2_description}
Score 3: {score3_description}
Score 4: {score4_description}
Score 5: {score5_description}

###Feedback:

0 comments on commit 195aa99

Please sign in to comment.