Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add model params for temp and model name #6

Merged
merged 3 commits into from
May 10, 2023
Merged

Conversation

johnlk
Copy link
Contributor

@johnlk johnlk commented May 10, 2023

No description provided.

@johnlk johnlk self-assigned this May 10, 2023
@github-actions
Copy link

null

3 similar comments
@github-actions
Copy link

null

@github-actions
Copy link

null

@github-actions
Copy link

null

@github-actions
Copy link

First, let's start with the letter grade. Based on the provided code diff, I'd grade it as a B - there are a few improvements that can be made.

Here are some improvements that could be made:

  1. Give a more detailed description of the purpose of the code. While the README provides some context, it's not immediately clear what this code is meant to do, especially at first glance.

  2. Standardize how arguments are presented in the README and the action.yml file. The table in the README is missing some arguments, and the descriptions in action.yml aren't fully fleshed out.

  3. Reorganize the code so that the source statement is at the very top, above the main function. This is purely for the sake of readability.

  4. Make the gpt.sh file more modular, so that other scripts can rely on its functions without too much modification.

Here's an example of how the code could be reorganized:

#!/usr/bin/env bash

source "$HOME_DIR/src/gpt.sh"

## Auto-reviews a Pull Request
## 
## Usage:
##   main.sh --github_token=<token> --open_ai_api_key=<token> --gpt_model_name=<name> --gpt_model_temp=<temp> --github_api_url=<url>

main() {
  eval "$(/root/bin/docpars -h "$(grep "^##?" "$HOME_DIR/src/main.sh" | cut -c 5-)" : "$@")"

  utils::env_variable_exist "github_token"
  utils::env_variable_exist "github_api_url"
  utils::env_variable_exist "open_ai_api_key"
  utils::env_variable_exist "gpt_model_name"
  utils::env_variable_exist "gpt_model_temp"

  export GITHUB_TOKEN="$github_token"
  export GITHUB_API_URL="$github_api_url"
  export OPEN_AI_API_KEY="$open_ai_api_key"
  export GPT_MODEL="$gpt_model_name"
  export GPT_TEMP="$gpt_model_temp"

  echo "GPT_MODEL: $GPT_MODEL"
  echo "GPT_TEMP: $GPT_TEMP"

  local -r pr_number=$(utils::get_pr_number)
  local -r commit_diff=$(github::get_commit_diff "$pr_number")
  local -r gpt_response=$(gpt::prompt_model "$commit_diff")
  echo "$gpt_response"
  github::comment "$gpt_response" "$pr_number"

  exit $?
}

source "$HOME_DIR/src/gpt.sh"

main "$@"

Note: I left the utils::check_success function in gpt.sh as is, since it's unlikely that there would be another use case for the function outside of gpt.sh.

Let me know if you have any questions!

@github-actions
Copy link

null

Add a small modifier

Test updating the param value type

Create test logs

More testing

Last test

Drop the temp. param for now
@github-actions
Copy link

Letter grade: C

Possible improvements:

  • Add comments to the code to increase readability
  • Reduce the length of the INITIAL_PROMPT string in gpt.sh to make it more readable
  • Use better variable names in main.sh

Code example with the changes:

diff --git a/README.md b/README.md
index 46b94e1..00bb8fd 100644
--- a/README.md
+++ b/README.md
@@ -4,6 +4,15 @@
 
 Named after Batman's assistant, Robin AI is an open source Github project that automatically reviews Github pull requests, providing a letter grade from A to F, suggested improvements, and sample code for improvement. It is deployed as a Github action and requires two parameters: `GITHUB_TOKEN`, which is automatically supplied by Github for the user, and `OPEN_AI_API_KEY`, which is an API key from Open AI's developer portal.
 
+## Arguments
+
+| Name                | Required | Default Value             | Description                                                                                                       |
+|---------------------|----------|---------------------------|-------------------------------------------------------------------------------------------------------------------|
+| `GITHUB_TOKEN`      | Yes      | Automatically supplied    | A Github access token with the `repo` and `pull_request` scopes.                                                  |
+| `OPEN_AI_API_KEY`   | Yes      | N/A                       | An API key from Open AI's developer portal.                                                                       |
+| `gpt_model_name`    | No       | `gpt-3.5-turbo-0301`      | The name of the GPT model to use for text generation.                                                             |
+| `github_api_url`    | No       | `https://api.github.com`  | The URL for the Github API endpoint. (Only relevant to enterprise customers.)                                      |
+
 ## Installation
 
 To use Robin AI in your Github project, you'll need to add it as a Github action. Here's how:
diff --git a/action.yml b/action.yml
index ef334de..36ffab9 100644
--- a/action.yml
+++ b/action.yml
@@ -7,6 +7,10 @@ inputs:
   OPEN_AI_API_KEY:
     description: 'Open API token'
     required: true
+  gpt_model_name:
+    description: 'The OpenAI used to generate the chat completion'
+    required: false
+    default: 'gpt-3.5-turbo-0301'
   github_api_url:
     description: 'URL to the API of your Github Server, only necessary for Github Enterprise customers'
     required: false
@@ -17,6 +21,8 @@ runs:
   args:
     - --github_token=${{ inputs.GITHUB_TOKEN }}
     - --open_ai_api_key=${{ inputs.OPEN_AI_API_KEY }}
+    - --gpt_model_name=${{ inputs.gpt_model_name }}
+    - --gpt_model_temp=${{ inputs.gpt_model_temp }}
     - --github_api_url=${{ inputs.github_api_url }}
 branding:
   icon: 'tag'
diff --git a/src/gpt.sh b/src/gpt.sh
index 67b042f..1350c1f 100755
--- a/src/gpt.sh
+++ b/src/gpt.sh
@@ -1,7 +1,5 @@
 #!/usr/bin/env bash
 
-INITIAL_PROMPT="I'm going to assign you a role. Your role is being a pull request code reviewer on our engineering team. \
-  As such we need you to respond with some constructive feedback on our code. Your main contribution to the team is providing \
-  crisp constructive feedback on how we can improve our code's quality, maintainability, and readability to name a few. The \
+INITIAL_PROMPT="You have been assigned the role of pull request code reviewer on our engineering team. Your contribution to the team is to provide constructive feedback on how we can improve our code's quality, maintainability, and readability. The \
   code will come as a git diff." 
 
source "$HOME_DIR/src/gpt.sh"
@@ -16,10 +16,12 @@ main() {
   utils::env_variable_exist "github_token"
   utils::env_variable_exist "github_api_url"
   utils::env_variable_exist "open_ai_api_key"
+  utils::env_variable_exist "gpt_model_name"
 
   export GITHUB_TOKEN="$github_token"
   export GITHUB_API_URL="$github_api_url"
   export OPEN_AI_API_KEY="$open_ai_api_key"
+  export GPT_MODEL="$gpt_model_name"
 
   local -r pr_number=$(utils::get_pr_number)
   local -r commit_diff=$(github::get_commit_diff "$pr_number")   

@github-actions
Copy link

Letter Grade: C

Improvements:

  • Remove the hardcoded GPT model name from src/gpt.sh script, instead use the value from gpt_model_name input.
  • Better formatting of message text in src/gpt.sh script to improve readability.

Code Block:

# Updated `src/gpt.sh` to use `GPT_MODEL` environment variable
GPT_MODEL=${GPT_MODEL:-gpt-3.5-turbo-0301}

INITIAL_PROMPT=$(cat <<EOF
I'm going to assign you a role. Your role is being a pull request code reviewer on our engineering team. \
As such we need you to respond with some constructive feedback on our code. Your main contribution to the team is providing \
crisp constructive feedback on how we can improve our code's quality, maintainability, and readability to name a few. The \
EOF
)

@johnlk johnlk merged commit 202f440 into main May 10, 2023
1 check passed
@johnlk johnlk deleted the john/add-model-params branch May 10, 2023 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant