-
Notifications
You must be signed in to change notification settings - Fork 586
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: added support for A21 and Amazon Titan models via bedrock api #1101
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Yomesh Shah <[email protected]>
Signed-off-by: Yomesh Shah <[email protected]>
Signed-off-by: Yomesh Shah <[email protected]>
@AlexsJones did you get a chance to review this for the changes and if it is ok to merge now ? |
I need to test it first, will try to get around to it ASAP |
@AlexsJones just adding some screenshots of the runs if that helps. |
If you're looking for a workaround, would suggest using LiteLLM to proxy Bedrock |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good start! i left some comments
case ModelA21J2UltraV1, ModelA21J2JumboInstruct: | ||
request = map[string]interface{}{ | ||
"prompt": prompt, | ||
"maxTokens": 2048, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this constant? why don`t we make it config in k8sgpt.yaml
case ModelAnthropicClaudeV2, ModelAnthropicClaudeV1, ModelAnthropicClaudeInstantV1: | ||
type InvokeModelResponseBody struct { | ||
Completion string `json:"completion"` | ||
Stop_reason string `json:"stop_reason"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't it be StopReason?
} | ||
return output.Completion, nil | ||
case ModelA21J2UltraV1, ModelA21J2JumboInstruct: | ||
type Data struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it good to define these types in case??
I suggest to put these out of case.
Closes #1076
📑 Description
Included support for A21 & Amazon Titan Model through the bedrock api, using a case structure to enable future flexibility to continue adding other models available. As Bedrock supports quite a few models not adding them all in a single merge.
✅ Checks
ℹ Additional Information