Forced JSON mode on all models that support it.

[lukehollenback]

This is a fix for Issue #56, ultimately caused by invalid JSON output being produced by the model, by forcing JSON mode on all supported models.

Was previously only forced on gpt-4-1106-preview, but OpenAI supports JSON mode on the latest GPT-3 and GPT-4 turbo models. See https://platform.openai.com/docs/guides/text-generation/json-mode.

Put gpt-4-turbo-preview, gpt-4-turbo, and gpt-3.5-turbo first in the conditional for efficiency. Put gpt-4-turbo - a model that does not exist yet - in the conditional to future-proof once GPT-4 Turbo goes out of preview.

[villesau]

Oh this was a good catch! Where did you find the list? 🤔 Looks like most if not all the chat models are listed?

[lukehollenback]

It's a combination of information listed here, here, and here.

It is basically all of the current-version turbo models. The standard models (e.g., gpt-4) still do not support JSON mode.

[simonvea]

Could you add gpt-4o to the list as well? It supports JSON mode as per the documentation.

[rairulyle]

Would be better to have something like this so that it is much easier to maintain in the future:

const SUPPORTS_JSON_FORMAT = [
  "gpt-4o",
  "gpt-4-turbo-preview",
  "gpt-4-turbo",
  "gpt-3.5-turbo",
  "gpt-4-0125-preview",
  "gpt-4-1106-preview",
  "gpt-3.5-turbo-0125",
  "gpt-3.5-turbo-1106",
];

// .......ommited

const response = await openai.chat.completions.create({
      ...queryConfig,
      // return JSON if the model supports it:
      ...(SUPPORTS_JSON_FORMAT.includes(OPENAI_API_MODEL)
        ? { response_format: { type: "json_object" } }
        : {}),
      messages: [
        {
          role: "system",
          content: prompt,
        },
      ],
    });