Is ChatGPT4 vision supported yet?

[riekusr]

Maybe i am missing it, but is there an ability to upload an image/document and prompt on this via this package?

[allanbunch]

You bet. msg.payload.model set to "gpt-4-vision-preview" does the trick. Here's a sample flow:

[
    {
        "id": "046b6275deafc8f6",
        "type": "tab",
        "label": "Image Chat Flow",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "1dae86463534d172",
        "type": "debug",
        "z": "046b6275deafc8f6",
        "name": "response",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload.choices[0].message.content",
        "targetType": "msg",
        "statusVal": "",
        "statusType": "auto",
        "x": 540,
        "y": 80,
        "wires": []
    },
    {
        "id": "a0343bf65dbb5779",
        "type": "inject",
        "z": "046b6275deafc8f6",
        "name": "chat payload",
        "props": [
            {
                "p": "payload.model",
                "v": "gpt-4-vision-preview",
                "vt": "str"
            },
            {
                "p": "payload.max_tokens",
                "v": "1024",
                "vt": "num"
            },
            {
                "p": "payload.messages",
                "v": "[{\"role\":\"user\",\"content\":[{\"type\":\"text\",\"text\":\"Hi there! Do I have carrots in my fridge and if so, can you tell me where I can find them? Thanks!\"},{\"type\":\"image_url\",\"image_url\":{\"url\":\"https://upload.wikimedia.org/wikipedia/commons/thumb/7/7b/Open_refrigerator_with_food_at_night.jpg/1280px-Open_refrigerator_with_food_at_night.jpg\"}}]}]",
                "vt": "json"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "x": 130,
        "y": 80,
        "wires": [
            [
                "2e1e0e07b3457efe"
            ]
        ]
    },
    {
        "id": "575b0c362c660c79",
        "type": "comment",
        "z": "046b6275deafc8f6",
        "name": "Chat about an image.",
        "info": "",
        "x": 140,
        "y": 40,
        "wires": []
    },
    {
        "id": "2e1e0e07b3457efe",
        "type": "OpenAI API",
        "z": "046b6275deafc8f6",
        "name": "chat completion",
        "service": "6bf3be50f1f3a262",
        "method": "createChatCompletion",
        "x": 340,
        "y": 80,
        "wires": [
            [
                "1dae86463534d172"
            ]
        ]
    },
    {
        "id": "6bf3be50f1f3a262",
        "type": "Service Host",
        "apiBase": "https://api.openai.com/v1",
        "secureApiKeyHeaderOrQueryName": "Authorization",
        "name": "OpenAI Auth"
    }
]

[riekusr]

Thank you for your work allan, very appreciated!