-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* playwright tests for chatbot * more tests * changelog * Update CHANGELOG.md * fix upload file delay (#4661) * fix * changes * changes --------- Co-authored-by: Abubakar Abid <[email protected]> * Forward `tqdm` constructor arguments to `Progress` component (#4475) * Forward tqdm constructor arguments to Progress component Signed-off-by: Ilya Trushkin <[email protected]> * lint * reorder args * added tests --------- Signed-off-by: Ilya Trushkin <[email protected]> Co-authored-by: Abubakar Abid <[email protected]> * Remove cleared_value (#4685) * Remove cleared_value * CHANGELOG * Add requirements.txt to dialogpt demo (#4686) * Add requirements.txt * Update demo notebook * Add torch * remove streaming demo + more tests * Fix blocks_kitchen_sink and streaming_stt demos (#4699) * Add code * Add json file * Remove streaming_stt demo * Undo generate_notebooks * Add blocks_kitchen_sink * fix tests * Update js/app/test/chatbot_multimodal.spec.ts Co-authored-by: pngwn <[email protected]> * Update js/app/test/chatbot_multimodal.spec.ts Co-authored-by: pngwn <[email protected]> * Update js/app/test/chatbot_multimodal.spec.ts Co-authored-by: pngwn <[email protected]> * Update js/app/test/chatbot_multimodal.spec.ts Co-authored-by: pngwn <[email protected]> * Update js/app/test/chatbot_multimodal.spec.ts Co-authored-by: pngwn <[email protected]> * Update js/app/test/chatbot_multimodal.spec.ts Co-authored-by: pngwn <[email protected]> * Update js/app/test/chatbot_multimodal.spec.ts Co-authored-by: pngwn <[email protected]> * Update js/app/test/chatbot_multimodal.spec.ts Co-authored-by: pngwn <[email protected]> * Update js/app/test/chatbot_multimodal.spec.ts Co-authored-by: pngwn <[email protected]> * update notebook * remove debug * remove debug --------- Signed-off-by: Ilya Trushkin <[email protected]> Co-authored-by: aliabid94 <[email protected]> Co-authored-by: Abubakar Abid <[email protected]> Co-authored-by: Ilya Trushkin <[email protected]> Co-authored-by: Freddy Boulton <[email protected]> Co-authored-by: pngwn <[email protected]>
- Loading branch information
6 people
committed
Jun 28, 2023
1 parent
dd97ee9
commit bf26b5f
Showing
9 changed files
with
124 additions
and
39 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| {"cells": [{"cell_type": "markdown", "id": 302934307671667531413257853548643485645, "metadata": {}, "source": ["# Gradio Demo: chatbot_multimodal"]}, {"cell_type": "code", "execution_count": null, "id": 272996653310673477252411125948039410165, "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": 288918539441861185822528903084949547379, "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "\n", "\n", "def add_text(history, text):\n", " history = history + [(text, None)]\n", " return history, gr.update(value=\"\", interactive=False)\n", "\n", "\n", "def add_file(history, file):\n", " history = history + [((file.name,), None)]\n", " return history\n", "\n", "\n", "def bot(history):\n", " response = \"**That's cool!**\"\n", " history[-1][1] = response\n", " return history\n", "\n", "\n", "with gr.Blocks() as demo:\n", " chatbot = gr.Chatbot([], elem_id=\"chatbot\").style(height=750)\n", "\n", " with gr.Row():\n", " with gr.Column(scale=0.85):\n", " txt = gr.Textbox(\n", " show_label=False,\n", " placeholder=\"Enter text and press enter, or upload an image\",\n", " ).style(container=False)\n", " with gr.Column(scale=0.15, min_width=0):\n", " btn = gr.UploadButton(\"\ud83d\udcc1\", file_types=[\"image\", \"video\", \"audio\"])\n", "\n", " txt_msg = txt.submit(add_text, [chatbot, txt], [chatbot, txt], queue=False).then(\n", " bot, chatbot, chatbot\n", " )\n", " txt_msg.then(lambda: gr.update(interactive=True), None, [txt], queue=False)\n", " file_msg = btn.upload(add_file, [chatbot, btn], [chatbot], queue=False).then(\n", " bot, chatbot, chatbot\n", " )\n", "\n", "if __name__ == \"__main__\":\n", " demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5} | ||
| {"cells": [{"cell_type": "markdown", "id": 302934307671667531413257853548643485645, "metadata": {}, "source": ["# Gradio Demo: chatbot_multimodal"]}, {"cell_type": "code", "execution_count": null, "id": 272996653310673477252411125948039410165, "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": 288918539441861185822528903084949547379, "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "import random\n", "import time\n", "\n", "# Chatbot demo with multimodal input (text, markdown, LaTeX, code blocks, image, audio, & video). Plus shows support for streaming text.\n", "\n", "def add_text(history, text):\n", " history = history + [(text, None)]\n", " return history, gr.update(value=\"\", interactive=False)\n", "\n", "\n", "def add_file(history, file):\n", " history = history + [((file.name,), None)]\n", " return history\n", "\n", "\n", "def bot(history):\n", " response = \"**That's cool!**\"\n", " history[-1][1] = \"\"\n", " for character in response:\n", " history[-1][1] += character\n", " time.sleep(0.05)\n", " yield history\n", "\n", "\n", "with gr.Blocks() as demo:\n", " chatbot = gr.Chatbot([], elem_id=\"chatbot\").style(height=750)\n", "\n", " with gr.Row():\n", " with gr.Column(scale=0.85):\n", " txt = gr.Textbox(\n", " show_label=False,\n", " placeholder=\"Enter text and press enter, or upload an image\",\n", " ).style(container=False)\n", " with gr.Column(scale=0.15, min_width=0):\n", " btn = gr.UploadButton(\"\ud83d\udcc1\", file_types=[\"image\", \"video\", \"audio\"])\n", "\n", " txt_msg = txt.submit(add_text, [chatbot, txt], [chatbot, txt], queue=False).then(\n", " bot, chatbot, chatbot\n", " )\n", " txt_msg.then(lambda: gr.update(interactive=True), None, [txt], queue=False)\n", " file_msg = btn.upload(add_file, [chatbot, btn], [chatbot], queue=False).then(\n", " bot, chatbot, chatbot\n", " )\n", "\n", "demo.queue()\n", "if __name__ == \"__main__\":\n", " demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| import { test, expect } from "@gradio/tootils"; | ||
|
|
||
| test("text input by a user should be shown in the chatbot as a paragraph", async ({ page }) => { | ||
| const textbox = await page.getByTestId('textbox'); | ||
| await textbox.fill("Lorem ipsum"); | ||
| await page.keyboard.press('Enter'); | ||
| const user_message = await page.getByTestId('user').first().getByRole('paragraph').textContent(); | ||
| const bot_message = await page.getByTestId('bot').first().getByRole('paragraph').textContent(); | ||
| await expect(user_message).toEqual("Lorem ipsum"); | ||
| await expect(bot_message).toBeTruthy(); | ||
| }); | ||
|
|
||
| test("images uploaded by a user should be shown in the chat", async ({ page }) => { | ||
| const fileChooserPromise = page.waitForEvent('filechooser'); | ||
| await page.getByRole('button', { name: '📁' }).click(); | ||
| const fileChooser = await fileChooserPromise; | ||
| await fileChooser.setFiles("./test/files/cheetah1.jpg"); | ||
| await page.keyboard.press('Enter'); | ||
|
|
||
| const user_message = await page.getByTestId('user').first().getByRole('img'); | ||
| const bot_message = await page.getByTestId('bot').first().getByRole('paragraph').textContent(); | ||
| const image_data = await user_message.getAttribute("src"); | ||
| await expect(image_data).toContain("cheetah1.jpg"); | ||
| await expect(bot_message).toBeTruthy(); | ||
| }); | ||
|
|
||
| test("audio uploaded by a user should be shown in the chatbot", async ({ page }) => { | ||
| const fileChooserPromise = page.waitForEvent('filechooser'); | ||
| await page.getByRole('button', { name: '📁' }).click(); | ||
| const fileChooser = await fileChooserPromise; | ||
| await fileChooser.setFiles("../../test/test_files/audio_sample.wav"); | ||
| await page.keyboard.press('Enter'); | ||
|
|
||
| const user_message = await page.getByTestId('user').first().locator('audio'); | ||
| const bot_message = await page.getByTestId('bot').first().getByRole('paragraph').textContent(); | ||
| const audio_data = await user_message.getAttribute("src"); | ||
| await expect(audio_data).toContain("audio_sample.wav"); | ||
| await expect(bot_message).toBeTruthy(); | ||
|
|
||
| }); | ||
|
|
||
| test("videos uploaded by a user should be shown in the chatbot", async ({ page }) => { | ||
| const fileChooserPromise = page.waitForEvent('filechooser'); | ||
| await page.getByRole('button', { name: '📁' }).click(); | ||
| const fileChooser = await fileChooserPromise; | ||
| await fileChooser.setFiles("../../test/test_files/video_sample.mp4"); | ||
| await page.keyboard.press('Enter'); | ||
|
|
||
| const user_message = await page.getByTestId('user').first().locator('video'); | ||
| const bot_message = await page.getByTestId('bot').first().getByRole('paragraph').textContent(); | ||
| const video_data = await user_message.getAttribute("src"); | ||
| await expect(video_data).toContain("video_sample.mp4"); | ||
| await expect(bot_message).toBeTruthy(); | ||
| }); | ||
|
|
||
|
|
||
| test("markdown input by a user should be correctly formatted: bold, italics, links", async ({ page }) => { | ||
| const textbox = await page.getByTestId('textbox'); | ||
| await textbox.fill("This is **bold text**. This is *italic text*. This is a [link](https://gradio.app)."); | ||
| await page.keyboard.press('Enter'); | ||
| const user_message = await page.getByTestId('user').first().getByRole('paragraph').innerHTML(); | ||
| const bot_message = await page.getByTestId('bot').first().getByRole('paragraph').textContent(); | ||
| await expect(user_message).toEqual('This is <strong>bold text</strong>. This is <em>italic text</em>. This is a <a href=\"https://gradio.app\" target=\"_blank\" rel=\"noopener noreferrer\">link</a>.'); | ||
| await expect(bot_message).toBeTruthy(); | ||
| }); | ||
|
|
||
| test("inline code markdown input by the user should be correctly formatted", async ({ page }) => { | ||
| const textbox = await page.getByTestId('textbox'); | ||
| await textbox.fill("This is `code`."); | ||
| await page.keyboard.press('Enter'); | ||
| const user_message = await page.getByTestId('user').first().getByRole('paragraph').innerHTML(); | ||
| const bot_message = await page.getByTestId('bot').first().getByRole('paragraph').textContent(); | ||
| await expect(user_message).toEqual("This is <code>code</code>."); | ||
| await expect(bot_message).toBeTruthy(); | ||
| }); | ||
|
|
||
| test("markdown code blocks input by a user should be rendered correctly with the correct language tag", async ({ page }) => { | ||
| const textbox = await page.getByTestId('textbox'); | ||
| await textbox.fill("```python\nprint('Hello')\nprint('World!')\n```"); | ||
| await page.keyboard.press('Enter'); | ||
| const user_message = await page.getByTestId('user').first().locator('pre').innerHTML(); | ||
| const bot_message = await page.getByTestId('bot').first().getByRole('paragraph').textContent(); | ||
| await expect(user_message).toContain("language-python"); | ||
| await expect(bot_message).toBeTruthy(); | ||
|
|
||
| }); | ||
|
|
||
| test("LaTeX input by a user should be rendered correctly", async ({ page }) => { | ||
| const textbox = await page.getByTestId('textbox'); | ||
| await textbox.fill("This is LaTeX $$x^2$$"); | ||
| await page.keyboard.press('Enter'); | ||
| const user_message = await page.getByTestId('user').first().getByRole('paragraph').innerHTML(); | ||
| const bot_message = await page.getByTestId('bot').first().getByRole('paragraph').textContent(); | ||
| await expect(user_message).toContain("katex-display"); | ||
| await expect(bot_message).toBeTruthy(); | ||
| }); | ||
|
|
||
|
|
||
| test("when a new message is sent the chatbot should scroll to the latest message", async ({ page }) => { | ||
| const textbox = await page.getByTestId('textbox'); | ||
| const line_break = "<br>" | ||
| await textbox.fill(line_break.repeat(30)); | ||
| await page.keyboard.press('Enter'); | ||
| const bot_message = await page.getByTestId('bot').first().getByRole('paragraph'); | ||
| await expect(bot_message).toBeVisible(); | ||
| const bot_message_text = bot_message.textContent(); | ||
| await expect(bot_message_text).toBeTruthy(); | ||
| }); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters