Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IPEX-LLM section (for local model deployment on Intel CPU/GPU) to tutorial "Connecting with Different Models" #107

Closed
wants to merge 7 commits into from

Conversation

shane-huang
Copy link

@shane-huang shane-huang commented Jun 12, 2024

This PR adds IPEX-LLM section as a new local model deployment option to the tutorial Connecting with Different Models.

IPEX-LLM is a library which allows users to run LLMs locally on Intel CPU and GPU with very low latency. With IPEX-LLM, user can leverage even the integrated GPU of low-cost PCs to run LLMs with a smooth experience.

dify can successfully connect to local LLMs deployed with IPEX-LLM, and there's a detailed quickstart guide for how to use dify with IPEX-LLM with a demo video at the IPEX-LLM docs website (see Run Dify with IPEX-LLM QuickStart Guide ).

It's the first time I contribute to dify docs. Any suggestions on styles or contents, please kindly let me know.

@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. documentation Improvements or additions to documentation enhancement New feature or request labels Jun 12, 2024
@shane-huang shane-huang changed the title add IPEX-LLM section (for local model deployment on Intel CPU/GPU) to tutorial "Connecting with Different Models" Add IPEX-LLM section (for local model deployment on Intel CPU/GPU) to tutorial "Connecting with Different Models" Jun 12, 2024
@crazywoola
Copy link
Member

Hello, sorry for the late response.

I read this doc

Regarding the deployment of dify itself, I notice it will start a dev environment instead of production environment. I suggest replacing this part to use docker ways which should have better performances for both frontend and backend part. And there are some conflicts as well.

@crazywoola crazywoola self-assigned this Jul 1, 2024
@crazywoola crazywoola self-requested a review July 1, 2024 01:46
Copy link
Member

@crazywoola crazywoola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments

@crazywoola
Copy link
Member

Hello this pr has conflicts that must be resolved, please use the latest main branch to reopen an pr. Thanks :)

@crazywoola crazywoola closed this Jul 3, 2024
@shane-huang
Copy link
Author

We will follow-up on your comments and reopen a PR based on latest main branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request size:S This PR changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants