-
-
Notifications
You must be signed in to change notification settings - Fork 589
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NodeJS Local Files Only - Headers Not Defined & Incorrect Path Splitters #520
Comments
Hi there 👋 Transformers.js requires Node.js v18+ to function correctly. Since Node 16 has reached EOL, we will not be adding support for it in future. See here for more information. |
Hello! - Thanks for the quick response! :) I upgrade to
|
I was able to fix this with the following code:
I think changing the semantics of this may benefit more diverse projects, I am building an Electron app for users to point to and use models wherever they may be on their computer. If I start setting Happy to make the changes myself, please let me know your thoughts! |
@xenova - bump! Let me know and I will start a PR for this |
I have the same issue too, I dont know why my model is there but it cannot file that path: this is not working too
but it actually there @axrati I think you should create an PR for this |
@hiepxanh Can you provide sample code and your directory structure so I can use as test case as well as mine? |
@xenova - the change here isn't major, and doesnt supply full vs relative path. Its an issue with how |
@xenova Bump! |
Hi there 👋 Feel free to fork the repository, then submit a pull request. In that way, I can review your changes. |
the same issue to me! |
@axrati thanks for you contribute,realy need you PR. |
@xenova - reminder for this PR! if you can approve the checks to run it'd help ~ |
same issue, any update now? |
System Info
Windows 10 -
10.0.19045 Build 19045
Alienware m17 R3
CPU -
Intel i7-10750H
Node version:
v16.14.2
main.mjs:
Package.json:
Clone this repository directly into root of project:
https://huggingface.co/Supabase/gte-small
Your final project outlook will look like this:
Environment/Platform
Description
When trying to import models locally, it looks like there are still HTTP requests trying to be fired off. Expected behavior is that when
local_files_only
is true, that it would only try to use local files.Secondarily, it looks like the paths to load assets is incorrect on a Windows computer. It is using
/
instead of\
for transformer assets. It also doesnt seem to be respecting relative path vs absolute path... perhaps that needs to be changed as well?Error output:
As you can see, the "Unable to read local path" is trying to reference
node_modules\@xenova\transformers\models\/gte-small/tokenizer.json
, which wouldn't be valid Windows path... That said, it looks to not be respecting the relative path (if you see the System Requirements section, you can see the model is a directory in the root of the project, and this is searching through your library innode_modules
)If you look at your code in
https://github.com/xenova/transformers.js/blob/main/src/utils/hub.js
, you can see on lines 55-56 that the constructor for a FileResponse is instantiating Headers. This leads me to believe that even if thegetFile
function had its first 2 criteria met (env.useFS && !isValidHttpUrl(urlOrPath))
), that its still executing unnecessary code for the protocol its trying to use.I am happy to help create a PR for this! Please reach out and let me know. Would be helpful to catch up with someone on the team for repo direction/etc.
Reproduction
Based on steps in Sys Reqs / Description
npm install
node main.mjs
The text was updated successfully, but these errors were encountered: