Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uncaught TypeError in audio-processor.js #15

Open
dskill opened this issue Jan 30, 2024 · 14 comments
Open

Uncaught TypeError in audio-processor.js #15

dskill opened this issue Jan 30, 2024 · 14 comments

Comments

@dskill
Copy link

dskill commented Jan 30, 2024

Getting this. I'm trying to view the page on windows while running it in WSL. Everything else seems OK.

Uncaught TypeError: Cannot read properties of undefined (reading 'set')
at AudioStreamProcessor.process (audio-processor.js:24:23)
process @ audio-processor.js:24

@zoq
Copy link
Contributor

zoq commented Jan 30, 2024

What browser are you using? Any chance you can try with Chrome?

@dskill
Copy link
Author

dskill commented Jan 30, 2024

Thx for the quick response. Yes, I'm using chrome currently.

@jprovencher
Copy link

If that can help, I have the same issue, Win11 PRO; OS: Win32, Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36

@zoq
Copy link
Contributor

zoq commented Jan 31, 2024

For a quick fix you can apply - #17 (comment)

@dskill
Copy link
Author

dskill commented Jan 31, 2024

also since it was mentioned in the other thread, here is my server output in case helpful:


==========
== CUDA ==
==========

CUDA Version 12.2.2

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

done loading
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
/usr/local/lib/python3.10/dist-packages/torch/nn/utils/weight_norm.py:30: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
  warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
INFO:root:[LLM] loaded: True██████████████--------------------| 67.76% [103/152 00:00<00:00]
INFO:websockets.server:connection open████████████████████████| 100.00% [152/152 00:01<00:00]
INFO:websockets.server:connection open
downloading ONNX model...
loading session
loading onnx model
reset states
INFO:root:New client connected

@zoq
Copy link
Contributor

zoq commented Jan 31, 2024

The output looks good and the progress bar you see is the warmup stage of WhisperSpeech. Let me know if the workaround that I posted works for you. We are currently setting up a Windows machine to replicate the issue.

@dskill
Copy link
Author

dskill commented Jan 31, 2024

Here's a video with that workaround. It seems to be working, except I don't hear any text to speech (probably not surprising considering the bug i worked around). but still.. progress!

output.mp4

@zoq
Copy link
Contributor

zoq commented Jan 31, 2024

I can see in the console that the connection to 8888 failed, which is the websocket port that we use to send the audio. I guess that port is forwarded (docker container)? Also, it takes some time until the service started, it has a warmup phase, so depending on the hardware it can take like 30 seconds or more until the system is actually running.

@dskill
Copy link
Author

dskill commented Feb 1, 2024

Thanks! I went ahead and tried again, restarting everything. This time it did work. The audio was a bit more delayed than in your video, but it did all work. So presumably in my last test I just needed to either wait for a warm up or the 8888 port forwarding somehow didn't work.

@zoq
Copy link
Contributor

zoq commented Feb 1, 2024

What GPU do you use?

@dskill
Copy link
Author

dskill commented Feb 1, 2024 via email

@zoq
Copy link
Contributor

zoq commented Feb 1, 2024

We are going to add latency outputs for each step to the demo tomorrow, so we compare it easier. The video was recorded on a 4090, so you should see very similar results.

@dskill
Copy link
Author

dskill commented Feb 1, 2024 via email

@sadimoodi
Copy link

We are going to add latency outputs for each step to the demo tomorrow, so we compare it easier. The video was recorded on a 4090, so you should see very similar results.

i am using the same GPU on windows, did u use this command to run the docker:?
docker run --gpus all --shm-size 64G -p 6006:6006 -p 8888:8888 -it ghcr.io/collabora/whisperfusion-3090:latest
did u get everything running eventually?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

4 participants