Increasing speed of Speaker Diarization pipeline with CPU

[Vanargh]

I am a huge fan of your work with pyannote and I greatly admire the idea as well as the work placed behind creating & maintaining this repository.

The issue is concerned with speed. Even though the speaker diarization pipeline obtained from

pipeline = torch.hub.load('pyannote/pyannote-audio', 'dia')

when applied over audio files, provides extremely great fine-grained segmentation; it is really slow especially when it is running on CPU, instead of a GPU, and consumes a lot of time for diarization tasks. Even after downloading the model into the local system, there is no noticeable difference in speed.

In some cases, real-time factor is between 0,5 and 1 with CPU. So it needs 30 to 60 minutes for a file of 1-hour duration. Are there any pre-trained pipelines that work nearly as well but are significantly faster? I would like to know why it is so slow even though the downloaded models seem rather small. And is there some way, we can make it work faster? Currently, I believe that the audio file is provided as raw wave form to the end-to-end pipeline. Is it possible to provide, mfcc or other features extracted from the audio file, directly to the pipeline to obtain results similar to those published in your paper and perhaps, making it faster?

[hbredin]

In some cases, real-time factor is between 0,5 and 1 with CPU. So it needs 30 to 60 minutes for a file of 1-hour duration. Are there any pre-trained pipelines that work nearly as well but are significantly faster?

No, there isn’t any.

I would like to know why it is so slow even though the downloaded models seem rather small. And is there some way, we can make it work faster?

You could increase the step to reduce the overlap between consecutive chunks in SAD and SCD step.

Currently, I believe that the audio file is provided as raw wave form to the end-to-end pipeline. Is it possible to provide, mfcc or other features extracted from the audio file, directly to the pipeline to obtain results similar to those published in your paper and perhaps, making it faster?

No. You would have to retrain a model for that.

[Vanargh]

Thank you for the answers @hbredin. Could you provide more inputs on how to increase the step size to reduce the overlap? Can this be easily incorporated within the existing pipeline and is there some readme/tutorials?

[asadullah797]

@Vanargh Did you get an answer.
I am also looking for speedup factors as my speaker Diarization inference is very slow even on 1 min audio.

[Massooma]

Hello, thank you for your admirable work.
I would like to know what is the average time for a diarization processing on an 1 hour audio please?

[prkumar112451]

a 10 min recording takes 30 seconds to diarize.
and generally I have found the mins of recording and processing time to be linearly dependent on each other.

so a 1 hr recording will take roughly 1.5 minutes to diarize.