Running on gpu

[danneauxs]

Duration: 44:43
Chunk generation speed: 26.4x realtime
End-to-end speed: 24.3x realtime
Chunks: 651/651
End-to-end processing time: 1:50

been playing around with putting this on GPU, running with five workers on my 8 gigabyte 4060 TI GPU.

Chunk generation speed is a timer that is started from the very first transformation So as soon as t3 starts tokenizing the timer starts and when s3 jen writes the final wav chunk it stops. It's more a RAW time.

end-to-end includes the initial text processing and chunking and concatenation into a single audio file.

Adding in ASR and conversion to M4B obviously increases end-to-end processing time.

Just wanted to put this out there that it's possible. And audio sounds great.

[danneauxs]

Duration: 44:42
Chunk generation speed: 39.8x realtime
End-to-end speed: 35.4x realtime
Chunks: 651/651
End-to-end processing time: 1:15

tweaked one setting in my config and now we're up to this.

Nope, more tweaks and a larger input file about 160k file
generated 8532.78s audio in 180.91s (chunks: 52.42x, end-to-end: 47.16x realtime)

[danneauxs]

Ran ASR on output no failures. Going to listen to the 3hour output to see if any noise hallucinations, but sopt checking 40/651 chunks showed no problems.