NEW: Orpheus Speech JavaScript library

[xenova]

This PR introduces the orpheus-speech JavaScript library (NPM), enabling natural speech synthesis 100% locally in your browser (or any other JS environment like Node, Deno, Electron, Bun, etc.). The library uses Transformers.js for inference, enabling WebGPU acceleration where possible. Since only the 3B model has been released, it may not immediately be accessible to most users (mainly due to out-of-memory errors), but we're releasing this in preparation for the upcoming smaller variants so that others can review it early!

I've done my best to align with the python API (taking some creative liberty to make it work with all JavaScript's quirks). I've also got a simple .play() function working which enables playback in-browser using the web audio API, although, I'll show how to use this better when I put out a full demo... something similar to this.

Example usage

import { OrpheusModel } from "orpheus-speech";

// Load the model
const model_id = "onnx-community/orpheus-3b-0.1-ft-ONNX";
const orpheus = await OrpheusModel({
  model_name: model_id,
  dtype: "q4f16",
  // device: "webgpu", // <-- If running in-browser, we recommend setting device to webgpu
});

// Create a new audio stream
const prompt = "Hey there my name is Tara, <chuckle> and I'm a speech generation model that can sound like a person.";
const voice = "tara";
const stream = orpheus.generate_speech({ prompt, voice });

// Iterate over the audio chunks
for await (const chunk of stream) {
  console.log(chunk.audio.length);
}

// Save the final audio to a file
const result = stream.data;
result.save("output.wav");

Sample output:

orpheus.mp4

[gianpaj]

Would you be able to deploy a sample website on Vercel/Netlify/etc. to try this? thanks

[kasapkiloet]

Loading the onnx model gives me the same error reported here:

RuntimeError: Aborted(). Build with -sASSERTIONS for more info.
at H (ort-wasm-simd-threaded.jsep.mjs:22:102)
at Db (ort-wasm-simd-threaded.jsep.mjs:63:98)
at ort-wasm-simd-threaded.jsep.wasm:0x13d067
at ort-wasm-simd-threaded.jsep.wasm:0x3d88a
at ort-wasm-simd-threaded.jsep.wasm:0x6679c
at ort-wasm-simd-threaded.jsep.wasm:0x10c03
at ort-wasm-simd-threaded.jsep.wasm:0xdc0292
at Oe.b. (ort-wasm-simd-threaded.jsep.mjs:80:119)
at Tc (ort-wasm-simd-threaded.jsep.mjs:119:227)
at ort-wasm-simd-threaded.jsep.wasm:0x1f6da

I got the error from a simple React bundle. Do we need to run it via node or web workers?

[xenova]

Since only the 3B model has been released, it may not immediately be accessible to most users (mainly due to out-of-memory errors),

Indeed, this is due to a memory error, meaning the 3B model is currently slightly too large to run in-browser. We do have some optimizations we can apply to try get this working, but to achieve real-time performance on more devices, I think a smaller model would be better!

[amuvarma13]

Thanks for this PR - this looks great - will look to merge shortly!

[Horschig]

Thanks for this PR - this looks great - will look to merge shortly!

keeping our hopes up for a release date of the smaller models, eh? :)