Description
I'm trying to manipulate the audio samples that come from the client and return back the altered stream from the server. My thought would be: create a new local stream from RTCAudioSource/createTrack() and a sink from RTCAudioSink using the original audio stream, get the samples from the original stream using the sink, manipulate them, push them into the new stream using .onData and then replace the original stream with my new stream. Using the audio-video-loopback as a start point I've got this:
`"use strict";
const { RTCAudioSink } = require("wrtc").nonstandard;
const { RTCAudioSource } = require("wrtc").nonstandard;
function beforeOffer(peerConnection) {
const audioTransceiver = peerConnection.addTransceiver("audio");
const videoTransceiver = peerConnection.addTransceiver("video");
const { track } = audioTransceiver.receiver;
const source = new RTCAudioSource();
const newTrack = source.createTrack();
const sink = new RTCAudioSink(track);
const sampleRate = 48000;
const samples = new Int16Array(sampleRate / 100); // 10 ms of 16-bit mono audio
const dataObj = {
samples,
sampleRate,
};
const interval = setInterval(() => {
// Update audioData in some way before sending.
source.onData(dataObj);
audioTransceiver.sender.replaceTrack(newTrack);
});
sink.ondata = (data) => {
// Do something with the received audio samples.
const newArr = data.samples.map((el) => el * 2);
dataObj[samples] = newArr;
};
return Promise.all([
audioTransceiver.sender.replaceTrack(track),
videoTransceiver.sender.replaceTrack(videoTransceiver.receiver.track),
]);
}
module.exports = { beforeOffer };`
Now this doesn't work, also tried returning the new stream in the replaceTrack return promise and I get no audio. I would prefer to just work with the original stream and edit that but I can't see any way of updating it without having to create a new track and using the onData method.
Any ideas?