Bandwidth Estimator

[Sean-Der]

The issue covers everything we want to do around Congestion Estimation and Pion. This covers a Bandwidth Estimator in Pion, and generating feedback for remote peers.

@mengelbart is the owner of this work. If you have feedback/suggestions/ideas he knows this all the best!

---

• bc30165 Implement Sender/Receiver Reports
• c8a26a2 Implement Transport Wide Congestion Control Feedback
• Design a interface/API that all Bandwidth Estimators will satify
• Merge the NetworkConditoner so we can test our implementations
  • Start with basic tests (hard limits on total bandwidth/start queueing at certain bitrate]
  • Implement full test suite from RFC8867
• Implement Google Congestion Control
  • Have it be Sender estimates only using TWCC. No REMB our Receiver Reports.
• Implement FlexFEC
  • This will be needed by servers attempting to deliver Simulcast/SVC
• Make WebRTC Bandwidth Estimation accessiable to others
  • Write blog posts about what you are implementing/challenges
  • Write a dedicated chapter in WebRTC for the Curious about Bandwidth Estimation
• Make work accessible for other projects.

In the future we will also need

• Switch from Transport Wide Congestion Control to cc-feedback-message
  • This hasn't been adopted by anyone yet
• Investigate AdaptiveFEC
  • Paper

---

• Mozilla article on RMCAT
• Transport Wide Congestion Control
• Google Congestion Control
• RMCAT
• NADA
• GstConf Video on Transport Wide Congestion Control
• ion-sfu Transport Wide Congestion Control

[jech]

Accurate SR/RR is the most urgent bit — without that, nothing works correctly. SR are essential for correct lipsynch, and GCC will yield acceptable performance on non-bufferbloated networks with just correct RR.

Let me expand on that. GCC consists of two congestion controllers:

• a fairly traditional loss-based controller, that reacts to the loss rate communicated in RR;
• a rather novel delay-based controller, that reacts to the second derivative of the packet arrival time.

The loss-based controller is trivial to implement, it works reasonably well, except that in the absence of a low-delay AQM at the bottleneck router it tends to cause latency proportional to the amount of buffering. The delay-based controller reacts earlier than the loss-based one, thus limiting latency, but it is underspecified and difficult to implement.

Here is Galène's implementation of the loss-based controller:

https://github.com/jech/galene/blob/master/rtpconn/rtpconn.go#L962

Here's Galène's implementation of SR and RR:

https://github.com/jech/galene/blob/master/rtpconn/rtpconn.go#L879
https://github.com/jech/galene/blob/master/rtpconn/rtpconn.go#L788

[Sean-Der]

Thank you so much @jech. I am going to move forward with everything until the TWCC. I am hoping I can make things good enough that you can simplify Galene.

When I start the delay based one I will send emails to the IETF list and see if I can get clarifications/help!

[aler9]

Sender and Receiver Report Generator

I've updated the PR with sender and receiver reports:

• reordered packets are now supported
• the incomplete bandwidth estimation part has been removed

Google Congestion Control Interceptor that uses Sender Reports - This is an optional interceptor that the user opts into. It then provides a callback with estimated bandwidth.

The estimated bandwidth is provided by the RTCP packet with payload 206, as confirmed in the paper, which is ReceiverEstimatedMaximumBitrate in pion/rtcp.

I've always seen ReceiverEstimatedMaximumBitrate and SenderReport shipped together in the same UDP packet - how can they be shipped together if there are two separate interceptors, one for Sender Reports and the other for congestion control?

What's the plan to make the two parts communicate with each other?

[dtaht]

I am just logging in on this bug so I can keep track.

[aler9]

Ok, since the SR/RR part has been merged, the bandwidth estimation part is the next one... but i repeat, in my opinion the bandwidth estimation must be a plugin of the RR interceptor, not a separate interceptor.

[Sean-Der]

@aler9 We couldn't not even worry about REMB if that makes things simpler! Looking at Chromium's RTCP parser I don't think they care. Mostly in rtcp_receiver.cc

We should start with a Receiver Report driven congestion controller. It reads the incoming Receiver Reports and adjusts accordingly. This would just be the loss based estimation of Google Congestion Control. We could then later add a TWCC driven congestion controller. That could be GCC/NADA/SCReAM.

If we do need to communicate between interceptors my plan was to use Attributes.

The only things we can't compromise with the CC design is

• User needs to be able to construct and choose which one they use
• User needs to get info in their code. My goal is to provide a example that looks like this

for estimate := range CongestionController.Estimate() {
  case high:
    //send frame from high.mp4
  case med:
    //send frame from med.mp4
   case low:
    // send frame from low.mp4
}

[dtaht]

My principal requests at this point would be:

ability to set and inspect the values of the ecn bits - and to be able to link an action to a change thereof
ability to set and inspect dscp also.

I REALLY want to gain the ability to put voice and video on different tuples also.

Scream and nada want to futz with ecn, and my gcc variant (unpublished) used the voice clock to calculate when a probe for more video bandwidth was succeeding or failing in a fq_codel environment. being able to set dscp meant that video calls could go into the wifi VI queue.

[Sean-Der]

@dtaht I had no idea DSCP/ECN would be needed. That is going to be somewhat of a pain because our ICE Agent only deals in bytes.

Putting audio/video on different tuples shouldn't be so bad! I just didn't do it before because we didn't have demand. I always perceived it as worse (because of extra port usage) so didn't worry about it. I would love to get it in though.

Have you ever seen any numbers about how much better Loss Based (GCC part 1) vs Latency Based (GCC part 1+2)? I do want to build everything eventually. I just don't want to invest in something with smaller returns then other Pion projects I would be ignoring.

Thank you so much for sharing all this knowledge. It is so hard to find people who knows this stuff and are willing to share.

[jech]

@dtaht what about hashing the SSID into the IPv6 flow-id? Would that yield the same benefits as using different ports?

[dtaht]

Sorry for the delay in reply. I need to put github notifications somewhere I read more regularly.

scream mandates ecn usage. worse, it presently requires l4s style markings, where I would vastly prefer to attempt SCE style.
an early version of nada did some intelligent things with ecn. I actually favor using ecn on the moral equivalent of iframes,
if there was a way for the encoder and congestion control mechanisms to communicate and some experimentation possible.

I really really really appreciate how clean the pion stuff is, compared to, say, trying to hack on the browsers or janus was. The browsers also make it really, really hard to get at the dscp/ecn headers, being single return (C, C++) based systems, whereas go can return multiple variables really cleanly.

All the gcc/scream/nada variants had REALLY slow probe times when I was last active in the rmcat working group circa 2012-2013. Since then "BBR" showed the way towards probing for more bandwidth and RTT separately which I think would be
interesting - my core idea was sending a ton of duplicate packets for a probe during a delta phase and seeing how many got through and observe rtt inflation before switching the encoder over to a higher rate. (I can point to a couple cool papers on this idea if I can remember the titles on google scholar This one was not horrible, but I can't find the one I'm thinking of

https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8737672)

You can't run out of udp ports with ipv6. There's a lot of ipv6 going around, especially in the lte world. Given that a given galene conference is under 100 people, and you can bind to as many addresses as you like in ipv6... as for ipv4, well, maybe not even try negotiating separate tuples for it, but I'd really like to try....

[dtaht]

while I'm citing papers, I adored @jech 's paper on mosh. https://arxiv.org/abs/1502.02402 - in my case I very much like the idea of an ipv6 address tunneled from elsewhere being an endpoint in webrtc. Short term this provides a degree of anyomnity not present in any webrtc implementation I know of, longer term might point to a way to proxying more users into a single conference.

[dtaht]

@jech I have NO idea what happens to flow labels on the path. None. It would be an interesting stat to collect. I'm under the impression that an early use case was for scribbling on it along the path as a load balancing technique, and have no idea if that still exists, but certainly the prospect of that crippled the flowlabel for any other purpose.

And I have no idea what you can accomplish by hashing the SSID onto it.

[dtaht]

@Sean-Der Going back to your workload... are you a funded entity? I have a few good connections that are looking for viable proposals, please contact me via email for more details.

In my case I cannot refute or confirm the claims of the scream folk without somehow getting access to the ecn bits. It's been 8 years of pointless debate vs a simulation, and the same goes for my own attempt at work here - and my dream has been to try and get something like this, located on a edge gateway, to work worth a damn, on fiber, at least, across town.

https://lola.conts.it/

Blasting raw video packets works. :) It's really hard to get gear that uses scanlines anymore, sad to say.

I've been at the jamaphone project for a really, really, really long time.

https://www.internetsociety.org/wp-content/uploads/2013/09/28_towards_imperceptible_latency.pdf

And see:

https://ccrma.stanford.edu/groups/soundwire/publications/papers/schuett_honorThesis2002.pdf