Summary

Motivation

Pion could be used in C/S topology, one end is a client and the other end is a server (just like Janus G/W does). But in this case, the server must have a routable (public) IP address. Many server environments may not have access to public IP address to bind to the local interface.

Janus provides the following option. Pion could provide something equivalent.

    -1, --nat-1-1=ip              Public IP to put in all host candidates,
                                  assuming a 1:1 NAT is in place (e.g., Amazon
                                  EC2 instances, default=none)

Describe alternatives you've considered

Use STUN server for the server-side nodes.

Description

This tries to better match the JSEP spec and will fix at least two issues: #1171 and #1178

• Move transceiver selection/creation in SetRemoteDescription and CreateOffer
• In SetRemoteDescription also create new Transceivers of type recvonly when no satisfying transceiver is available
• In CreateOffer generate unique mid in number format avoiding possible collisions with remote provide mids and to also already handle a future implementation of m= section rejection and reuse
• Now generateMatchedSDP will just find the transceivers with the required mid since they are already selected previously.

More details in these extract from JSEP:

JSEP 5.10 (Applying a Remote Description) says:

For each m= section, the following steps MUST be performed

[...]

If the m= section is not associated with any RtpTransceiver
(possibly because it was dissociated in the previous step),
either find an RtpTransceiver or create one according to the
following steps:

JSEP 5.2 (Constructing an Offer) says:

[...]

An m= section is generated for each RtpTransceiver
that has been added to the PeerConnection, excluding any stopped
RtpTransceivers;

Note that we are currently directly associating a mid to a transceiver in CreateOffer, instead the spec says to also keep a m= section index mapping to a transceiver and set the transceiver mid only when applying the local description. This is needed to support rollback of proposed offer/answer but currently we don't have support and tests for rollback situations.

Reference issue

Fixes #1171 Fixes #1178

Your environment.

• Version: v2.1.8
• Browser: None
• Other Information - I am trying to create a softphone using SIP and this library. The remote SIP & RTC service have no problem, because I have done a softphone using Node.js: https://github.com/ringcentral/ringcentral-softphone-js . I am trying to make a GoLang version, but I am stuck.

What did you do?

I have written the following code: https://github.com/ringcentral/ringcentral-softphone-go/blob/master/softphone.go#L137-L220

Just for your reference, here is the Node.js version of the GoLang code: https://github.com/ringcentral/ringcentral-softphone-js/blob/ce833c613d5e4bb959235e37addefd10cc9cb81e/index.js#L140-L177 . The Node.js code works like a charm.

What did you expect?

I expect that the peerConnection.OnTrack callback function will be invoked so that I can get the input audio

What happened?

peerConnection.OnTrack never invoked and I am stuck.

Some questions:

Could it because I specify the wrong codec? If so, I expect some error message like: no codec to handle the remote track. remote SDP sample:

v=0
o=- 5305405169010475891 908790132696250821 IN IP4 104.245.57.182
s=SmcSip
c=IN IP4 104.245.57.182
t=0 0
m=audio 20040 RTP/SAVPF 109 111 18 0 8 9 96 101
a=rtpmap:109 OPUS/16000
a=fmtp:109 useinbandfec=1
a=rtcp-fb:109 ccm tmmbr
a=rtpmap:111 OPUS/48000/2
a=fmtp:111 useinbandfec=1
a=rtcp-fb:111 ccm tmmbr
a=rtpmap:18 g729/8000
a=fmtp:18 annexb=no
a=rtpmap:0 pcmu/8000
a=rtpmap:8 pcma/8000
a=rtpmap:9 g722/8000
a=rtpmap:96 ilbc/8000
a=fmtp:96 mode=20
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=sendrecv
a=rtcp:20041
a=rtcp-mux
a=setup:actpass
a=fingerprint:sha-1 E2:F8:90:C6:37:7E:82:17:3C:63:CF:55:68:4E:1D:16:17:2C:D9:4D
a=ice-ufrag:atwbXWRm
a=ice-pwd:32SCWdSoU7BgkXxet5g5mlId5E
a=candidate:muEZ8Bb5Zhy8XFSL 1 UDP 2130706431 104.245.57.182 20040 typ host
a=candidate:muEZ8Bb5Zhy8XFSL 2 UDP 2130706430 104.245.57.182 20041 typ host

How can I port the Node.JS code to GoLang? Because I know Node.js version works like charm. And the code even works in browser (with browser WebRTC). So if pion/webrtc is compatible with browser WebRTC, I should be able to get this GoLang version work.

Could it becuase of this https://github.com/pion/webrtc/issues/879? I temporarily worked around the issue: https://github.com/ringcentral/ringcentral-softphone-go/blob/master/softphone.go#L133

Hi, we are trying to sending message using windows environment.

We have 1 server instance run on the Windows from NAT network and other one client instance run on the Windows from NAT network on other PC.

As the result we cant exhange messages and we cant see any errors.

Can you help as run this solution using Windows enviroment, because is it important for us?

If you need we can gve you as information you want.

Thanks.

PS we are using this example: https://github.com/pions/webrtc/tree/master/examples/pion-to-pion

How could I replace gstreamer with something more lightweight c lib

or pure go lib if I only want to work with audio (non-GUI app)?

Any ideas?

Thanks in advance :)

The latest working commit c87c3ca45386f2889887ac70633d71841793bc3c Current master doesn't establish connection at all, I think issue related because of latest ICE changes. Tested with chrome browser. Network map: client -> NAT -> server with real ip

Currently the media stream labels are hardcoded in the SDP offer and answer generation. The RTCPeerConnection.AddTrack should be extended to support adding media stream labels.

TODO

• [ ] Add a variadic streams argument to the AddTrack method as specified in the RTP Media API
• [ ] Store the media stream ids on the RTCTrack object as specified here
• [ ] Add the media streams to the SDP offer and answer. Some of the required logic already exists. However, the stream label is currently hardcoded to the track label. The procedure is specified in the JSEP spec

Your environment.

• Version: pionWebrtc v3.1.17, interceptor v0.1.5 and above
• Browser: Chrome newest version
• Other Information - Using simulcast feature with 16 video/audio tracks source

What did you do?

I'm building a livestreaming project and has a peer between chrome and pion (server), it was running by pm2. In my test case, i'm using ChromeA to peer and pushing video data to pionA, after that i forward data from pionA to pionB and pionB forward to ChromeB

PionA & PionB has only 1 simulcast peer, PionA is forwarding 16 video/audio data to PionB PionB is forwarding that data to ~100 ChromeB Clients.

In the img below, I checked my instance Mem between 2 version of pion interceptor, process running about 2mins ago. The Mem in v0.1.4 is stable and increase very low. But in v0.1.5 and above, it keep increasing too much and seem like never stop. It will get over ~30GB after 10mins

interceptor v0.1.4

interceptor v0.1.5 and above

What did you expect?

The Mem should like interceptor v0.1.4

What happened?

Memory keep increasing in simulcast.

Your environment.

• Version: v3.1.7
• Browser: Chrome
• Other Information - peerconnection.go

What did you do?

Send offer twice in very short time(within 100ms). In the first offer, I just add datachannel in SDP In the second offer, I add two tracks(audio, video)

But the second offer doesn't reflects well and see error log in PeerConnection as below Incoming unhandled RTP ssrc(123412341234), OnTrack will not be fired. incoming SSRC failed Simulcast probing

This issue is raised because REMOTE DESCRIPTION is changed during processing the first offer's startRTP(). (startSCTP() takes long time for init) So I think before completing startRTP() of the first offer, it should prevent to change REMOTE DESCRIPTION or filter the simulcastStream by undeclaredMediaProcessor().

And there is no way to check it startRTP() is in progress. Adding some function for checking it in PeerConnection might be helpful for user.

• I call PeerConnection's functions sequentially as below.

SetRemoteDescription(offer) -> answer, err := CreateAnswer(nil) -> SetLocalDescription(answer) 

What did you expect?

The second offer should reflect well.

What happened?

The second offer doesn't reflect well

Updated interceptor design based on TrackLocal:

Changes:

1. Interceptor now have Bind methods similar to track, which are called for every track (local/remote) for rtp, and once per peerconnection for rtcp.
2. Added interceptors separately to API, instead of under setting engine. Did this because interceptors must be created per peerconnection, and InterceptorFactory seemed confusing. Also API was already a depedency of Receiver/Sender, so this was was much simpler. Similarly to MediaEngine, Interceptors now must be configured for each PeerConnection and added to API. Default interceptors can later be added in NewAPI method.
3. Instead of using interfaces with single a single function (eg: RTPWriter with a single WriteRTP function), I use function types. This makes implementing an interceptor simpler, because there is no need to create a separate struct for each interface (inline functions can be used instead).

type WriteRTP func(p *rtp.Packet, attributes map[interface{}]interface{}) (int, error)
type ReadRTP func() (*rtp.Packet, map[interface{}]interface{}, error)
type WriteRTCP func(pkts []rtcp.Packet, attributes map[interface{}]interface{}) (int, error)
type ReadRTCP func() ([]rtcp.Packet, map[interface{}]interface{}, error)

type Interceptor interface {
	BindReadRTCP(read ReadRTCP) ReadRTCP     // TODO: call this
	BindWriteRTCP(write WriteRTCP) WriteRTCP // TODO: call this

	BindLocalTrack(ctx *TrackLocalContext, write WriteRTP) WriteRTP
	UnbindLocalTrack(ctx *TrackLocalContext)

	BindRemoteTrack(ctx *TrackRemoteContext, read ReadRTP) ReadRTP
	UnbindRemoteTrack(ctx *TrackRemoteContext)

	io.Closer
}

Questions:

1. Please see interceptor_peerconnection.go and interceptor_track_remote.go. They are quite ugly. I'm open to any suggestions.
2. ReadRTCP is currently present on both RTPSender and RTPReceiver, while WriteRTCP is on PeerConnection. I think it would make much more sense to move ReadRTCP to PeerConnection as well. The current solution using DestinationSSRC() method is very opionated and causes some rtcp packets to be duplicated (in case they belong to multiple ssrc, or they are in a batch with another rtcp packet that belong to a different rtcp, see session_rtcp.go/destinationSSRC). In fact in our project we went to great length to undo this behaviour and get a single ReadRTCP method, without packet duplications. Is there any plan to modify this behaviour? (It would make calling BindReadRTCP in the Interceptor much easier).

Any suggestions are welcome.

@Sean-Der @at-wat please check

Summary

Currently every candidate created will open up a new UDP connection. This makes it hard to configure some loadbalancers/firewalls that do not support configuring port ranges, and can also impact performance if there are thousands of forwarded ports. I would like to suggest a single port mode option that allows listening on a single UDP port instead. This consists of 2 parts:

1. Server reflexive candidates should not open a separate port, but use the same port as the corresponding host candidate. Some examples in the rfc suggests this kind of behaviour, although it's never stated explicitly

RFC: https://tools.ietf.org/html/rfc8445#appendix-B.2

The initiating agent obtains a host candidate on its IP address on network C (10.0.1.100:2498) and a host candidate on its IP address on network A (192.168.1.100:3344). It performs a STUN query to its configured STUN server from 192.168.1.100:3344.

2. The rfc contains some notes about the possiblity of using a single ICE candidate for multiple ICE sessions. I think it would be possible to gather local candidates once, and share them through the whole lifetime of the server. The packets can be sorted based on the source address returned by PacketConn.ReadFrom .

RFC: https://tools.ietf.org/html/rfc8445#section-8.3.1

(a candidate may be used by multiple ICE sessions, e.g., in forking scenarios)

Motivation

To make loadbalancer/firewall configurations simpler by only listening on a single UDP port. Will be also useful once TCP support is implemented, because the server can listen on tcp port 80 (some hotel/public wifis only allow outgoing requests to TCP port 80 and 443).

Implementation ideas

I can think of two ways to implement this, and I would be happy to do either of them (or anything else if someine has a better idea):

• make candidate gathering pluggable (with a Candidate interface), and write a plugin for single port mode
• make a configuration option for single port mode, and modify the candidate struct to handle multiple connections (and multiple agents).

Questions

• What do you think of the idea in general?
• Would you like (1.) to be part of the core functionality?
• Would you like (2.) to be part of the core functionality?
• Would you mind if I made the gathering part of the agent pluggable in order to support this?

In step 5.3.1 (https://www.w3.org/TR/webrtc/#dfn-update-the-negotiation-needed-flag), since we do not have a sender we can only choose to signal that we need negotiation. This typically happens in a case where the offerer asks for a recv only, but the answering side does not provide it. This only triggers if an on negotiation needed handler is set.

I believe the fix I applied is mostly correct by interpreting 5.3.1 and how in the Google WebRTC code, a non-nil sender is created during SetRemoteDescription. My only concern with the fix is that this would basically rely on the handler of negotiation needed to determine they need to go add that track. If they don't you can imagine an endless loop between offerer and answer.

@Sean-Der, while working on this, I realized that SetRemoteDescription may be non-compliant due to https://www.rfc-editor.org/rfc/rfc8829#section-5.10-6.6.2.2.2.2. However, if I make us compliant by removing https://github.com/pion/webrtc/blob/1c20cd4d7b48d2ab8f166486730eb2475f3f8333/peerconnection.go#L1084-L1088, the newly added test fails. Not sure how to reconcile that one as my knowledge & reasoning stops there :).

Your environment.

• Browser: chrome 108.0.5359.124
• Other Information - stacktraces, related issues, suggestions how to fix, links for us to have context

What did you do?

What did you expect?

that ontrack event on addTranceiver side fires as intend

What happened?

when i do the following code in ontrack event e.transceiver.direction = "sendrecv"; e.transceiver.sender .replaceTrack(localStream.getAudioTracks()[0]) this doesn't trigger ontrack event in yourConn side the caller there is nothing received in the other side yourConn also that mean yourConn seem to not triggerd there

yourConn2.ontrack = (e) => {

           remoteVideo = document.getElementById("wbrtcremote");
           transceiversRemotePeer = new MediaStream([
             e.transceiver.receiver.track,
           ]);
           remoteVideo.srcObject = transceiversRemotePeer;
           console.log(
             `kind of this track that has arrived to remote peer is ${e.transceiver.receiver.track.kind}`
           );
           console.log("transceiversRemotePeer.receiver.track.onunmute");
           // remoteVideo.srcObject = e.streams[0].getAudioTracks()[0];
         
        //send backTrack

        e.transceiver.direction = "sendrecv";
        e.transceiver.sender
          .replaceTrack(localStream.getAudioTracks()[0])
          .then(() => {
             //check Tranciever
          let senderList = yourConn2.getSenders();
          let recieverList = yourConn2.getReceivers();
          console.log("recieverList in remotepeer is ", recieverList);
          console.log("senderList in remotepeer is ", senderList);
            console.log("Track replaced on transceiver's sender");
            console.log(
              "current direction in remotPeer after changing it is ",
              e.transceiver.direction
            );
          });
    };

but with addTrack used it's triggers theres on yourConn side yourConn2.addTrack(localStream.getAudioTracks()[0]);

configuration code for yourConn side to send and receive is :

          yourConn.addTransceiver(streams.getAudioTracks()[0]);

    yourConn.ontrack = (e) => {
     transceiversLocalPeer = new MediaStream([
          e.transceiver.receiver.track,
        ]);
        localVideo.srcObject = transceiversLocalPeer;
    }

behavior that happend with modified transceiver

yourConn after the call is done or connection is established just as fine yourConn (the caller ) can send audio but not recieve yourConn2 (the calle) can receive audio but can't send

ontrack event for yourConn => not triggered ontrack event for yourConn2 => triggered

This PR contains the following updates:

| Package | Type | Update | Change | |---|---|---|---| | goreleaser/goreleaser-action | action | major | v3 -> v4 |

Release Notes

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

• [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.

here i belive they miss something unless i do 
what they recommended to do in new api to do that 
 

 let handleRenegotiation = async (description) => {
 try {
    if (description) {

 const offerCollision = description.type == "offer" &&
        (makingOffer || yourConn.signalingState != "stable");
     
        ignoreOffer = !polite && offerCollision;


 if (ignoreOffer) {
          return;
      }

 yourConn.setRemoteDescription(description)
 if (description.type == "offer") {
  await yourConn.setLocalDescription();
       
            wbsc.emit("SEND_EVENT_EMIT_CALL_AUDIO", {
              data: { type: "answer", answer: yourConn.localDescription, id: connectedUser },
            });
      }
    }


  }catch (e) {
    console.log(e);
  }
  }

i believe this could work if i'm impolite peer 

what if i'm polite peer 

how to do it should i split the conditions there to check who am i specifically    ? as in here or similar to

or how i can access 2 peersConnection without  clarify that

yourConn = new RTCPeerConnection(config);

yourConn2 = new RTCPeerConnection(config);

can i use 1 object in this case for both connection to work for both end points in generaly without 
yourConn and yourConn2?

or i missed the configuration of onnegotiationneeded ?

also is this how i can implement onnegotiationneeded  ob both side 
 

 yourConn.onnegotiationneeded = async () => {
                  console.warn("yourConn.onnegotiationneeded");
                  try {
                    makingOffer = true;
                    await yourConn.setLocalDescription();
                    wbsc.emit("SEND_EVENT_EMIT_CALL_AUDIO", {
                      data: {
                        type: "negotiation",
                        description: yourConn.localDescription,
                        id: connectedUser,
                      },
                    });
                  } catch (e) {
                    console.log("renegotiationneded fail is ", e);
                  } finally {
                    makingOffer = false;
                  }
                };

yourConn2.onnegotiationneeded = async () => {
              console.warn("yourConn2.onnegotiationneeded");
              const offer = await yourConn2.createOffer();
              try { 
              
                
                
               
                makingOffer = true;
                await yourConn2.setLocalDescription(); 
                wbsc.emit("SEND_EVENT_EMIT_CALL_AUDIO", {
                  data: { type: "negotiation", description: yourConn2.localDescription, id: connectedUser },
                })
              }
              catch (e) {console.log("renegotiationneded fail is ",e) }
              finally { makingOffer = false }

            }
            

if so how in handleRenegotiation ()
i accept description with yourConn.setRemoteDescription(description) that rollback my offer and then create answer and send it to the remote peer as i'm the one how sent the offer what i did here is wrong and should be fix? also onnegotiationneeded fires when or where? as i guees i as a polite peer do iceRestart() -> onnegotiationneeded (sending localdesscription ) to handleRenegotiation () pass ignoreOffer guard in await yourConn.setLocalDescription(); does it rollback my existing offer or what should it do? after that what happend with onnegotiationneeded does i throw to next step or webrtc will re run onnegotiationneeded again ? and within it what happend to my sdp and icecandidate if that the logic i should stick with as they recomended who restart ice first and making the other peer restart his ice secondly

Your environment.

• Version: v3.1.7

What did you do?

SCTPTransport's Stop method was called in ion-sfu relay peer's Close method.

What did you expect?

The underlying SCTPTransport to be stopped in a timely manner.

What happened?

The call to Stop seems to have caused a deadlock; the method never returns. Here is the relevant block trace from pprof:

108214 2 @ 0x7cb6f0 0x7cb6df 0x481102 0x7cb645 0x7cb5f5 0x7f7dc2 0x7f30c5 0x80b128 0x80aaf8 0x8ecc78 0x90cc88 0x18edbca 0x476a61
#	0x7cb6ef	sync.(*Mutex).Lock+0x6f						/usr/local/go/src/sync/mutex.go:90
#	0x7cb6de	github.com/pion/transport/connctx.(*connCtx).Close.func1+0x5e	/go/pkg/mod/github.com/pion/[email protected]/connctx/connctx.go:154
#	0x481101	sync.(*Once).doSlow+0xc1					/usr/local/go/src/sync/once.go:74
#	0x7cb644	sync.(*Once).Do+0x64						/usr/local/go/src/sync/once.go:65
#	0x7cb5f4	github.com/pion/transport/connctx.(*connCtx).Close+0x14		/go/pkg/mod/github.com/pion/[email protected]/connctx/connctx.go:152
#	0x7f7dc1	github.com/pion/dtls/v2.(*Conn).close+0x1a1			/go/pkg/mod/github.com/pion/dtls/[email protected]/conn.go:921
#	0x7f30c4	github.com/pion/dtls/v2.(*Conn).Close+0x24			/go/pkg/mod/github.com/pion/dtls/[email protected]/conn.go:341
#	0x80b127	github.com/pion/sctp.(*Association).close+0xc7			/go/pkg/mod/github.com/pion/[email protected]/association.go:463
#	0x80aaf7	github.com/pion/sctp.(*Association).Close+0xd7			/go/pkg/mod/github.com/pion/[email protected]/association.go:443
#	0x8ecc77	github.com/pion/webrtc/v3.(*SCTPTransport).Stop+0x77		/go/pkg/mod/github.com/pion/webrtc/[email protected]/sctptransport.go:130
#	0x90cc87	github.com/pion/ion-sfu/pkg/relay.(*Peer).Close+0x287		/go/pkg/mod/github.com/playback-sports/[email protected]/pkg/relay/relay.go:336
#	0x18edbc9	github.com/playback-sports/relayer/pkg.(*Relayer).Start+0x9e9	/build/pkg/relayer.go:256

This happens very rarely but figured I'd file a report anyway.