Streaming WebRTC in unity

[Regis]

Hi,

I've been following this tutorial : https://antmedia.io/webrtc-streaming-in-unity/ in order to try and receive a WebRTC stream into unity.

I've downloaded the https://github.com/ant-media/WebRTC-Unity-SDK WebRTC unity SDK.

I created a Ant Server on Linode and broadcast into it using OBS.

The unity sample does not look like the one in the tutorial/

I opened the AMSStreaming scene In Unity, and modified the url in the AMSStreaming script to point to my server : "ws://my_server_ip:5080/WebRTCApp/websocket"

and provided the WebRTCClient with the correct stream id and url.

I left the stun server to the default stun server address : stun:stun.l.google.com:19302

Running the sample, I set the comboBox to "Play" and click "Start".

The connection works fine and 

Quote

            else if(mode == MODE_PLAY) {
                webRTClient.Play();

gets executed.

However nothing else happens, no RTCTrackEvent are received.

I have also placed a breakpoint in the CreatePeerConnection() Method :

Quote

            localPC.OnIceConnectionChange = state => { 
                switch (state)
                {
                    case RTCIceConnectionState.Connected:
                        //PeerConnected();
                        //break;
                    case RTCIceConnectionState.New:
                    case RTCIceConnectionState.Checking:
                    case RTCIceConnectionState.Closed:
                    case RTCIceConnectionState.Completed:
                    case RTCIceConnectionState.Disconnected:
                    case RTCIceConnectionState.Failed:
                    case RTCIceConnectionState.Max:
                        Debug.Log("IceConnectionState: "+state);
                        break;
                    default:
                        throw new ArgumentOutOfRangeException(nameof(state), state, null);
                };
            };

That never gets hit, so the ICEConnectionChange never changes.

Could that be the problem ?

How can I move forward debugging this ?

Thanks

 

[Burak]

Hi Regis,

Do you have the logs for the sent and received ice candidates? This may be a good point to start debugging. Also whole Unity logs can help us more.

By the way, I worked on the Unity SDK but to be honest I'm not a Unity expert. If you are good at Unity we can help each other and improve the SDK together. How does it sound?

[Regis]

Hi Burak,

Here are the logs from unity. There are no other logs in the example provided by Ant Media.

Streaming Mode:2
UnityEngine.Debug:Log (object)
Unity.WebRTC.AntMedia.SDK.AMSStreamingSample:StartButtonPressed () (at Assets/AntMedia/Samples/AMSStreamingSample.cs:134)
UnityEngine.EventSystems.EventSystem:Update () (at Library/PackageCache/com.unity.ugui@1.0.0/Runtime/EventSystem/EventSystem.cs:501)

Waiting for websocket connection...
UnityEngine.Debug:Log (object)
Unity.WebRTC.AntMedia.SDK.AMSStreamingSample/<StartStreaming>d__20:MoveNext () (at Assets/AntMedia/Samples/AMSStreamingSample.cs:75)
UnityEngine.MonoBehaviour:StartCoroutine (System.Collections.IEnumerator)
Unity.WebRTC.AntMedia.SDK.AMSStreamingSample:StartButtonPressed () (at Assets/AntMedia/Samples/AMSStreamingSample.cs:138)
UnityEngine.EventSystems.EventSystem:Update () (at Library/PackageCache/com.unity.ugui@1.0.0/Runtime/EventSystem/EventSystem.cs:501)

Connection open!
UnityEngine.Debug:Log (object)
Unity.WebRTC.AntMedia.SDK.WebRTCClient:<ConnectToWebSocket>b__22_0 () (at Assets/AntMedia/SDK/AntMediaSDK.cs:199)
NativeWebSocket.WebSocket/<Connect>d__27:MoveNext () (at Library/PackageCache/com.endel.nativewebsocket@c5101c0776/WebSocket/WebSocket.cs:472)
UnityEngine.UnitySynchronizationContext:ExecuteTasks ()

Sending Message:{"command":"play","streamId":"medinbox"}
UnityEngine.Debug:Log (object)
Unity.WebRTC.AntMedia.SDK.WebRTCClient:SendWebSocketMessage (string) (at Assets/AntMedia/SDK/AntMediaSDK.cs:276)
Unity.WebRTC.AntMedia.SDK.WebRTCClient:SendPlayMessage () (at Assets/AntMedia/SDK/AntMediaSDK.cs:309)
Unity.WebRTC.AntMedia.SDK.WebRTCClient:Play () (at Assets/AntMedia/SDK/AntMediaSDK.cs:111)
Unity.WebRTC.AntMedia.SDK.AMSStreamingSample/<StartStreaming>d__20:MoveNext () (at Assets/AntMedia/Samples/AMSStreamingSample.cs:87)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

 

I'm happy to help you improve the SDK with pleasure,  but I am stuck at getting it to work.

Have this sample been tested on your side recently ?

As explained. I am only changing the URL of the ant server nothing else...

in CreatePeerConnection()

Quote

localPC.OnIceCandidate = candidate => { 
                Debug.Log("ICE candidate created:"+ candidate.Candidate);
                SendCandidateMessage(candidate.SdpMid, (long)candidate.SdpMLineIndex, candidate.Candidate); 
            };

SendCandidateMessage does not get Called and the localPC connection changed does not change.

Is everything working fine on your side, when you create a stream with the ant media server and try to connect to it ?

 

[Regis]

On 1/20/2023 at 8:07 PM, Burak said:

Hi Regis,

Do you have the logs for the sent and received ice candidates? This may be a good point to start debugging. Also whole Unity logs can help us more.

By the way, I worked on the Unity SDK but to be honest I'm not a Unity expert. If you are good at Unity we can help each other and improve the SDK together. How does it sound?

I'm happy to help you with the unity part. But I'd need help with the WebRTC bit.

Do you confirm that the sample is working on your side when you try it ?

My company is very keen to be able to broadcast 360 videos inside VR Headset.

[Burak]

Thank you @Regis. Lets schedule a meeting together next week. I will tell the WebRTC part and general structure for the SDK. You can tell me the Unity thing. Then we can continue to improve it together. I will write you in DM for the meeting details.