Publish Timeout Response from AntMedia Server for Pre-encoded h264 Video Packets

[Muhammad Usman Bashir]

Dear Community, 

I hope you guys are doing well. I am working on a WebRtc Linux based CPP Program which is interacting with Ant Media Enterprise Edition 2.4.4-SNAPSHOT 20220728_1826. I have integrated websocket client for AntMedia Server using this documentation; webrtc-websocket-messaging-reference.

The complete setup is working fine with RAW Audio and RAW Video Packets and I am able to play them using Antmedia Sample Player. I am getting these RAW Audio and Video frames from FFMPEG. But when I send custom H264 encoded Video packets (Pre-Encoded Video Frames) using WebRtc, I am getting Publish timeout error on the AntMedia Server side. I am getting these Pre-encoded Video frames from FFMPEG. 

2022-08-04 06:39:30,026 [vert.x-eventloop-thread-1] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - running stop operations in executor for stream: rtpstream123 Received audio frame: 0  Processed audio frame: 0Received video frame: 0  Processed video frame: 0
2022-08-04 06:39:30,026 [vert.x-eventloop-thread-1] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - Publish Stats StreamId: rtpstream123 TransferedByte: 564653 Stream Duration: 1659595170026 Bitrate: 0 Source IP: 110.93.193.154 User-Agent: [WebSocket++/0.7.0]

Quote

I have tested the same Linux WebRtc Client on my Web client which has support for H264, Web client is successfully playing the video.

I have attached detailed logs from the server and client side as well. What I have tried on my end;

• Linux firewall is closed, not UDP traffic is being blocked. 
• Using Google STUN on Linux WebRtc Client; 'urls' : 'stun:stun1.l.google.com:19302
• Bitrate on Linux side is 512K, 1024K.
• I have also verified from WebRtc Native logs also by debugging that video packets are being send to network layer.
• I have tried Twilio NTS Service for ICE Servers (STUN + TURN), but didn't work

I am also unable to increase ( settings.webrtc.client.start.timeoutMs=10000 )timeout from the AntMedia Server Dashboard. I am found multiple articles related to my issue but those articles are not helpful;

Publish Timeout Error; Google Groups

WebRtc Ingest is not started; Ant Media Server Github

Publish Timeout Error without any candidate issue; Ant Media Server Github

Your help might save days for me. Thanks a lot.

Quote

In Screenshots, I have compared Linux logs for RAW and Pre-encoded video sending process.

webrtc_linux_client_logs_pre_encoded.txt webrtc_linux_client_log_raw_video.txt Ant-media-Server-logs.txt

Edited August 5, 2022 by Muhammad Usman Bashir
Added some more details regarding issue

[mekya]

Hi Usman,

Just let me approach the problem in a simple way. 

Quote

The complete setup is working fine with RAW Audio and RAW Video Packets and I am able to play them using Antmedia Sample Player. I am getting these RAW Audio and Video frames from FFMPEG. But when I send custom H264 encoded Video packets (Pre-Encoded Video Frames) using WebRtc, I am getting Publish timeout error on the AntMedia Server side. I am getting these Pre-encoded Video frames from FFMPEG. 

If it's working for raw packets but not working for pre-encoded packets. Then there is some different in pre-encoded packets pipeline. 

If you think increasing timeout value will help you to debug the problem, as mentioned just set the

settings.webrtc.client.start.timeoutMs=20000

in application's configuration file. For LiveApp, it's /usr/local/antmedia/webapps/LiveApp/WEB-INF/red5-web.properties.

After that please restart the server

sudo service antmedia restart

 

One of my recommendation is using the web browser to test if the timeout value has increased or not. You can just disable adding or sending ICE candidates in the JS side and you'll monitor when the server reports publishTimeout. In order to disable adding or sending ICE candidates, please comment out these lines that calls addCandidate and sending candidates to the server. Currently, they are in the following lines(the exact lines will change as there are changes on the JS side)

https://github.com/ant-media/StreamApp/blob/master/src/main/webapp/js/webrtc_adaptor.js#L980

https://github.com/ant-media/StreamApp/blob/master/src/main/webapp/js/webrtc_adaptor.js#L569 

Regards,

A. Oguz

[Muhammad Usman Bashir]

Hi Oguz, 

First of all thanks for your response. I think you have some confusion about my implementation. Let me clear it first;

• I am not using AntMedia JS Clients for publishing and testing; I have implemented my own C++ WebRtc Client which is communicating and publishing with AntMedia.

Secondly, I don't have access to increase the following property;

• settings.webrtc.client.start.timeoutMs=20000

As I am using a test account. In simple words, I haven't deployed the Antmedia Server by myself. The Important thing is that I have deployed Antmedia Server on my local machine and for this local AntMedia Server, both RAW and pre-encoded video frames are being successfully broadcasted and played on Antmedia Server.

The local deployed Ant media Server has proved that my implementation has no fault for RAW or Pre-encoded Video frames, but pre-encoded video frames are not working on cloud based Ant media server. Even I haven't increased any timeout for local deployed Ant media Server. It worked on antmedia's default settings.

Just a quick reminder from previous post;

• Linux firewall is closed, not UDP traffic is being blocked. 
• Using Google STUN on Linux WebRtc Client; 'urls' : 'stun:stun1.l.google.com:19302
• I have tried Twilio NTS Service for ICE Servers (STUN + TURN), but didn't work

Any further assistance from your side, will be helpful.

Kind regards, 

 

[mekya]

Hi Usman,

Then I think the problem is not about the changing the settings or raw or pre-encoded frames. If your code is working locally but not working on remote server, it's likely that it cannot gather ICE candidates to have a connection between server and ffmpeg. 

If the JS client is working on local/remote server, it means the server is performing normally. My recommendation is that you can just install it to one of the instance in the cloud and you can try to find out why the cpp implementation does not have ICE connection.

 

Regards,
A. Oguz