Connection drops - bitrate issue

[Chai]

Support,

We started noticing this issue since morning after updating webrtc_adapator.js. 

Any ideas how to resolve this? 

2021-07-21 17:20:22,550 [vert.x-eventloop-thread-16] INFO  i.a.enterprise.webrtc.WebRTCAdaptor - Client:2082844632 for stream paris-chronicles-with-patrick-12-episodes-1 current video bitrate: 1500000 audio bitrate: 128000 webrtc client target bitrate: 2759000

2021-07-21 17:20:22,588 [https-openssl-nio2-0.0.0.0-5443-exec-10] ERROR i.a.e.w.WebSocketEnterpriseHandler - Error: java.io.IOException: java.io.IOException: Connection reset by peer

at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:324)

at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:259)

at org.apache.tomcat.websocket.WsSession.sendCloseMessage(WsSession.java:613)

at org.apache.tomcat.websocket.WsSession.doClose(WsSession.java:498)

at org.apache.tomcat.websocket.WsSession.doClose(WsSession.java:460)

at org.apache.tomcat.websocket.server.WsHttpUpgradeHandler.upgradeDispatch(WsHttpUpgradeHandler.java:176)

at org.apache.coyote.http11.upgrade.UpgradeProcessorInternal.dispatch(UpgradeProcessorInternal.java:54)

at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:59)

at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)

at org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.doRun(Nio2Endpoint.java:1620)

at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

at java.base/java.lang.Thread.run(Thread.java:829)

Caused by: java.io.IOException: Connection reset by peer

at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)

at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)

at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)

at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:245)

at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)

at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishRead(UnixAsynchronousSocketChannelImpl.java:395)

at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:191)

at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213)

at java.base/sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:306)

... 1 more

2021-07-21 17:20:22,588 [https-openssl-nio2-0.0.0.0-5443-exec-10] INFO  i.a.enterprise.webrtc.WebRTCMuxer - unregister webrtc client(1808618790) to webrtc muxer(720p - codec:H264) for stream: paris-chronicles-with-patrick-12-episodes-1 current viewer count:114 decremented:true

2021-07-21 17:20:22,588 [https-openssl-nio2-0.0.0.0-5443-exec-10] INFO  i.a.enterprise.webrtc.WebRTCClient - Clearing native stats timer for stream:paris-chronicles-with-patrick-12-episodes-1 and viewer hashcode:1808618790

[Yash]

Hi Chai,

Hope you are doing well.

Could you please tell me more about the issue you are getting during stream and the how can we reproduce it at our end.

Looking forward to hear from you soon.

 

Regards,

Yash 

 

 

[Chai]

Yeah,

Yes. A publisher started the stream on their mobile(using WiFi) and there are about 150 viewers viewing the stream. 

After the publisher streams for 5 minutes, the image gets frozen briefly before the stream ends automatically. This is causing lot of issues of our publishers since yesterday. Does that make sense and the logs attached? Do you need more info? This is a business stopping issue for us. 

--

Chaitanya Malla

[Ant Media Support]

Hi Chai,
As you mentioned you have started facing issue after updating webrtc_adapator.js. Could you please let us know that what changes you have made and have you tried by reverting the changes that you have made.

Regards,
Yash

 

Star Ant Media Server on GitHub and get a chance to win an AMS Annual Enterprise Edition License.

On Wed, 21 Jul at 10:05 PM , Chaitulittle <chaitulittle@gmail.com> wrote:

Yeah,

Yes. A publisher started the stream on their mobile(using WiFi) and there are about 150 viewers viewing the stream. 

After the publisher streams for 5 minutes, the image gets frozen briefly before the stream ends automatically. This is causing lot of issues of our publishers since yesterday. Does that make sense and the logs attached? Do you need more info? This is a business stopping issue for us. 

On Wed, Jul 21, 2021 at 11:10 AM 'Yash Tandon' via Ant Media Server <ant-media-server@googlegroups.com> wrote:

Hi Chai,
Hope you are doing well.
Could you please tell me more about the issue you are getting during stream and the how can we reproduce it at our end.
Looking forward to hear from you soon.

Regards,
Yash 

--
You received this message because you are subscribed to a topic in the Google Groups "Ant Media Server" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ant-media-server/ghqmPmxDpPE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ant-media-server+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ant-media-server/2e9d15d3-afa1-4dfe-8aa1-25cf0d35ff35n%40googlegroups.com.

--

Chaitanya Malla

--
You received this message because you are subscribed to the Google Groups "Ant Media Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ant-media-server+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ant-media-server/CAMDrddE0Z8p9veS6MXeyBb1YwZ1kiAAYGRSe268mkw6bCno0aw%40mail.gmail.com.

44678:1264838

[Chai]

Yes,

We reverted the changes and we will retest in an hour with a live audience. Here is the diff of webrtc changes: https://gist.github.com/chai2/df422819b4bb42a0f8dbf4a607f34a31

 

[Yash]

Sure, please let us know after your test if you again face the same issue.

 

Regards,

Yash

 

 

[Chai]

Yash,

We are still noticing some errors but the connections don't seem to drop now. But, we are on WiFi this time. We haven't tried with 4G connection yet. Looks like there is a massive audio/video sync issue and then the video freezes. 

Our adaptive streaming settings:

FPS: 20fps

Resolution:

720p - 1500(video) - 128(audio)

Can you explain this bitrate information especially "webrtc client target bitrate: 4723000"? 

2021-07-21 20:08:29,828 [vert.x-eventloop-thread-9] INFO  i.a.enterprise.webrtc.WebRTCAdaptor - Client:570505553 for stream how-to-create-the-life-of-your-dreams-1 current video bitrate: 1500000 audio bitrate: 128000 webrtc client target bitrate: 4723000

2021-07-21 20:08:29,832 [vert.x-worker-thread-16] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - Dropping video frame. Pending Frame:1, Frame number: 15514 Last Frame Number: 15514 Max Frame Queue Size: 20 Received Audio Frame Count: 77575 Drop video frame count: 6919 for stream: how-to-create-the-life-of-your-dreams-1

2021-07-21 20:08:29,841 [EncoderQueue - 2166] INFO  i.a.e.w.codec.VirtualVideoEncoder - setRateAllocation bitrate: 3390000 framerate: 60 adjusted bitrate:3390000 for client: 24444316

2021-07-21 20:08:29,973 [ForkJoinPool.commonPool-worker-9] INFO  i.a.e.w.codec.VirtualVideoEncoder - frame id: 1885893296 capture TimeNs: 775700000000

2021-07-21 20:08:30,057 [EncoderQueue - 2280] INFO  i.a.e.w.codec.VirtualVideoEncoder - setRateAllocation bitrate: 921000 framerate: 60 adjusted bitrate:920000 for client: 1453621302

2021-07-21 20:08:30,199 [EncoderQueue - 2641] INFO  i.a.e.w.codec.VirtualVideoEncoder - setRateAllocation bitrate: 2736000 framerate: 60 adjusted bitrate:2736000 for client: 555290720

[Chai]

Yash,

FWIW - We had the display like this before the stream froze and the stream dropped. Have you seen something like that before?

[Yash]

Hi Chai,

As per logs that you have mentioned, it looks like there are lot of video frame drops which could be the reason behind the issue. Sometimes frame drops could be because of network issue as well.

You can also check this --> https://antmedia.io/webrtc-video-play-pixelating-blurring-issues-troubleshooting/ 

 

Regards,

Yash