Jump to content
  • 0

[Urgent] Streaming ending abruptly


Chai
 Share

Question

Hi,

Our servers are consistently running into this issue which is causing major bottlenecks in our workflows. 
Any ideas how we can resolve this issue?

May 11, 2021 8:33:26 PM io.vertx.core.impl.BlockedThreadChecker
WARNING: Thread Thread[vert.x-eventloop-thread-31,5,main]=Thread[vert.x-eventloop-thread-31,5,main] has been blocked for 6498801 ms, time limit is 2000 ms
io.vertx.core.VertxException: Thread blocked
at io.antmedia.enterprise.webrtc.WebRTCMuxer.unRegisterWebRTCClient(WebRTCMuxer.java:246)
at io.antmedia.enterprise.webrtc.WebRTCClient.stopPartial(WebRTCClient.java:1094)
at io.antmedia.enterprise.webrtc.WebRTCClient.lambda$stop$17(WebRTCClient.java:1137)
at io.antmedia.enterprise.webrtc.WebRTCClient$$Lambda$545/0x00000008407b3840.run(Unknown Source)
at io.antmedia.enterprise.webrtc.WebRTCClient.lambda$executeOnSignallingThread$12(WebRTCClient.java:965)
at io.antmedia.enterprise.webrtc.WebRTCClient$$Lambda$510/0x0000000840796c40.handle(Unknown Source)
at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:366)
at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38)
at io.vertx.core.impl.EventLoopContext$$Lambda$485/0x000000084072f040.run(Unknown Source)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base@ 11.0.10/java.lang.Thread.run(Thread.java:834)
Link to comment
Share on other sites

  • Answers 6
  • Created
  • Last Reply

Top Posters For This Question

6 answers to this question

Recommended Posts

  • 0

Thanks, Tahir for the prompt response.

Is there something we can fix on our AWS instances interim while we wait for the next coming version? 

Also - any insights on the root cause analysis of how this issue got introduced?

Link to comment
Share on other sites

  • 0

Hi,

 

I fear it is not possible since it is related with the source code of the server. If you are using the license from Ant Media on AWS I can send you a SNAPSHOT maybe?

The issue was, in summary a synchronization problem: https://docs.oracle.com/javase/tutorial/essential/concurrency/syncmeth.html

Cheers

 

Link to comment
Share on other sites

  • 0

Yes,

Can you send us the snapshot?

Also, we are getting publish timeout errors. How can we solve this?

TCEncoderAdaptor - Encode video runner is returning because encoder adaptor is stopped for explore-the-australian-wilderness-1
2021-05-15 21:29:26,659 [vert.x-eventloop-thread-16] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - leaving stop() for explore-the-australian-wilderness-1
2021-05-15 21:29:26,659 [vert.x-eventloop-thread-16] INFO  i.a.AntMediaApplicationAdapter - Closing broadcast stream id: explore-the-australian-wilderness-1
2021-05-15 21:29:26,668 [vert.x-eventloop-thread-16] INFO  i.a.AntMediaApplicationAdapter - Setting timer to call live stream ended hook for stream:explore-the-australian-wilderness-1
2021-05-15 21:29:26,708 [vert.x-eventloop-thread-27] INFO  i.a.AntMediaApplicationAdapter - POST Response Status:: 301
2021-05-15 21:29:30,810 [https-openssl-nio2-0.0.0.0-5443-exec-7] INFO  i.a.e.s.WebSocketLocalHandler - Web Socket opened
2021-05-15 21:29:32,545 [https-openssl-nio2-0.0.0.0-5443-exec-3] ERROR i.a.e.w.WebSocketEnterpriseHandler - Error: EOFException: 
2021-05-15 21:29:36,572 [https-openssl-nio2-0.0.0.0-5443-exec-2] ERROR i.a.e.w.WebSocketEnterpriseHandler - Error: EOFException: 
2021-05-15 21:29:36,781 [https-openssl-nio2-0.0.0.0-5443-exec-9] INFO  i.a.e.s.WebSocketLocalHandler - Web Socket opened
2021-05-15 21:29:39,663 [https-openssl-nio2-0.0.0.0-5443-exec-4] ERROR i.a.e.w.WebSocketEnterpriseHandler - Error: EOFException: 
2021-05-15 21:29:39,746 [https-openssl-nio2-0.0.0.0-5443-exec-7] INFO  i.a.e.s.WebSocketLocalHandler - Web Socket opened
2021-05-15 21:29:44,039 [https-openssl-nio2-0.0.0.0-5443-exec-2] INFO  i.a.e.s.WebSocketLocalHandler - Web Socket opened
2021-05-15 21:29:44,679 [https-openssl-nio2-0.0.0.0-5443-exec-6] ERROR i.a.e.w.WebSocketEnterpriseHandler - Error: EOFException: 
2021-05-15 21:29:54,355 [vert.x-eventloop-thread-0] INFO  i.antmedia.statistic.StatsCollector - System cpu load:0 process cpu load:0 available memory: 63138384 KB used memory(RSS): 984432 KB
2021-05-15 21:29:54,356 [vert.x-eventloop-thread-0] INFO  i.antmedia.statistic.StatsCollector - Vertx worker queue size:0 WebRTCVertx worker queue size:0
2021-05-15 21:29:55,474 [https-openssl-nio2-0.0.0.0-5443-exec-10] INFO  i.a.e.s.WebSocketLocalHandler - Web Socket opened
2021-05-15 21:34:50,938 [https-openssl-nio2-0.0.0.0-5443-exec-1] ERROR i.a.e.w.WebSocketEnterpriseHandler - Error: EOFException: 
2021-05-15 21:34:51,704 [https-openssl-nio2-0.0.0.0-5443-exec-1] INFO  i.a.e.s.WebSocketLocalHandler - Web Socket opened
2021-05-15 21:34:54,355 [vert.x-eventloop-thread-0] INFO  i.antmedia.statistic.StatsCollector - System cpu load:0 process cpu load:0 available memory: 63132260 KB used memory(RSS): 989796 KB
2021-05-15 21:34:54,355 [vert.x-eventloop-thread-0] INFO  i.antmedia.statistic.StatsCollector - Vertx worker queue size:0 WebRTCVertx worker queue size:0
2021-05-15 21:34:55,068 [https-openssl-nio2-0.0.0.0-5443-exec-4] ERROR i.a.e.w.WebSocketEnterpriseHandler - Error: EOFException: 

Link to comment
Share on other sites

  • 0

Please send an e-mail to support@antmedia.io for the snapshot.

We need to check your ICE connection process, can you elaborate your way of sending the broadcast? Also, you should check your firewall, for most of the time UDP ports are getting blocked for webRTC and people get publish time out errors. So please make sure that UDP ports between 5000-65000 are open for server and there are no blockage at the broadcaster.

 

Link to comment
Share on other sites

  • 0
Tahir,

Thanks for the reply. We will check our UDP ports. We don’t have a firewall, but our web application is on Heroku and Ant Instance is on AWS.  

Our process outline:

Our web application is hosted on Heroku. Our Ant Media servers are on AWS. A webpage with integrated WebRTC code will publish the stream which publishes media files to Ant Instance on AWS. 

The viewer pages(on web application) hosted on Heroku will “fetch” for the streams and play them. 

- Tahir 

--
Chaitanya Malla
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...