iOS SDK RTCVideoTrack `isEnabled` issue

[Yury Grinenko]

Hey guys!

I'm facing a problem in detecting of participant disables or enables video stream. The iOS client locally sets `isEnabled` property of RTCVideoTrack instance to `true` or `false`, but corresponding video track in remote media stream on the "other side" of call still has `isEnabled` as `true`. And instead of video stream participant receives black frames. In such scenario I would like to hide video view and display placeholder, but I can't figure out how to detect videoTrack state change. 

Is there any way to detect if remote participant disables/enables his video track? 

Thanks

[Mahmut As1m Onat]

Hi Yury,

I'm not experienced in IOS but if the video is disabled, I expect the video track that is sent using WebSocket to be undefined or null. Can you try checking the video parameter of it?

{
command : "publish",
streamId : streamId,
token : token,
video: thiz.localStream.getVideoTracks().length > 0 ? true : false,
audio: thiz.localStream.getAudioTracks().length > 0 ? true : false,
};

Best,
Asim

[Yury Grinenko]

Hi Mahmut!

I don't explicitly send `publish` commands in p2p call. Moreover, in conference calls I send `publish` command without `video` and `audio` parameters. 

In iOS, there is WebRTCClient object, which is responsible for audio and video tracks creation. It adds tracks into RTCPeerConnection object, which is, as I understand, responsible for streams transfer. 

So both callers have RTCPeerConnection objects. They both add local tracks to connection, store local tracks, and then receive remote video stream object from RTCPeerConnection's delegate method (when other participants join with their streams). 

This RTCMediaStreamTrack object cointains audio and video tracks. I can't obtain remote  RTCMediaStreamTrack directly from RTCPeerConnection, only receive it from delegate method (peerConnection ... didAddStream...). So I store this object as well to bind video track to UI. Each participant stores the next objects:

1. Local video track

2. Local audio track

3. Remote media stream with audio and video tracks

All tracks contain `isEnabled` flag. Example iOS app manages this flag to turn on/off video track if user decides to do it during the call. The problem is if user locally changes `isEnabled` flag in his locally stored local video track, this change will not be reflected on the other participant side in his stored remote media stream. And these is no callback in RTCPeerConnection to notify and remote stream state change. And no way to retrieve new updated remote media stream object.

The other way I tried is to dynamically add or remove video track to RTCPeerConnection depending on required video state, but result is the same - remote participant doesn't get any change, and stored remote media stream object does't contain any change.

Thanks

четверг, 30 апреля 2020 г., 19:15:38 UTC+3 пользователь mahmut asım Onat написал:

Hi Yury,

I'm not experienced in IOS but if the video is disabled, I expect the video track that is sent using WebSocket to be undefined or null. Can you try checking the video parameter of it?

{
command : "publish",
streamId : streamId,
token : token,
video: thiz.localStream.getVideoTracks().length > 0 ? true : false,
audio: thiz.localStream.getAudioTracks().length > 0 ? true : false,
};

Best,
Asim

[Yury Grinenko]

Also, as I read from WebRTC documentation, negotiation may occur after the track has been added or removed from connection. Right now `peerConnectionShouldNegotiate` delegate's method is empty, as in example app.  Should it be implemented?

четверг, 30 апреля 2020 г., 19:15:38 UTC+3 пользователь mahmut asım Onat написал:

Hi Yury,

I'm not experienced in IOS but if the video is disabled, I expect the video track that is sent using WebSocket to be undefined or null. Can you try checking the video parameter of it?

{
command : "publish",
streamId : streamId,
token : token,
video: thiz.localStream.getVideoTracks().length > 0 ? true : false,
audio: thiz.localStream.getAudioTracks().length > 0 ? true : false,
};

Best,
Asim