Need more details about the FrameColorSpace format of Wikitude frames with Android Camera2 API : Wikitude

Start a new topic

Need more details about the FrameColorSpace format of Wikitude frames with Android Camera2 API

TeamThales RaRv

started a topic 7 days ago

SDK : Wikitude Unity Pro SDK 9.6.

Unity : 2020.3.2f1

Hello,

Our application is retrieving frames given by Wikitude using the OnCameraFrameAvailable method.

Then we use a WebRTC library (LiveSwitch : https://www.liveswitch.io/) to send the frame across the network a another user.

LiveSwitch provides different video encodings : "NV12", "NV21", "YV12", "I420" ...

When the option "Enable Camera2 API" on the WikitudeCamera is disabled, the frames are correctly encoded, sent and displayed on the other device. The encoding we use in LiveSwitch is I420.

However, when we enable the Camera2 API on the WikitudeCamera, The frames' ColorSpace seems to change, and they are no longer encoded correctly. We have tried many different set-up with no success.

I have found this thread on your forums, but it is 6 years old now and it was not really clear what format was used at the end, and if the U and V planes were interleaved or not : https://support.wikitude.com/support/discussions/topics/5000082012

I have attached my discussion with the LiveSwitch support with more explanation and reproduction steps, please read it to get more details.

It would be great if you could give us more details about the exact format of the frames given by the WikitudeCamera, and maybe how we can use the RowStride and PixelStride, etc... Anything would help.

Best regards,

Thales team