Hello, We are using Unity 2020.3.2 to build for Android. We are using the Wikitude SDK 9.6 Professional Edition.
We are facing an issue when using the DeX feature of a Samsung Android device that is connected to an external screen : the wikitude camera acts as if the device was on portrait mode whereas we only allow landscape mode on our application. This leads to the camera visualization being rotated by 90° on the external screen.
Out setup looks like this :
- Put the Samsung device in Landscape orientation
- Connect the Samsung device to a screen through HDMI
- Activate the DeX feature on the Samsung device - Using the Samsung device as a touch pad to control the mouse on the screen, launch the application
-
When activating the Wikitude Camera, the video feeedback on the screen
is rotated by 90°. By turning the Samsung device to portrait, the video
is now properly oriented, but this orientation constraint is not
desired.
This behaviour of forcing the orientation of the camera
does not only appear on our application; it also happens when using the
default Android Camera application.
This leads to the real issue we are raising : we are using the Wikitude Plugin.OnCameraFrameAvailablecallback to send the camera frames to remote players. However the frames we gather through this callback are NOT
affected by this rotation (displaying these frames shows a perfect
landscape video) which causes a 90° rotation between the video stream on
the local player, and the remote video stream.
Digging up a bit the Wikitude code, it seems that the BackgroundCamera, which shows the camera visualization, uses some Wikitude library code to display the camera (WikitudeSDK.PlatformBridge.GetRenderTexture()). The Plugin.OnCameraFrameAvailable on the other side uses the Unity WebCamTexture class to gather the frames. We tried to reproduce the WebCamTexture behaviour by setting up an empty scene with a WebCamTexture of our own and the result was the same, i.e the camera was properly seen on landscape.
Do you have any idea why the two video streams use a different algorithm path (leading to different result in our specific case) ? Is there any workaround that could help us dealing with this issue ?
Additional information : - We tried the Wikitude SDK 9.7 and 9.13 and the result was the same - We tried several devices and only the most recent one had no rotation offset in any of its application - Considering the fact that the Camera behaviour might change depending on the device, we can not consider "rotating back" the CameraFrame when detecting the DeX as this would have a negative impact on some devices
TeamThales RaRv
Hello,
We are using Unity 2020.3.2 to build for Android.
We are using the Wikitude SDK 9.6 Professional Edition.
We are facing an issue when using the DeX feature of a Samsung Android device that is connected to an external screen : the wikitude camera acts as if the device was on portrait mode whereas we only allow landscape mode on our application. This leads to the camera visualization being rotated by 90° on the external screen.
Out setup looks like this :
- Put the Samsung device in Landscape orientation
- Connect the Samsung device to a screen through HDMI
- Activate the DeX feature on the Samsung device
- Using the Samsung device as a touch pad to control the mouse on the screen, launch the application
- When activating the Wikitude Camera, the video feeedback on the screen is rotated by 90°. By turning the Samsung device to portrait, the video is now properly oriented, but this orientation constraint is not desired.
This behaviour of forcing the orientation of the camera does not only appear on our application; it also happens when using the default Android Camera application.
This leads to the real issue we are raising : we are using the Wikitude Plugin.OnCameraFrameAvailable callback to send the camera frames to remote players. However the frames we gather through this callback are NOT affected by this rotation (displaying these frames shows a perfect landscape video) which causes a 90° rotation between the video stream on the local player, and the remote video stream.
Digging up a bit the Wikitude code, it seems that the BackgroundCamera, which shows the camera visualization, uses some Wikitude library code to display the camera (WikitudeSDK.PlatformBridge.GetRenderTexture()).
The Plugin.OnCameraFrameAvailable on the other side uses the Unity WebCamTexture class to gather the frames. We tried to reproduce the WebCamTexture behaviour by setting up an empty scene with a WebCamTexture of our own and the result was the same, i.e the camera was properly seen on landscape.
Do you have any idea why the two video streams use a different algorithm path (leading to different result in our specific case) ? Is there any workaround that could help us dealing with this issue ?
Additional information :
- We tried the Wikitude SDK 9.7 and 9.13 and the result was the same
- We tried several devices and only the most recent one had no rotation offset in any of its application
- Considering the fact that the Camera behaviour might change depending on the device, we can not consider "rotating back" the CameraFrame when detecting the DeX as this would have a negative impact on some devices
Kind regards,
Thales team