Hello Wikitude Team,
I have gone through the documentation of :
Plugins API Wikitude SDK Unity 9.10.0 Documentation
And downloaded the NativeSDKExamples for Android, from what I understand I should be able to modify the code in the NativeSDKEamples to my needs for custom camera input. Then export an .aar and add that in my wikitude Unity project and add the desired calls in SimpleInputPluginController from the Unity examples.
So in total
1)Customize NativeSDKExamples for Android
2)Export into aar
3)Import in Unity
4)Customize Unity Camera Input
Would that approach work or do I have to take a different route to use a Android class to feed the input?
The query here is about both Android & Unity so I am not sure which post topic to include it in, please modify that if you feel like it should be in a different category.
where did you read that you have to package the native SDK examples as an AAR and import it into Unity?
The Wikitude Unity SDK already provides the Plugin component to feed camera frames into... And some samples to show how to do that.
You could have a look into CustomCameraController.cs located under Assets/Wikitude/Samples/Scripts/InputPlugin if you open the sample project. There is a method called SendCameraFrame() that shows how to send custom frames to the SDK.
I didn't actually read that bad assumption I guess.
I assumed some of the files included in the AndroidExamples are what create the aar file included in Unity SDK as android_component.aar and therefore that I could customize those and export my own aar with the input I want.
So the approach you are suggesting would be to make a different kind of aar which takes the frames I want ( YUV Frames ) and exposes them so that I call it from SendNewCameraFrame.
Is it the frameData variable the one I have to replace with the byte buffer of the YUV frame I have in Android?
var plane = new CameraFramePlane();
plane.Data = CallMyAndroidFuntion();
plane.DataSize = (uint)_frameDataSize;
plane.PixelStride = 4;
plane.RowStride = _feed.width;
var planes = new List<CameraFramePlane>();
you don't have to create an aar... That can be done with the libs included in the SDK.
And yes, thats the struct that needs to be populated and send to the SDK. Here the scripting reference.
My question was if
1)I have to modify the existing aar which includes how the WikitudeCamera to directly change the input
2)Create my own aar so that the android code I have communicates with Unity.
But it is clear now I have to go with option 2.
So the java Android code I have to use will forward the image into the C# script and populate the variables you shared.
Will update the thread once I am finished for future reference in case other devs look for the same thing.
Yes, the second one is the option to go. Although I think that you can also add just plain Java code under Assets/Plugins/Android... So it doesn't even have to be packaged in an aar.
I would love to see some updates on your progress. Keep us posted!