Start a new topic

Access Camera Texture In OpenGL

Hi, attempting to use a custom renderer with Wikitude on Android and I'm having trouble accessing the camera data in an OpenGL shader. I've tried several approaches to accomplish this:


  • Create a custom camera, and set its SurfaceTexture to a texture I created in a GL thread. This doesn't seem to work. Looking at the gpu trace with gapid, the texture I create doesn't seem to be filled with data, and if I don't call "RenderExtension.onDrawFrame()" nothing at all is drawn.
  • Use the built in camera capture capabilities and at runtime call "wikitudeSDK.getCameraManager().setTextureId" with texture I generate in my renderer. This results in basically the same behavior as mentioned earlier.

I'm also unclear on how some of the rendering actually works. For instance, I've taken the ExternalRenderingActivity example and replaced GLRenderer with a subclass that issues some other drawing command. To my surprise, when RenderExtension.onDrawFrame() is called in the super onDrawFrame any draw commands I issued in my subclass don't seem to have any effect, even though they were issued later.

In short, I'm looking for a way to use Wikitude's tracking, but perform entirely custom rendering, but I can't seem to get the camera data in my opengl context.

Thanks


Another note:


I've found that if I take the SDK sample, and comment out line 43 of GLRenderer (mWikitudeRenderExtension.onDrawFrame(unused);) the CustomCameraActivity will quit after ~4 minutes or running. This does not occur with the line uncommented.


Any idea why?


Thanks


Hi darren,

Seems that I have to answer quite a lot of questions here. Let's see if we can clarify some of them:


First, the Wikitude Native SDK should fulfil all your needs in regards to rendering flexibility. If you have your own custom rendering, you only need to call the WikitudeRenderExtensions update and draw methods. `Update` triggers internal Wikitude SDK calculations and `draw` updates the given OpenGL texture with the current camera image (in case you didn't setup your own input plugin). I'm not sure if you also want your own camera access implementation or if the standard Wikitude implementation is enough. In case you want your own camera implementation, you don't need to call `draw` (I created a internal ticket to investigate the crash after ~4 minutes).


If you're fine with the default Wikitude camera and you want the camera image rendered into a OpenGL texture, make sure that you set the texture id like you mentioned before (getCameraManager().setTextureId) and that you call the render extension `draw` method before you do any other rendering.


Did you had a look at our example application? They also do some custom rendering (rendering of the augmentations) after the camera image is rendered.


I guess we will write some more messages back and forth, so don't hesitate to ask any further question you have ;)


Best regards,

Andreas

Login or Signup to post a comment