Wikitude

Hi,

My first question (https://support.wikitude.com/support/discussions/topics/5000081332) was not clear maybe so here is a better one : 

I would like to use Wikitude for tracking, but I would like to keep my own rendering engine.

Based on the SimpleClientTracking example I saw how I can make my own external render for the 3D model, but I would like to render the background picture too.

It seems the RenderExtension does the job by itself in the GLRenderer, but is it possible and how to make me drive the background render (the camera picture)? No getting the picture, just asking for the background picture texture binding would be enough.

I really need to drive the render operations by myself to order them and make them match my custom rendering engine.

Thanks for any help you could provide me.

Regards, 

    Vincent

Hi Andreas,

When you say "after" does it mean after the method onDrawFrame() has returned (so I cannot know exactly when) or is it done in the onDrawFrame() method, before the method returns ?

Other point, is the texture in RGB ?

Thanks again.

Regards, 

   Vincent

Hi Andreas.

So, here is my loop : 

• Main 3d loop start
• AR onDrawFrame()
• 3D render background picture from specified index with RBG texture datas
• 3D render other stuffs
• Main 3D loop end

Hi,

It helps, but still some doubts : 

According to your summary, I can choose to use the renderExtention or not ? I understood from our previous exchanges that I need to manage the renderExtension instance to call onDrawFrame() at least to have the AR working.

Here is what I do : 

I create a new texture in my rendering engine, and I set Wikitude camera manager to use the same texture index. I just expect from my AR engine to bind the camera picture to this context, the rendering engine managing the creation (in RGB888), render, etc...

So, let's start from your first case : I provide the RGB texture, I render it. What do I need to do in my Wikitude AR library to  have this offscreen rendering (i.e. texture binding) ?

The texture I saw in index 0 is probably the Android capture of the camera in Luminance you use internally.

Last point : 

To set Wikitude opengl context, I have 

startupConfiguration.setTextureId()

and

mWikitudeSDK.getCameraManager().setTextureId()

Is there any difference ? 

Thanks for you help.

Regards, 

    Vincent

Hi Andreas, 

I think I call renderExtension methods as requested.

Here is an extract : 

package com.innersense.osmose.ar.wikitudeandroid;

import android.app.Activity;
import android.content.Context;
import android.util.Log;

import com.badlogic.gdx.Gdx;
import com.badlogic.gdx.math.Matrix3;
import com.badlogic.gdx.math.Matrix4;
import com.badlogic.gdx.math.Quaternion;
import com.badlogic.gdx.math.Vector3;
import com.wikitude.NativeStartupConfiguration;
import com.wikitude.WikitudeSDK;
import com.innersense.osmose.ar.wikitude.ARWikitudeEngine;
import com.wikitude.common.camera.CameraSettings;
import com.wikitude.common.camera.CameraSettings.CameraPosition;
import com.wikitude.common.camera.CameraSettings.CameraResolution;
import com.wikitude.common.rendering.RenderExtension;
import com.wikitude.rendering.ExternalRendering;
import com.wikitude.tracker.ImageTarget;
import com.wikitude.tracker.ImageTracker;
import com.wikitude.tracker.ImageTrackerListener;
import com.wikitude.tracker.TargetCollectionResource;
import com.wikitude.tracker.TargetCollectionResourceLoadingCallback;

/**
 * Created by Vincent on 31/01/2017.
 */

public class ARWikitudeEngineAndroid extends ARWikitudeEngine implements ImageTrackerListener, ExternalRendering {

    private static final String TAG = "ARWikitudeEngineAndroid";


    private WikitudeSDK mWikitudeSDK;
    /**
     * Android activity instance.
     */
    private Activity activity;
    /**
     * The vuforia license key to use.
     */
    private final String wikitudeLicenseKey;
    private TargetCollectionResource mTargetCollectionResource;

    RenderExtension renderExtension;

    public ARWikitudeEngineAndroid(String datasetFileName, boolean isAbsolute, String licenseKey) {
        super(datasetFileName, isAbsolute);
        this.wikitudeLicenseKey = licenseKey;
    }


    @Override
    public void onARStart() {
        super.onARStart();

        mWikitudeSDK = new WikitudeSDK(this);
        NativeStartupConfiguration startupConfiguration = new NativeStartupConfiguration();
        startupConfiguration.setLicenseKey(wikitudeLicenseKey);
        startupConfiguration.setCameraPosition(CameraPosition.BACK);
        startupConfiguration.setCameraResolution(CameraResolution.AUTO);
        startupConfiguration.setTextureId(7); //TODO : set runtime ?

        mWikitudeSDK.onCreate(activity.getApplicationContext(), activity, startupConfiguration);

        mTargetCollectionResource = mWikitudeSDK.getTrackerManager().createTargetCollectionResource(datasetFileName, new TargetCollectionResourceLoadingCallback() {
            @Override
            public void onError(int errorCode, String errorMessage) {
                Log.e(TAG, "Error : Failed to load target collection resource. Reason: " + errorMessage);
            }

            @Override
            public void onFinish() {
                mWikitudeSDK.getTrackerManager().createImageTracker(mTargetCollectionResource, ARWikitudeEngineAndroid.this, null);
            }
        });
    }

    @Override
    public void onARResume() {
        super.onARResume();
        mWikitudeSDK.onResume();
        renderExtension.onResume();
    }

    @Override
    public void onARDestroy() {
        super.onARDestroy();
        mWikitudeSDK.clearCache();
        mWikitudeSDK.onDestroy();
    }

    @Override
    public void onARPause() {
        super.onARPause();
        mWikitudeSDK.onPause();
        renderExtension.onPause();
    }

    @Override
    public void onARConfigurationChanged() {
        super.onARConfigurationChanged();
    }

    @Override
    public void onRenderExtensionCreated(final RenderExtension renderExtension) {
	
        //set texture ID
        mWikitudeSDK.getCameraManager().setTextureId(this.renderingEngine.getImageIndex());

        this.renderExtension = renderExtension;
        this.renderExtension.onSurfaceCreated(null, null);
    }

    @Override
    public void onTargetsLoaded(ImageTracker imageTracker) {
    }

    @Override
    public void onErrorLoadingTargets(ImageTracker tracker, int errorCode, final String errorMessage) {
    }

    @Override
    public void onImageRecognized(ImageTracker tracker, final String targetName) {
    }

    @Override
    public void onImageTracked(ImageTracker imageTracker, ImageTarget imageTarget) {
    }

    @Override
    public void onImageLost(ImageTracker imageTracker, String s) {

    }

    @Override
    public void onExtendedTrackingQualityChanged(ImageTracker imageTracker, String s, int i, int i1) {

    }


	//Rendering engine main render loop
    @Override
    public void onBeginRenderFrame() {
        super.onBeginRenderFrame();

        //make a new render loop
        renderExtension.onDrawFrame(null);

    }
}

Do I miss some renderExtension lifecycle call ?

Just to be sure to understand well, using

startupConfiguration.setTextureId()

 is enougth to have the texture being binded instead of rendered during the onDrawFrame() call ?

Thanks.

Regards,

    Vincent

Hi  Andreas,

Yes, the texture id to "7" si for debug/test usage only. I'll use setTextureID too to be sure, with the same index.

It would be difficult for me to create a little project since our rendering engine is not designed to be working like that, but I'l think about it, it could make us gain some important time.

Thanks again, I'll be back here with more questions or maybe some good results ASAP !

Regards,

    Vincent

Hi, 

I created a bitbucket repository with a test project to have the minimalist test code, I prefer to avoid setting it public, could you send me one or more emails to give read access on this repository ? 

So we can have this common code base to understand each other.

Thanks.

Regards, 

    Vincent

Hi Andreas,

I just send an email.

I downloaded the last release, I'm upgrading the project with Sdk 2.1.0 so I could test the lastest features (separate rendering from tracking)

Thanks.

Regards, 

     Vincent

Hi,

I gave you access on the repository, don't hesitate if you have any questions.

I did not found documentation about separating rendering from tracking on the last version, did you wrote anything about it ?

Thanks.

Regards, 

    Vincent

Hi Andreas, 

I found something about renderExtension.onUpdate() and onDrawFrame(), I did the same from the from here http://www.wikitude.com/external/doc/documentation/latest/androidnative/renderingnative.html#external_rendering_api

Still not rendering the background picture from camera... don't really understand where is the problem...

Thanks

Regards, 

    Vincent