Start a new topic
Solved

Plugin API getModelViewMatrix() doesn't seem to return correct matrix

Hello,


I'm using  javascript API for iOS ver 7.2.1, writing a plugin to report the screen position of a detected target image back to the webview through javascript.


I wrote my plugin based on the FaceDetectionPlugin sample.

In my code, I attempted to draw a rectangle around the detected image, similar to the one in  WTSimpleImageTrackerViewController in the Native SDK sample. 


However the matrix returned from getModelViewMatrix() doesn't seem correct.  Instead of a rectangle around the detected target image, it results in drawing a small cross at the center of the detected image.


Please refer to the attached file.

zip

Having the positon of a target image relatively to the screen is a feature I would love to have ! Really hope that you can make it.


@Wikitude could this plugin could eventually work with the Cordova SDK ?

Any comment from support team?


1 person likes this

Hello,



my apologies for the late response. Would it be possible for me to see a screenshot of the incorrect rendering? I'm afraid I don't have the time to have a closer look at your code currently. I'm hoping having a look at the problem visually might be revealing.



- Daniel

Hello,


I have attached a screen shot.  Notice the tiny orange cross in the middle of the surf board, which is the result of the matrix returned by the API.

PNG
(5.7 MB)

Good morning,



that's interesting. Is the little cross being tracked correctly (staying in the center of the target when the camera or taget is moved)?



- Daniel

Yes, the cross stays in the center of the target even when the target or the camera is moved; so, it is being  correctly tracked. 

Hi,



if the target is being tracked correctly, the pose can't be that wrong. I would recommend that you have a look at the "CustomCamera" plugin sample. It does exactly what you are trying to do; it renders a simple quad as an augmentation from within the plugin on an image target using getModelViewMatrix. You should actually be able to directly copy&paste most of the code. Don't be intimidated by the amount of rendering code, most of it pertains to the fullscreen effect that is being rendered.


I just ran the sample and found it to be working correctly.


Here's the relevant excerpt:


const float TARGET_HEIGHT_IN_PIXELS = 320.0f;
            
wikitude::sdk::Matrix4 toSizeInPixelsScaling;

// scaled to be able to re-use the screen aligned quad (side length = 2) and thus
// avoid having to create additional vertex data and correspondig code
toSizeInPixelsScaling.scale(0.5f, 0.5f, 1.0f);
            
toSizeInPixelsScaling.scale(TARGET_HEIGHT_IN_PIXELS, TARGET_HEIGHT_IN_PIXELS, 1.0f);
            
wikitude::sdk::Matrix4 modelView = targetToDraw.getModelViewMatrix();
wikitude::sdk::Matrix4 projection = targetToDraw.getProjectionMatrix();

wikitude::sdk::Matrix4 scaleMatrix;
scaleMatrix.scale(targetToDraw.getTargetScale().x, targetToDraw.getTargetScale().y, 1.0);
            
wikitude::sdk::Matrix4 modelViewProjection = projection * modelView * scaleMatrix * toSizeInPixelsScaling;
            
WT_GL_ASSERT(glUseProgram(_augmentationShaderHandle));
GLint modelViewProjectionLocation;
WT_GL_ASSERT_AND_RETURN(modelViewProjectionLocation, glGetUniformLocation(_augmentationShaderHandle, "uModelViewProjectionMatrix"));
WT_GL_ASSERT(glUniformMatrix4fv(modelViewProjectionLocation, 1, GL_FALSE, modelViewProjection.get()));
            
setVertexAttributes(_augmentationShaderHandle);
            
WT_GL_ASSERT(glDrawElements(GL_TRIANGLES, sizeof(_indices)/sizeof(_indices[0]), GL_UNSIGNED_SHORT, 0));



- Daniel

Here's a small video demonstrating what the sample does. The solidly coloured rectangle is rendered from within the plugin.


- Daniel

mov

Hello,


Thank you for the thorough explanation, Daniel!

Following the sample code, I can now draw a quadrilateral that fits around the recognized target.

So, it means the matrix returned by the API does not contain the scale component (which resulted in a cross)... but that leaves me with another question...


Why "const float TARGET_HEIGHT_IN_PIXELS = 320.0f;" ?

What is the rational behind the number "320" ?

Good morning,



the pose is set up in a way such that you need to apply it to an augmentation in the pixel coordinates of the image target. The value 320 is simply the height of the surfer target as can be seen in the screenshot I attached. For targets of different size you would need to change this value accordingly.


Applying the value as a scale simply turns the [-0.5, 0.5] quad into a [-TARGET_HEIGHT/2, TARGET_HEIGHT/2] quad.


Can't say I remember why it is set up like that, but I'm sure there some legacy reason for it.



- Daniel


Hello,


Thank you for the explanation, Daniel.

After further investigation into several other .wtc files, it seems the thumbnail images stored in a .wtc bundle are always aspect fitted into a 320 x 320 bound.   The getTargetScale() API always returns scale.x = 1.0 when the target image is in landscape; conversely, scale.y = 1.0 when the target image is in portrait.

Scale of 1 is always 320 pixels, so "const float TARGET_HEIGHT_IN_PIXELS = 320.0f;" should work for any targets.

Login or Signup to post a comment