Start a new topic

Native iOS UIView Manipulation

Native iOS UIView Manipulation


Hi Wikitude Team,

I've managed to get the native iOS example to run and track an image in opengl, great. I however was hoping that the native iOS SDK would allow me to manipulate native UIViews and place them on the tracked object via the objects .center or .layer.transform. Can this be done? My naive approach to move the whiteSquare UIView kind of works, but the tracking is imperfect, probably because im not taking something into account (not an expert in 3D manipulation). Any help of how to overlay the tracked image with a UIView would be great, thanks...

 

    GLKVector3 vector = GLKVector3Make(0, 0, 0);

    

    GLKMatrix4 RESULT;

    memcpy(RESULT.m, trackedTarget.modelViewProjection, 16*sizeof(float));

    

    GLKVector3 movedVector = GLKMatrix4MultiplyVector3WithTranslation(RESULT, vector);

    

    //NSLog(@"%d, vector: %f %f", , 320./2 * movedVector.x, 640./2 * movedVector.y);

    dispatch_async(dispatch_get_main_queue(), ^{

        whiteSquare.center = CGPointMake(320./2 * movedVector.x, 568./2 * - movedVector.y);

    });

So I'm guessing this is not possible. Then I'll look for alternatives.

Hi Patrick,
Sure it's possible. The Wikitude Native SDK returns the modelView and projection matrix that are needed in a OpenGL environment to place objects in 3d space. You simply need to use and convert those values properly when applying them to a UIView subclass. I don't have any sample code right now on my machine, but could send you some snippets afterwards. 

It's just a matter of converting values from one coordinate system to a different one.

Best regards

Andreas

Any sample code would be great, I simply can't figure out the right values to use to get as accurate tracking as within opengl. If I can get this to work we'd be purchasing the sdk.

Hi Patrick,
Here is a snippet how you could convert the coordinate systems. In this case, self is a UIView subclass. Maybe this helps to get you on track.

 

// transformation is a 4x4 matrix (modelView * projection)

 

/// Convert core matrix type to iOS matrix type

    CATransform3D transform;

    transform.m11 = transformation.f;

    transform.m12 = transformation.f;

    transform.m13 = transformation.f;

    transform.m14 = transformation.f;

    transform.m21 = transformation.f;

    transform.m22 = transformation.f;

    transform.m23 = transformation.f;

    transform.m24 = transformation.f;

    transform.m31 = transformation.f;

    transform.m32 = transformation.f;

    transform.m33 = transformation.f;

    transform.m34 = transformation.f;

    transform.m41 = transformation.f;

    transform.m42 = transformation.f;

    transform.m43 = transformation.f;

    transform.m44 = transformation.f;

 

 

/// ======== Convert iOS to Core so that the core matrix can be applied correctly ========

    

    CATransform3D translateDrawable = CATransform3DMakeTranslation(-.5f, .5f, 0.f);

    drawableTransform = CATransform3DConcat(translateDrawable, drawableTransform);

    

    CATransform3D convertScale = CATransform3DMakeScale(1.f/_drawableSize.width, -1.f/_drawableSize.height, 1.f);

    drawableTransform = CATransform3DConcat(convertScale, drawableTransform);

    

    

    /// ======== Core matrix is now applied => Convert Core back to iOS ========

    

    

    CATransform3D screenScale = CATransform3DMakeScale(self.superview.bounds.size.width*.5f, -self.superview.bounds.size.height*.5f, 1.f);

    drawableTransform = CATransform3DConcat(drawableTransform, screenScale);

    

    CATransform3D screenTranslation = CATransform3DMakeTranslation(self.superview.bounds.size.width*.5f, self.superview.bounds.size.height*.5f, 0.f);

    drawableTransform = CATransform3DConcat(drawableTransform, screenTranslation);

    

    CATransform3D distanceTranslation = CATransform3DMakeScale(1.f, 1.f, -distance);

    drawableTransform = CATransform3DConcat(drawableTransform, distanceTranslation);

    

    

    // apply the converted matrix

    ;

 

 

 


Best regards

Andreas

Thanks, it did, and we've just purchased the sdk.

Had to adapt the code a bit, but in case anyone else runs into this the following code will move the UIView whiteSquare correctly (use with the native iOS tracker example):

 

- (void)baseTracker:(nonnull WTBaseTracker *)baseTracker didTrackTarget:(nonnull WTImageTarget *)trackedTarget

{

    ;

    ;

    

    

    // transformation is a 4x4 matrix (modelView * projection)

    

    /// Convert core matrix type to iOS matrix type

    

    CATransform3D drawableTransform;

    drawableTransform.m11 = trackedTarget.modelViewProjection;

    drawableTransform.m12 = trackedTarget.modelViewProjection;

    drawableTransform.m13 = trackedTarget.modelViewProjection;

    drawableTransform.m14 = trackedTarget.modelViewProjection;

    drawableTransform.m21 = trackedTarget.modelViewProjection;

    drawableTransform.m22 = trackedTarget.modelViewProjection;

    drawableTransform.m23 = trackedTarget.modelViewProjection;

    drawableTransform.m24 = trackedTarget.modelViewProjection;

    drawableTransform.m31 = trackedTarget.modelViewProjection;

    drawableTransform.m32 = trackedTarget.modelViewProjection;

    drawableTransform.m33 = trackedTarget.modelViewProjection;

    drawableTransform.m34 = trackedTarget.modelViewProjection;

    drawableTransform.m41 = trackedTarget.modelViewProjection;

    drawableTransform.m42 = trackedTarget.modelViewProjection;

    drawableTransform.m43 = trackedTarget.modelViewProjection;

    drawableTransform.m44 = trackedTarget.modelViewProjection;

    

    

    /// ======== Convert iOS to Core so that the core matrix can be applied correctly ========

    

    CGSize _drawableSize = CGSizeMake(1, 1);

    CATransform3D translateDrawable = CATransform3DMakeTranslation(-.5f, .5f, 0.f);

    drawableTransform = CATransform3DConcat(translateDrawable, drawableTransform);

    

    CATransform3D convertScale = CATransform3DMakeScale(1.f/_drawableSize.width, -1.f/_drawableSize.height, 1.f);

    drawableTransform = CATransform3DConcat(convertScale, drawableTransform);

    

    

    /// ======== Core matrix is now applied => Convert Core back to iOS ========

    

    

    CATransform3D screenScale = CATransform3DMakeScale(self.view.bounds.size.width*.5f, -self.view.bounds.size.height*.5f, 1.f);

    drawableTransform = CATransform3DConcat(drawableTransform, screenScale);

    

    CATransform3D screenTranslation = CATransform3DMakeTranslation(self.view.bounds.size.width*.5f, self.view.bounds.size.height*.5f, 0.f);

    drawableTransform = CATransform3DConcat(drawableTransform, screenTranslation);

    

    CATransform3D distanceTranslation = CATransform3DMakeScale(1.f, 1.f, 1.f);

    drawableTransform = CATransform3DConcat(drawableTransform, distanceTranslation);

    

    

    // apply the converted matrix

    dispatch_async(dispatch_get_main_queue(), ^{

        ;

        //NSLog(@"%@", whiteSquare);

    });

    

    

}

 

I just want to add for people who will use this code, that this code moves the center of the whiteSquare. So you need to make the whiteSquare like this: if you have a width w and height h for UIView whiteSquare, then set the left of the whiteSquare to -w / 2, and the top of the whiteSquare to -h / 2. So if width = 100 and height = 150, then set the left = -50 and top = -75. This seemed obvious, but just in case anybody has trouble to put this code to work, this might help.

OR, you can set the x, y, left, and top as you wish, and then don't forget to set the center to 0, 0 using this code:

whiteSquare.center = CGPointMake(150, 150);

or

;

 

@Martin, @Patrick,
THX for sharing your learnings! 

Best regards

Andreas

I was pointed in this direction as I am trying to augment UILabels into the scene.  

Would these transformations still work for UI label?  I tried using this code to augment the UIView whiteSquare, but I was not having any success.  I think it was because I was not initializing whiteSquare properly.  Where is the best place to do this?

 

Ultimately, I am trying to augment text into the scene and I am lost/confused.

Any help would be appreciated.

 

Thanks

Hi Collin,
A possible initialization point four whiteSquare could be `-baseTracker:didRecognizedTarget:`.

Once it works for the Square (I assume it's a UIView subclass), it should work the same for UILabels.

Best regards

Andreas

For testing, I ran the following code in viewDidLoad (whiteSquare is a UIButton, but the same would work for a UIView, which I initially used, before testing with buttons):

 

    whiteSquare = ;

    ;

    whiteSquare.backgroundColor = ;

    ;

    

    whiteSquare.frame = CGRectMake(0, 0, 44, 60);

    whiteSquare.center = CGPointMake(0, 0);

    ;

 

 

Thanks for the quick responses!  

I am able to get my UILabel to show up but when it is projected in the view it is way too large.  Could you possible explain the transformations that are taking place and how I can change them to make the scale much smaller?

 

Is there a way to make the same thing with Android SDK ?

I have used this trick for a iOS application and now I should make a porting to Android ...

Hi Alessandro,
As far as I know there is no way to do this with Android views. Sorry ;(

Best regards,

Andreas
Login or Signup to post a comment