Our client has tasked us to create an iPad app (to be placed in-store) and not available for download in app stores.
They create customised rings for customers and right now they are rendering photorealistic renders (Rhino) of the designs for client to sign off.
They would like customers to be able to try the 3D rings on the customers' fingers when they come to pay the deposit.
1. Besides printing a marker and sticking on the finger, any other ways we can allow them to have AR tracking on the finger?
2. What do you suggest we do to improve the realism of the gems e.g. diamonds etc. Again, we are only doing on iOS and on iPad (latest model) so perhaps we can leverage on iOS Metal graphics API support to push the real time rendering?
1. You can work with markers that you put on the fingers and work with occlusion models (For the occlusion you’d need to have a cylinder). You can also use our Plugins API to e.g. integrate a special library which allows you to recgonize fingers or otther body parts.
2. You can e.g. work with Unity and make use of the advanced Unity rendering.
Thx and greetings
In addition to what Nicola mentioned, and assuming you are using Unity, I would add the following:
1. I would also recommend using Image Tracking, since it offers the fastest and most stable tracking. The markers don't have to be square (see our extended tracking sample), so try to experiment with different shapes and sizes to find the perfect balance between tracking quality and user experience. If this is supposed to be used in the store only, the markers could even be attached to actual rings, which would make them easier to be put on and stay on the fingers. Just make sure that the marker is always flat and never bent, otherwise the tracking quality would suffer.
Additionally, in the videos you can notice that they make the markers disappear. They do this by stretching the camera texture over a cylinder placed on top of where the marker should be. In Unity, you have access to the camera texture on the GPU, so you could update the UVs of the cylinder every frame to simulate this effect, but you will need someone with a bit of graphics programming experience to achieve this. Otherwise, just use the cylinder as an occlusion model (as Nicola mentioned; please see the Object Tracking examples) to make sure that only the front of the ring is displayed.
2. The Wikitude Plugin for Unity supports Metal, so there shouldn't be any restrictions in what you can do in Unity.
For the rendering itself, you could try the Gem Shader from Unity (Asset Store, Youtube Example), which might not be the most realistic, but could probably be tweaked to achieve the expected result. Or you can look for other shaders on the asset store, but this one is free and probably a good starting point. For the metal part of the ring, the default PBR shader in Unity should be good enough, if properly configured. Adding an AO texture, especially on the rings that have an intricate design, will make the details look much better. AO can also be used between the ring and the finger, to make the it look like the ring is actually on the finger, not just rendered on top.
You can also try to experiment with the PostProcessing Stack and HDR rendering to make the rings shine in a more natural way, or add AO at runtime, but the performance on mobile might not be very good.