Start a new topic

Get target image from camera in Unity

Get target image from camera in Unity

1 person has this problem

Hi,


Probably the best way to access the camera information is to use our Plugin API. Please check the Barcode / QR Scanner sample that we provide for an example of how to use it.

Additionally, to temporarily disable tracking, but still keep the camera running, simply disable the tracker in the scene.


Best regards, 

Alexandru

Hi,


We are using Aruco codes to spawn objects in our scene while the Wikitude tracking is happening. Our first approach after seeing that the wikitudeCamera.CameraTexture wasn't working was to use the WebCamTexture, but that made Wikitude to stop working at all. Then we discovered that the Enable Camera Rendering needed to be enabled and that a Camera had to be attached to the MiraARCamera object for the wikitudeCamera.CameraTexture to work. After doing that, we saw that the format of the Texture2D being recorded by the wikitudeCamera.CameraTexture wasn't allowing us to use Texture2D.GetPixels32() so we used the code from the top of this page posted by Alexandru to get the right format and that made it possible for everything to work together. The only thing that some other people have mentioned on this thread and that I mentioned in my previous post is that the Background Camera is constantly rendering, causing unnecessary workload. My question is if there would be any other way of accessing the camera feed other than the one mentioned or if not, if there would be a way of enabling/disabling the Wikitude tracking altogether so that users choose when to scan Aruco codes and while that is happening, tracking is disabled to lower the workload.


Thank you,

Aniol

Hi, 


Could you please describe what your use case is and how exactly you're trying to achieve that. Perhaps we can find a different solution or workaround.


Thank you,

Alexandru

I can confirm what Justin said about destroying/disabling the camera not working on iOS, then you go back to having the wikitudeCamera.CameraTexture as a white texture. Has anyone figured out a way of making this more efficient? The fact of having the Camera Rendering enabled lowers performance quite a bit. Thanks.

In case anybody is trying to do this, the earlier method doesn't work -- at least on iOS, wikitudecamera.cameraTexture doesn't get updated unless the BackgroundCamera's camera is present and active (as soon as you disable the camera, cameraTexture no longer updates). So I had to just send that camera backwards in depth so I could draw over it. This works differently in the editor, which made this a huge pain to debug. 


And I was only able to do that because of the comment above that happens to mention the hidden gameobject. Not sure why that's hidden -- only makes working with wikitude harder. 

Hi, 


There currently isn't a proper way to do this.


However, the background is rendered by a hidden object named "BackgroundCamera", so you could simply destroy it and rendering will also stop. This object is created during the Start method of the WikitudeCamera, so make sure to wait for that before searching for the BackgroundCamera. Also make sure to enable clearFlags on your main camera, as they are set to DepthOnly by default.


Here is a short snippet of how this can be done:


IEnumerator Start() {

    yield return null;

    Destroy(GameObject.Find("BackgroundCamera"));

    Camera.main.clearFlags = CameraClearFlags.Color;

}


Best regards,

Alexandru

Hi! is there a way to render the camera texture to a RawImage without Wikitude also rendering the camera feed in the background?

 

I can render to a custom image just fine with rawImage.texture = wikitudeCamera.CameraTexture; but only as long as the WikitudeCamera component has Enable Camera Rendering set to true in the Editor, which makes it render the texture to the background as well. If I set it to false I get no background but also no texture (white image) to render to the RawImage.

 

Is this even possible (outside of covering the background with something or doing ugly hacks)?

 

Thanks!

My mistake. 

I updated the previous post.


Best regards,

Alexandru

The link just takes me back to the forum home?

Hi.


I apologize for the late reply.


Here is a snippet of code that captures the frame and saves it on disk

 

using UnityEngine;
using System.Collections;
using Wikitude;
using System;
using System.IO;

public class ScreenshotCapture : MonoBehaviour {

	private WikitudeCamera _wikitudeCamera;

	private RenderTexture _targetRenderTexture;
	private Texture2D _cameraFeed;

	void Start () {
		_wikitudeCamera = FindObjectOfType<WikitudeCamera>();
		if (_wikitudeCamera == null) {
			Debug.LogError("No WikitudeCamera found!");
			Destroy(this);
		} else {
			CreateTextures();
		}
	}
	
	private void CreateTextures() {
		_targetRenderTexture = new RenderTexture (Screen.width, Screen.height, 24);
		_cameraFeed = new Texture2D (Screen.width, Screen.height);
	}

	void Update () {
		if (Input.GetKey(KeyCode.Space) || Input.GetMouseButton(0)) {
			string path = Path.Combine(Application.persistentDataPath, "wikitudeFeedImg_" + System.DateTime.Now.ToString("hh_mm_ss") + ".png");
			Debug.Log("path: " + path);
			StartCoroutine(CaptureScreenshot(_wikitudeCamera, path));
		}
	}

	private IEnumerator CaptureScreenshot(WikitudeCamera wikitudeCamera_, string path_) 		{
		yield return new WaitForEndOfFrame ();
		
		if (_targetRenderTexture.width != Screen.width || _targetRenderTexture.height != Screen.height) {
			CreateTextures();
		}

		Graphics.Blit(wikitudeCamera_.CameraTexture, _targetRenderTexture);
		RenderTexture.active = _targetRenderTexture;

		_cameraFeed.ReadPixels(new Rect(0, 0, _cameraFeed.width, _cameraFeed.height),0,0);
		_cameraFeed.Apply();

		byte[] bytes;
		bytes = _cameraFeed.EncodeToPNG();
		File.WriteAllBytes(path_, bytes);
	}
}

 


Best regards,

Alexandru.

So after a lot of digging (on Android), I have a hunch that WikitudeCamera.CameraTexture is not a 'proper' Texture2D or it's in the wrong format or something else entirely. I was able to render the video feed to a rawImage on the UI by casting WikitudeCamera.CameraTexture to a Texture. (eg rawImage.texture = (Texture)WikitudeCamera.CameraTexture;) which is what Wikitude do with the BackgroundCamera that renders the feed to the screen.


I don't know why this is the case and would hope Wikitude will be able to shed some late on this.

I've been having the same problem as @m0j1 42420. WikitudeCamera.CameraTexture is the right size but is just white when copied to a rawImage.

Hi Alexandru,
Thanks to your instructions I could reach what I had in mind , but the problem is I can't do it with Wikitude.

The problem is I couldn't get Wikitude Camera feed. I tried WikitudeCamera.CameraTexture and also tried to use RenderTexture. But none worked for me and I get a blank Texture.

I'll appreciate if you can help me with this.

Thanks very much
Best Regards

Hi Alexandru,
Thanks very much. I wanted to get back to this project and work on it a bit more but I couldn't this time download the script you mention (maybe because of forum update).

I'll appreciate if you can kindly upload it again.

Thanks very much
Best Regards


Hi,

Here is a quick script I made as proof of concept of how this problem can be approached. It relies on dummy transforms attached to the corners of the targets, but these can also be generated automatically if you know the aspect ratio of the target you are tracking. This information is not currently available in the Unity API, but we will see if it can be added in the future. Until then you need to do a bit of bookeeping for this. When the image is tracked, the positions of the corners are transformed into screenspace coordinates. I then update a quad with UVs created from these coordinates and display it. The script needs to respond to OnEnter and OnExitFieldOfVision events from the Trackable to work correctly.

There are some downsides to this approach. The mesh update has a bit of lag so the cropped image is a bit behind the feed and there is some warping at extreme angles that could be improved with a more dense mesh, but I hope this will be a useful starting point. If you have any questions, please let me know.

Best regards,
Alexandru

1 person likes this
Login or Signup to post a comment