Start a new topic

Viewer interferes with embedded video on rest of app

I am using the wikitude cordova plugin (see stack below). When the app first starts, HTML videos run with no problem from both local and remote files.


However, once the viewer has been opened and closed, returning to the initial webview shows that any of the videos that were running are now frozen, and trying to open new videos results in empty video elements, with the javascript error promise reading: "AbortError: The operation was aborted."


This error message will appear for files that had loaded and run successfully before the viewer was opened, and/or in previous sessions.


This makes me think the viewer is interfering with background video activity.


*I cannot even create/load entirely new video elements once the viewer has run, although I can before.


Can you advise on this?


Thanks,

Neal


*We have a way to record a video feed from the viewer, but cannot show it to the user immediately after they close the viewer, only when they close/reopen the app.


3 people have this problem

Hi Neal,


Could you please provide the following details:


  • Which version of the SDK are you using?
  • What Cordova version are you using?
  • What device does this happen with (model details and OS version)?
  • Is this happening with the sample app or in your own app? If it happens with your own app, do the video samples of the sample app work on your device? If so, please check for differences in your implementation compared to the sample app (especially with the background handling)
  • Could you send exact steps to reproduce the issue also with the sample app?


Thx and greetings

Nicola

My apologies, here is a more elaborate run down of problem:


*I understand your side may say, 'if it works on our sample, we are okay'. But maybe someone on iOS side can take a quick look at it.



Problem:

######################################################################

My app runs on Ionic and uses the Wikitude Augmented Reality plugin (cordova plugin).


When the app opens the Wikitude viewer, and then the closes it to return to the main app, I can no longer play HTML5 videos.



Before using the viewer, I can continually spin up (javascript) video elements and play them, then destroy and recreate.



It looks like the (problem/after using viewer) videos never successfully load:

Tried ‘onload’ event to console.log back, it never fires Tried ‘cache-buster’ urls, both for local (sanitized) url’s and external url’s Tried ‘.pause() —> .load() —> .play()’ sequence on video object to no avail Tried setting ‘.currentTime’ equal to 0, after .load(), then .play() to no avail

 


I can load jpg’s to <img> tags in the same scenario, even with cache-buster url’s. So loading content is possible.



This issue does NOT arise on the Wikitude sample app, which runs Cordova w/ simple JQuery (no Ionic). 

I can use the Wikitude viewer, then close it and return to the main app, and spin up HTML5 videos with no problems.



I think this problem is due to the cordova-plugin-ionic-webview

I think that the Wikitude Viewer interferes with this perhaps because either:

It doesn’t create a child-view in proper hierarchy in iOS, making videos play, but in viewer layer out of sight Wikitude Viewer connects the HTML body.background to a live video feed from camera, perhaps this locks up video resources for entire app


######################################################################





Demo Code:

######################################################################

 startVideo(videoUrl) {


 //YOU WILL NEED TO COPY A LOCAL FILE TO YOUR SYSTEM TO TEST (OR TRY REMOTE URL, THOSE BREAK ALSO)

 videoUrl = 'assets/videos/test_movie.mov';


 let videoForRemoving = <HTMLVideoElement> document.getElementById('videoEl');

 if (videoForRemoving) {

 videoForRemoving.remove();

 }


 let video = <HTMLVideoElement> document.createElement('video');

 video.id = 'videoEl';

 let testMov = videoUrl;


 video.setAttribute('src', testMov);


 //YOU WILL NEED TO ADD '<div id="fileExplorer"></div>' TO CURRENT VIEW’S HTML

 let fileExploreEl = document.getElementById('fileExplorer');

 fileExploreEl.insertBefore(video, fileExploreEl.firstChild);



 video.load();

  video.currentTime = 0;

  video.play();


 }

######################################################################







Tech Stack:

######################################################################

Ionic:


   Ionic CLI                     : 5.2.1 (/Users/nealbrennan/.nvm/versions/node/v10.16.0/lib/node_modules/ionic)

   Ionic Framework               : @ionic/angular 5.3.0

 @angular-devkit/build-angular : 0.901.12

   @angular-devkit/schematics    : 9.1.12

   @angular/cli                  : 9.1.12

   @ionic/angular-toolkit        : 2.3.0


Cordova:


   Cordova CLI       : 9.0.0 (cordova-lib@9.0.1)

 Cordova Platforms : ios 5.1.1

   Cordova Plugins   : cordova-plugin-ionic-keyboard 2.2.0, cordova-plugin-ionic-webview 5.0.0, (and 6 other plugins)


Utility:


 cordova-res : not installed

   native-run  : 0.2.7 (update available: 1.0.0)


System:


 Android SDK Tools : 26.1.1 (/Users/nealbrennan/Library/Android/sdk/)

   ios-deploy        : 1.10.0

   ios-sim           : 8.0.2

   NodeJS            : v10.16.0 (/Users/nealbrennan/.nvm/versions/node/v10.16.0/bin/node)

   npm               : 6.9.0

   OS                : macOS Catalina

   Xcode             : Xcode 11.5 Build version 11E608c


Plugins:


 com.wikitude.phonegap.wikitudeplugin 9.2.0 "Augmented Reality - Wikitude SDK Plugin"

 cordova-plugin-device 2.0.2 "Device"

 cordova-plugin-file 6.0.2 "File"

 cordova-plugin-insomnia 4.3.0 "Insomnia (prevent screen sleep)"

 cordova-plugin-ionic-keyboard 2.2.0 "cordova-plugin-ionic-keyboard"

 cordova-plugin-ionic-webview 5.0.0 "cordova-plugin-ionic-webview"

 cordova-plugin-splashscreen 5.0.2 "Splashscreen"

 cordova-plugin-statusbar 2.4.2 "StatusBar"

 cordova-plugin-whitelist 1.3.3 "Whitelist"

######################################################################

I have created two repositories from the Wikitude Cordova Samples Repo.


It turns out the Samples repo does not break web services on the parent app, but it also does not use WKWebview. The cordova app uses it's default UIWebview, which cannot be pushed to App Store.


Simply adding the cordova WKWebview plugin to the Sample project makes it unable to play videos (after the view has been used).


Here are two demo's for this, one with and one without the WKWebviewer. Instructions will be on the screens when you open these:


https://github.com/NealBrennan75/WikitudeVideoBugDemo_WKWebview

https://github.com/NealBrennan75/WikitudeVideoBugDemo_UIWebview


3 people like this

Hi there

Did you found any workaround yet? I've exactly the same problem :-/

We are having the same issue as well. Our new app launch is end of September. May have to move on to another AR provider if they dont fix this bug soon. Let's go team Wikitude.


1 person likes this

Hi,


Sorry for the inconvenience. This issue seems to be related the use of the deprecated UIWebView by Cordova in their previous release. We already removed all references to UIWebView from our SDK and all extensions in our 8.9 release and exclusively use WKWebView since then.


The Cordova 9.0 library includes a plugin named CDVUIWebViewEngine, which contains references to UIWebView. They very recently released their 10.0 version, which replaces the previous plugin with CDVWebViewEngine. As expected, this updates all their UIWebView references with WKWebView.


Our current plugin should already be compatible with the new Cordova release. Are you still having the same issue when building your projects with Cordova 10.0?


If that's the case, please specify the device and iOS version that you are using.



- Damian.

We are still having this issue even after upgrading to Cordova 10.0. One thing to note, when we build the app and upload it to iTunes Connect, we still get a warning email from Apple indicating that something in our app is still using UIWebView. Trying to figure out what plugin that might be.

We have tested on just about every iPhone model from 5s to 11. Definitely not a model specific issue. We have also tested on iOS version 13.5, 13.6 and 13.6.1.

This is our current Cordova setup:

Cordova Packages:

    cli: 10.0.0
        common: 4.0.2
        create: 3.0.0
        lib: 10.0.0
            common: 4.0.2
            fetch: 3.0.0
            serve: 4.0.0

Project Installed Platforms:

    android: 9.0.0
    ios: 6.1.0

Project Installed Plugins:

    com.moust.cordova.videoplayer: 1.0.1
    com.wikitude.phonegap.wikitudeplugin: 9.2.0
    cordova-custom-config: 5.1.0
    cordova-plugin-androidx-adapter: 1.1.1
    cordova-plugin-androidx: 2.0.0
    cordova-plugin-app-version: 0.1.9
    cordova-plugin-camera: 4.1.0
    cordova-plugin-cocoapod-support: 1.6.2
    cordova-plugin-crop: 0.3.1
    cordova-plugin-device: 2.0.2
    cordova-plugin-file: 6.0.2
    cordova-plugin-filepath: 1.5.8
    cordova-plugin-geolocation: 4.0.2
    cordova-plugin-googleplus: 7.0.2
    cordova-plugin-ionic-keyboard: 2.2.0
    cordova-plugin-ionic-webview: 5.0.0
    cordova-plugin-media-capture: 3.0.3
    cordova-plugin-sign-in-with-apple: 0.1.2
    cordova-plugin-splashscreen: 5.0.2
    cordova-plugin-statusbar: 2.4.2
    cordova-plugin-streaming-media: 2.3.0
    cordova-plugin-whitelist: 1.3.3
    cordova-sqlite-storage: 5.0.1
    cordova.plugins.diagnostic: 6.0.1

Environment:

    OS: macOS Catalina 10.15.6 (19G73) (darwin 19.6.0) x64
    Node: v12.16.3
    npm: 6.14.8

This is our current Ionic setup:

 

Ionic:

   Ionic CLI                     : 6.10.1 (/usr/local/lib/node_modules/@ionic/cli)
   Ionic Framework               : @ionic/angular 5.2.3
   @angular-devkit/build-angular : 0.901.11
   @angular-devkit/schematics    : 9.1.11
   @angular/cli                  : 9.1.11
   @ionic/angular-toolkit        : 2.3.0

Cordova:

   Cordova CLI       : 10.0.0
   Cordova Platforms : android 9.0.0, ios 6.1.0
   Cordova Plugins   : cordova-plugin-ionic-keyboard 2.2.0, cordova-plugin-ionic-webview 5.0.0, (and 19 other plugins)

Utility:

   cordova-res (update available: 0.15.1) : 0.14.0
   native-run                             : 1.0.0

System:

   Android SDK Tools : 26.1.1 (/Users/eduardomoreno/Library/Android/sdk)
   ios-deploy        : 1.9.2
   ios-sim           : 8.0.2
   NodeJS            : v12.16.3 (/usr/local/bin/node)
   npm               : 6.14.8
   OS                : macOS Catalina
   Xcode             : Xcode 11.6 Build version 11E708

 I'm going to figure out what is still using UIWebView and will get rid of it. Hopefully, that will resolve the issue. I just find it strange that everything works fine until the user launches the AR world. Once they have been in the AR world, videos are no longer viewable. It makes no difference if you shut down the AR world or just pause it.

The plugin cordova-plugin-googleplus was still using UIWebView and that is why we were getting those warnings from Apple. I ended up removing that plugin and all UIWebView warnings went away. However, even after removing all use of UIWebView within our app, the issue with no video playback after loading the Wikitude AR world remains.

Sorry about the delay. We are looking further into this right now.


Once we identify the root cause we will issue a fix or temporary workaround.


Will try to get back to you about this before the end of the week.



- Damian

Hi,


We are still investigating. The source of the problem could be in the plugin transitions to and from the controller that holds the ArchitectView. The ArchitectView itself seems fine, its structure shouldn't be causing this sort of issues and these also don't appear in other extensions.


As a temporary workaround, you could generate all the video elements you needed before loading the Architect world. Set them to hidden by default and display/hide them as required through the app logic. I'm aware this is inconvenient and has its limitations, but it can work for the time being.


We will keep you updated on further progress.



- Damian

Thanks Damian for the update. Do you feel at this point you may have something next week that will fix the issue? Any timeframe to pass along to the group?


We have a launch scheduled for Oct and would rather not implement any workarounds as you suggested above. We have other items we are currently working on in our app for the Oct launch.

Hi,

This issue is more deeply rooted than expected, finding a proper solution it's taking longer than we anticipated at first.

We are aiming to have it patched by the next release, but I'm afraid I can't confirm a specific date for the fix at the moment.

I understand you have your deadlines. I can only say that we are working on it, my apologies once again.


- Damian

Hi,


We can confirm from our end that this issue is specific to iOS 13. Our SDK works as expected on iOS 12 and 14.

This is quite likely due to a bug in the internal WebKit version that Apple uses in iOS 13, which was fixed at some point by the WebKit version used in 14.


We are still investigating for a way to work around it on the affected devices. If there is further progress on our end, we will announce it.


For now, just letting you know that this feature will work for any customers that are currently keeping their device's OS up to date.



- Damian

Login or Signup to post a comment