toggleOverlay proposal - #1439
Conversation
| 1. Set |session|'s [=XRSession/browser overlay shown=] boolean to `true`. | ||
| 1. If no other user agent UI or platform state requires the [=XRSession/visibility state=] to be {{XRVisibilityState/"hidden"}}, set |session|'s [=XRSession/visibility state=] to {{XRVisibilityState/"visible-blurred"}}. | ||
| 1. Abort these steps. | ||
| 1. Otherwise, [=Shut down the session|shut down=] |session|. |
There was a problem hiding this comment.
WIthout a way to feature detect support this feels like a bit of a footgun. At minimum I think if we want this behavior we need to add a member boolean that can indicate support; but I wonder if returning a bool (that indicates if toggling was succesful) or a promise (bool is true for shown, false for hidden and rejected if overlay couldn't be toggled), would also be acceptable.
There was a problem hiding this comment.
The developer will call this API if they want the user to enter some information on the 2D web page. I can't think of a scenario where they wouldn't want to exit WebXR if overlay is not supported.
There was a problem hiding this comment.
Maybe they can work around it, or if they can summon their own keyboard maybe they can pipe the data through? If it's payment, maybe there's some improvements that could be made to the payments API, etc.
Requesting a feature leading to an exit without a way to validate just feels like a massive footgun to me; maybe the reason the page wanted the overlay was optional to begin with.
There was a problem hiding this comment.
If they go through the trouble of building an alternate input systen, they will likely choose that as their only option.
If we do NOT exit, developers will just assume it worked since they only test on Quest which leaves a broken workflow on other platforms.
There was a problem hiding this comment.
In thinking about it a bit more, I think I'm not opposed to failure meaning you end the session (since we have the event you should be listening for), but I do think we should have a way to feature detect. At the very least, maybe you want to switch to display a message and a button or a different message if you know the overlay won't work.
Add a note that browser overlays can display cross-origin content, unlike the HTML-in-Canvas API. Require XRSession.toggleOverlay() to be invoked with transient activation and throw SecurityError otherwise.
|
/agenda toggleOverlay proposal |
|
|
||
| 1. Let |session| be [=this=]. | ||
| 1. If |session|'s [=XRSession/ended=] value is `true`, throw an "{{InvalidStateError}}" {{DOMException}} and abort these steps. | ||
| 1. If |session| is not an [=immersive session=], throw a "{{NotSupportedError}}" {{DOMException}} and abort these steps. |
There was a problem hiding this comment.
NIT: I think redundant with the line that this feature shouldn't ever be granted to non-immersive sessions above?
There was a problem hiding this comment.
Is that listed somewhere else in this algorithm?
There was a problem hiding this comment.
Not in this one, but it's in the spec text you have about when the feature should be granted. I don't think we have algorithms for enabling most of those features though. I'm fine with leaving this line in but just wanted to raise the potential (IMO) redundancy
|
This seems reasonable to me with these changes as we discussed (and a couple of minor nitpicks). I'd like to allow the others to weigh in though. |
mwyrzykowski
left a comment
mwyrzykowski
left a comment
There was a problem hiding this comment.
What are the use cases for this? Payments are mentioned, but all major browsers already support the payments API which seems preferable to opening a browser window. E.g., some UAs may open native system UI for payments instead of the entire browser.
It would be good to understand what the goals are here and if they are already achievable via existing web APIs. Opening a browser window seems like it does not exactly fit with the rest of the WebXR API.
FWIW Quest browser does not support payments. Is there any browser that supports payments inside of a WebXR session?
The goal is to have a 2d page that can potentially open another tab or display third party content. |
I don't think so but support couldn't be added if that was needed.
Specifically Apple is wondering, what content would be displayed that is relevant to an immersive session? It seems a bit out of place with the rest the WebXR API to be showing browser windows. We are wondering if all content desired could be displayed via existing web APIs? |
It's very hard to do so which is why it hasn't happened yet. It unclear if users can trust UI that is displayed inside an immersive session.
It's mostly convenience + the ability to show third party content
|
TBH, I think HTML in canvas could likely handle the last 2 and maybe even the last 3. Though I guess cross-origin libraries would be banned? |
Sure, it could do those things in VR but it's a lot easier to just do it in 2D especially if your site also supports a 2d experience. |
|
If you just smack a canvas on a quad texture can you not basically create a 2D page with HTML-in-Canvas? It's just restricted to your origin? |
Not right now because we have not defined hit testing with html-in-canvas |
| undefined updateRenderState(optional XRRenderStateInit state = {}); | ||
| Promise<undefined> updateTargetFrameRate(float rate); | ||
| [NewObject] Promise<XRReferenceSpace> requestReferenceSpace(XRReferenceSpaceType type); | ||
| undefined toggleOverlay(); |
There was a problem hiding this comment.
I think this might need to be a promise and we update the algorithm to reject if it's still changing/pending. I'm looking into if we could implement this, and even if we have a path that's not the suspend/resume session bit, I think it's going to be async for us.
And payments is possible with existing web APIs, UAs would just need to support it. It would be their choice to exit the immersive session or remain within it. It doesn't seem like there is a compelling use case to justify adding this to the WebXR specification at this time |
Not all sites want or can use the web payments API.
I don't quite follow. This is something developers have asked for. Why would we not let them use the browser if it's available? It seems anti-web that we force them to do everything in script. |
To me that's more an argument for us working to try to solve that than implement this, but l do agree that payments is the big workflow that this enables that html-in-canvas doesn't, and maybe there's others. There's also the simplicity argument. I think @mwyrzykowski was implying that the UA could override the payments implementation if there's a WebXR session to create a way to show it within the WebXR session if they so desired, which achieves your goal. |
Exactly. There doesn't seem to be any reason to add API surface to WebXR here. |
I disagree that there is no reason at all to add this. |
|
@cabanier Apple discussed a bit would like to understand the use cases as the justification for adding this is not obvious. For instance, why can't the sites use the payments API? Are there other use cases which are not possible with existing web APIs and, if so, which ones? |
|
FWIW, I think this may be better discussed in a meeting at this point? /agenda |
|
As developers working on a large WebXR application, I'd like to add a real-world data point in support of @cabanier's proposal. This would genuinely help us, and the alternatives raised in this thread don't cover our case. HTML-in-Canvas is the closest fit, but its lack of cross-origin support is a hard blocker for us. The 2D content we need to surface mid-session is almost entirely cross-origin:
None of these can render in-canvas. They also depend on the browser's own form autofill, which in-canvas can't provide. On top of that, being able to show our existing legal and support pages as-is, instead of rebuilding them in 3D, is a large practical saving. Payment Request API / UA-handled payments doesn't cover our case either: our hosted-checkout and 3-DS flows aren't expressible through it even where it's fully supported. Its unavailability on Quest, where the overwhelming majority of our users are, is secondary; even with it, these flows still require rendering cross-origin web content. I think the skepticism here is fair for a generic 3D game: tutorials, lobbies, and similar content can reasonably live in-engine. But for a transactional, cross-origin-heavy, compliance-heavy application, It's worth being precise about why leaving the session is the wrong model here, not just an inconvenience. In practice the user is still mid-experience: this flow continues a session they already consented to. But the only path available today forces us to end that session and start a new one, gated by the user activation that correctly governs any new session entry. That round trip is also a full teardown and cold start: reference spaces, layers, GPU resources, and session-scoped media and connections are all lost, then rebuilt. That cost is engineering-manageable, but what no amount of engineering recovers is showing this cross-origin content without leaving the session at all. This would close a real gap for us. |
|
As a game developer with millions of players in WebXR, @Ledzz comments above are right on the mark. Digital sales funnels are extremely difficult to create even in ideal circumstances, and friction has to be minimized at every step. Every extra click added to the process, you lose about half your remaining audience, if you're lucky. Even the simplest possible hypothetical sales funnels in WebXR now involves way too many steps and flow-breaking, bad UX. Needs to be native-like levels of payment friction (or better) for web developers to compete. I don't imagine this proposal is a panacea for this problem, but it would significantly cut down on friction and potentially make IAP a realistic possibility for my games, which til now it has not been, because conversion would be so low due to the current required UX. Further, as a game designer, I can imagine many reasons why you'd want cross-origin 2D browser access within an immersive session inside trusted UI. Many game design-y ideas that we can't even speculate on here, or more practical reasons - "Login with Apple" mid-session for persistent data, for example. So I'm in favor of this proposal overall. Thanks for considering! |
|
/agenda discuss toggleOverlay after developer feedback |
|
I discussed with the standards engineers and these are great use cases but we think they should be solved via targeted web APIs. E.g., 3-D Secure / bank challenge flows without leaving the session
Identity / KYC verification without leaving the session
Third-party authentication without leaving the session
|
I noticed that this is only implemented in Chrome and not in any other browser. It's also unsupported in Quest.
This has a similar concern with trusted UI.
This proposal got negative feedback when it was proposed as an experiment in Blink. |
|
Re: "Being sure they are dealing with trusted UI", why do these mechanisms have those concerns when showing the browser window doesn't? |
mwyrzykowski
left a comment
mwyrzykowski
left a comment
There was a problem hiding this comment.
Speaking with Apple's web standards engineers, we think showing a browser window is the incorrect approach. Instead, we should work with other W3C groups to achieve these goals.
Why is it the incorrect approach? It solves many problems in a lightweight solution. |
The pushback is these use cases are applicable outside of WebXR and if the existing web APIs cannot achieve such use cases, we should work with the corresponding WG. I.e., WebXR should not be concerned with API surface to solve use cases involving payments or verification. That is the responsibility of other WGs. Additionally, there isn't an existing precedent for W3C APIs to show the system browser. The proposal as-is seems to sidestep working with the appropriate WGs which own payments, credential verification, etc instead of working with them. |
|
Again, not entirely sure which side I fully come down on, but:
I think @cabanier's point is that there are existing Windowing/redirect/popup mechanisms that enable this today outside of a WebXR session, without the need for additional APIs, and that's essentially what he's proposing to re-enable here.
I also assert that outside of our working group fullscreen is really the only other precedent for fully taking over the rendering of your monitor/view, and even that is different than an XR session, since XR Sessions come with an expectation of interactivity that fullscreen doesn't really have, so by our very nature the immersive web specs are a bit unprecedented for some of these scenarios. I am certainly sympathetic to the argument though that we should hook into existing APIs where they exist, and are shipped, to make those work within an immersive session, but I also think a UA would be well within their rights to address those APIs by popping the browser window up inside an immersive session if they so chose to service those APIs. |
|
@cabanier I also haven't seen an answer to my question re: trusted UI? |
I'm sorry I missed your question! |
6 participants
Preview | Diff