Apple has reportedly acquired U.K. special effects studio IKinema, a startup that may be useful in Apple’s quest to bolster its mobile devices with AR special effects and in its more far-flung attempts to enter the AR/VR headset market.
The company issued its standard confirmation for the deal to TechCrunch “Apple buys smaller companies from time to time, and we generally don’t discuss our purpose or plans.” The news was first reported by FT after rumors were floated by MacRumors. .
IKenema has used motion-tracking work to live-animate the bodily movements of digital characters, but the team has also stockpiled this information to create realistic models of movements of digital characters in digital environments, specifically in the context of games and virtual reality titles.
These models were highlighted in the startup’s RunTime product that integrated into game engines like Epic Games’ Unreal Engine. RunTime was powering avatar interactions in experiences like The Void’s “Star Wars: Secrets of the Empire,” a virtual reality experience at Disney resorts, as well as in works by studios like Capcom Linden Lab, Microsoft Studios, Nvidia, Respawn and Square Enix.
RunTime being used in Impulse Gears’ PSVR game, Farpoint
The company’s Orion product allowed motion capture with lower-cost input, essentially syncing limited input like head and hand movement with motion models that could allow for a hybrid solution that still looked realistic. The technology was being used for visualizations by teams at NASA, Tencent and others.
What does Apple want with this company?
There are a handful of spots that this tech could prove helpful, the most obvious of which would be bringing special AR effects to the iOS camera, juxtaposing the spatial data that the camera can gather from a real-world space with digital AR models. This could theoretically allow something like an AR figure to walk up your stairs or sit on a chair. The issue iKenema doesn’t solve in these scenarios is computer vision segmentation to be able to tell what surface is a table versus a floor versus a couch cushion, but enabling digital models to interact with these spaces is a big advance.
Well, a little bit more of a reach for Apple would be using this tech in the context of a VR or AR avatar system. And although iKenema worked a lot with motion capture, they did so with the explicit purpose of designing models for digital humans to interact with digital environments in real time. Their solution was already being used by virtual reality developers to give VR gamers a way to visualize how their own bodies moved in VR given limited inputs.
Facebook Horizon’s legless avatars
Virtual reality systems typically only know the location of your head and hands, given trackers are on the controllers and headset, but IKinema’s solution allowed developers to make the rest of users’ in-game bodies look more natural inside games. This is a pretty difficult challenge, and it’s the reason plenty of VR titles have made avatar systems that are missing legs, necks, arms and shoulders, because everything looks awful if the movements are off.
Apple’s computer vision needs are only heightening as they bowl forward on AR and VR devices while also aiming to bolster the camera on the iPhone as a point of differentiation from Google and Samsung devices.
Peter Jackson’s next six-hour epic is finally coming out this year—and in a first for the acclaimed director, the film will launch directly to a streaming service. It will also be broken up into episodes.
The Beatles: Get Back, an expansive documentary originally announced for a theatrical run this August, has had its release strategy tweaked. On Thursday, Jackson and Disney confirmed that the entire project will launch exclusively on Disney+ during this year’s American Thanksgiving holiday. Each third of the documentary will launch on the streaming service on November 25, 26, and 27. As of press time, Disney hasn’t said how the film will reach audiences outside of Disney+’s supported territories. Neither Jackson nor Disney clarified how the original theatrical run might have worked or whether the global pandemic forced anyone’s hand.
Today’s news confirms that Jackson had an abundance of footage to work with. Roughly three years ago, the remaining Beatles handed him access to a musical holy grail: over 60 hours of previously unseen video recordings, mostly capturing the Beatles working on the album Let It Be and rehearsing for, and then performing, the band’s legendary 1969 rooftop concert in London.
Jackson stitched the footage together with access to what Disney calls “over 150 hours of unheard, restored audio”—meaning yes, somehow Apple Corps. still has some tapes in hiding after this many Beatles special edition albums, anthologies, video games, and Cirque du Soleil collaborations. For further context on the Let It Be recording sessions, the film will be paired with a physical book full of photos and original interviews, now delayed to an October launch.
Jackson’s comments in today’s news, as provided by Disney to members of the press, imply that he indeed sought to release a long documentary: “I’m very grateful to the Beatles, Apple Corps., and Disney for allowing me to present this story in exactly the way it should be told.” He also commented on the original documentary footage, filmed by Michael Lindsay-Hogg, as something that is “not nostalgia—it’s raw, honest, and human.”
The Beatles: Get Back will launch on a Thursday, thus breaking Disney+’s latest initiative of launching new series episodes on Wednesdays instead of Fridays. If anyone can break a newly sacrosanct Disney+ rule, it has to be the Beatles.
Listing image by The Walt Disney Company / Apple Corps / Wingnut Films
This week, Microsoft and Bethesda confirmed that Starfield will be coming exclusively to Xbox Series X/S and PC next year. And while that kind of exclusivity deal had been hinted at and heavily suspected by many since Microsoft’s $7.5 billion acquisition of Bethesda’s parent company, the announcement still came as sad news for PlayStation 5 owners hoping to play the upcoming space epic.
Bethesda Senior Vice President of Marketing and Communications Pete Hines said he can certainly understand how PS5 owners must feel. In a video interview with GameSpot Wednesday, he offered his sympathy and an apology to PS5 owners upset about the move.
“I don’t know how to allay the concerns of consumer and PlayStation 5 fans other than to say I’m a PlayStation 5 player as well and I’ve played games on that console and there’s games I’m going to continue to play on it,” Hines said. “But if you want to play Starfield [it’s] Xbox and PC. Sorry. All I can say is I apologize because I’m certain that that’s frustrating to folks, but there’s not a whole lot I can do about it.”
Look on the bright side
At the same time, Hines seemed to suggest that Xbox Cloud Gaming could provide a way for players who don’t have an Xbox or a gaming PC to access Starfield through the “Xbox ecosystem.” Hines mentioned that Xbox chief Phil Spencer “has talked about how they’re looking to expand that and… looking to bring Xbox games to folks who don’t own a Series S or X or even a PC but want to play the games that we’re bringing to Game Pass.” That’s an apparent reference to Microsoft’s recently announced plans to expand Xbox Cloud Gaming to many Smart TVs and generic web browsers like Chrome, Safari, and Edge.
Hines also said Starfield‘s exclusivity could help its developers focus on the gameplay experience rather than compatibility with additional platforms. “I’m here to tell you, and any [developer] will tell you this, [when] you go to fewer platforms, your development gets more streamlined,” he said. “You’re not worrying about, ‘Well, how does it work on this box versus how does it work on that box…’ We’re not making it on that box, so it just needs to run as well as possible on this one [and] on a PC. A narrow focus always helps…”
That statement echoes comments Bethesda producer Todd Howard made to The Telegraph this week, when he said that “by focusing on those platforms [Xbox and PC], you really get to lean in a lot on making it the best it can be for those systems.”
In that interview, Howard also said he had “a little bit” of reservation about not having a PS5 version of Starfield. “You don’t ever want to leave people out, right?” But Howard also suggested that Xbox Cloud Gaming means “we see it actually opening up more and more and more so that people’s ability to play our games—via Game Pass and other things—their ability to play our games doesn’t go down. It goes up dramatically… I will just say I want everybody to have the ability to play it in some fashion.”
Back in October, Howard said it was “hard to imagine” a game like the upcoming Elder Scrolls VI being exclusive to Microsoft’s platforms. But that was before Microsoft’s acquisition of Bethesda was finalized in March, at a time when Howard admitted the two companies hadn’t fully discussed the details of any multiplatform publishing agreements. “We haven’t gone through all of that, to be honest,” he said at the time.
Everything we’ve feared about the Facebookening of Oculus and its virtual reality ecosystem is starting to come true.
A Wednesday blog post has confirmed that Oculus, the VR-specific arm of Facebook, is now displaying advertisements in select VR games and apps to their players. As Facebook has since emphasized in emails sent directly to the press, these ads will leverage “first-party info from Facebook to target these ads”—and FB has yet to announce any limitations for what Facebook account data may be leveraged. (Ars Technica was not briefed about this news ahead of the announcement, and we did not get the opportunity to request the comments that other members of the media received.)
FB’s additional clarifying statements about biometric and use data inside of VR are carefully worded to clarify that the company does examine specific use data as it sees fit, and for now, that data won’t apply to its new advertising platform. Facebook says it processes and keeps track of the following data, uploaded by users while connected to any Oculus services:
“Weight, height, or gender information that you choose to provide to Oculus Move [a pre-installed fitness suite]”
“Movement data” that Facebook uses to “keep you safe from bumping into real-world objects”—in other words, every single way your head and hands move around within VR and relative spatial data about the rooms you play VR within, which researchers have concluded can be used to create a recognizable biometric profile after only minutes of training
“The content of your conversations with people on apps like Messenger, Parties, and [Oculus] chats or your [Oculus] voice interactions”
For now, Facebook continues to tell users that “data that are processed on the device” are not uploaded to Facebook servers, which include “raw images” from Oculus headset sensors and “images of your hands” in its hand-tracking interface. Meanwhile, if you’d like to know how much of your use data inside of Facebook (and Instagram and other FB-connected services) might be leveraged by its combined advertising network, clear the rest of your day’s schedule and dive in.
Today’s announcement emphasizes that this advertising option is meant to generate “new ways for developers to generate revenue. The thing is, Facebook itself created a revenue blocker for VR game and app creators up until now, since its “app policies” agreement has always forbidden third-party advertising services inside of any products. Now that Facebook can operate the advertising platform and skim revenue off the top, things have changed.
How rapidly will the downstream soon run?
Facebook itself suggests that advertising is a key element in its VR business going forward: “This is a key part of ensuring we’re creating a self-sustaining platform that can support a variety of business models.” It also admits that product pricing can vary with advertising in the mix: “It helps us continue to make innovative AR [augmented reality]/VR hardware more accessible to more people.”
That news is unsurprising to anyone who follows Facebook’s quarterly financial results, which revolve largely around its targeted advertising platforms that deftly move from app to app and from service to service. Meanwhile, rival VR hardware manufacturers like HTC have loudly shot back at Facebook’s cheap-hardware sales approach.
Recently, HTC Vive general manager Dan O’Brien said the following to Ars Technica:
When pressed about Oculus as VR’s top-selling consumer option, O’Brien was frank: HTC wants to make its VR money from upfront purchase revenue, not from “downstream” opportunities. He described at length the business model of “some brands” subsidizing expensive hardware at a lower MSRP “with the hope of monetizing downstream on shared services” and “maybe using data-mining tactics to understand user behavior and then run a program that also generates downstream income.”
But also: notice the official mention of augmented reality in Facebook’s Wednesday pitch. The most recent Facebook Connect presentation revolved around Oculus research and hardware, included a wide-open pitch hosted by longtime Oculus lead Michael Abrash. He spoke of the company’s ambitions for Google Glass-like hardware that people may one day wear in public, full of real-time virtual images embedded in your nearby surroundings and high-level processing of all nearby audio and conversations. While we aren’t surprised that Facebook might want its eventual always-on-your-face device to tap into its advertising ecosystem, today’s announcement is a clear warning: if such a product should reach the market, it, like the $299 Oculus Quest 2, could very well be priced to move—but at a cost outside of shoppers’ dollars and cents.
As a reminder, all new Oculus-branded hardware going forward requires a Facebook account to work. Meanwhile, hardware sold before that rules change went into effect will require a ToS agreement beginning January 1, 2023. And the company’s combined ToS can penalize users for creating phantom or dummy Facebook accounts for the sole purpose of enabling connected Oculus VR features; by agreeing to that ToS, Facebook can void your account and its related purchases, should they be found in violation of its rules.
And as Facebook continues acquiring VR-focused video game developers, particularly the makers of megahit Beat Saber, those fully owned development houses could reasonably become prime targets for Facebook’s internal advertising tools. Big companies don’t acquire successful, smaller ones for charity, after all.