Walking into the office of Viktor Prokopenya — which overlooks a central London park — you would perhaps be forgiven for missing the significance of this unassuming location, just south of Victoria Station in London. While giant firms battle globally to make augmented reality a “real industry,” this jovial businessman from Belarus is poised to launch a revolutionary new technology for just this space. This is the kind of technology some of the biggest companies in the world are snapping up right now, and yet, scuttling off to make me a coffee in the kitchen is someone who could be sitting on just such a company.
Regardless of whether its immediate future is obvious or not, AR has a future if the amount of investment pouring into the space is anything to go by.
In 2016 AR and VR attracted $2.3 billion worth of investments (a 300 percent jump from 2015) and is expected to reach $108 billion by 2021 — 25 percent of which will be aimed at the AR sector. But, according to numerous forecasts, AR will overtake VR in 5-10 years.
Apple is clearly making headway in its AR developments, having recently acquired AR lens company Akonia Holographics and in releasing iOS 12 this month, it enables developers to fully utilize ARKit 2, no doubt prompting the release of a new wave of camera-centric apps. This year Sequoia Capital China, SoftBank invested $50 million in AR camera app Snow. Samsung recently introduced its version of the AR cloud and a partnership with Wacom that turns Samsung’s S-Pen into an augmented reality magic wand.
The IBM/Unity partnership allows developers to integrate into their Unity applications Watson cloud services such as visual recognition, speech to text and more.
So there is no question that AR is becoming increasingly important, given the sheer amount of funding and M&A activity.
Joining the field is Prokopenya’s “Banuba” project. For although you can download a Snapchat-like app called “Banuba” from the App Store right now, underlying this is a suite of tools of which Prokopenya is the founding investor, and who is working closely to realize a very big vision with the founding team of AI/AR experts behind it.
The key to Banuba’s pitch is the idea that its technology could equip not only apps but even hardware devices with “vision.” This is a perfect marriage of both AI and AR. What if, for instance, Amazon’s Alexa couldn’t just hear you? What if it could see you and interpret your facial expressions or perhaps even your mood? That’s the tantalizing strategy at the heart of this growing company.
Better known for its consumer apps, which have been effectively testing their concepts in the consumer field for the last year, Banuba is about to move heavily into the world of developer tools with the release of its new Banuba 3.0 mobile SDK. (Available to download now in the App Store for iOS devices and Google Play Store for Android.) It’s also now secured a further $7 million in funding from Larnabel Ventures, the fund of Russian entrepreneur Said Gutseriev, and Prokopenya’s VP Capital.
This move will take its total funding to $12 million. In the world of AR, this is like a Romulan warbird de-cloaking in a scene from Star Trek.
Banuba hopes that its SDK will enable brands and apps to utilise 3D Face AR inside their own apps, meaning users can benefit from cutting-edge face motion tracking, facial analysis, skin smoothing and tone adjustment. Banuba’s SDK also enables app developers to utilise background subtraction, which is similar to “green screen” technology regularly used in movies and TV shows, enabling end-users to create a range of AR scenarios. Thus, like magic, you can remove that unsightly office surrounding and place yourself on a beach in the Bahamas…
Because Banuba’s technology equips devices with “vision,” meaning they can “see” human faces in 3D and extract meaningful subject analysis based on neural networks, including age and gender, it can do things that other apps just cannot do. It can even monitor your heart rate via spectral analysis of the time-varying color tones in your face.
It has already been incorporated into an app called Facemetrix, which can track a child’s eyes to ascertain whether they are reading something on a phone or tablet or not. Thanks to this technology, it is possible to not just “track” a person’s gaze, but also to control a smartphone’s function with a gaze. To that end, the SDK can detect micro-movements of the eye with subpixel accuracy in real time, and also detects certain points of the eye. The idea behind this is to “Gamify education,” rewarding a child with games and entertainment apps if the Facemetrix app has duly checked that they really did read the e-book they told their parents they’d read.
If that makes you think of a parallel with a certain Black Mirror episode where a young girl is prevented from seeing certain things via a brain implant, then you wouldn’t be a million miles away. At least this is a more benign version…
Banuba’s SDK also includes “Avatar AR,” empowering developers to get creative with digital communication by giving users the ability to interact with — and create personalized — avatars using any iOS or Android device.Prokopenya says: “We are in the midst of a critical transformation between our existing smartphones and future of AR devices, such as advanced glasses and lenses. Camera-centric apps have never been more important because of this.” He says that while developers using ARKit and ARCore are able to build experiences primarily for top-of-the-range smartphones, Banuba’s SDK can work on even low-range smartphones.
The SDK will also feature Avatar AR, which allows users to interact with fun avatars or create personalised ones for all iOS and Android devices. Why should users of Apple’s iPhone X be the only people to enjoy Animoji?
Banuba is also likely to take advantage of the news that Facebook recently announced it was testing AR ads in its newsfeed, following trials for businesses to show off products within Messenger.
Banuba’s technology won’t simply be for fun apps, however. Inside two years, the company has filed 25 patent applications with the U.S. patent office, and of six of those were processed in record time compared with the average. Its R&D center, staffed by 50 people and based in Minsk, is focused on developing a portfolio of technologies.
Interestingly, Belarus has become famous for AI and facial recognition technologies.
For instance, cast your mind back to early 2016, when Facebook bought Masquerade, a Minsk-based developer of a video filter app, MSQRD, which at one point was one of the most popular apps in the App Store. And in 2017, another Belarusian company, AIMatter, was acquired by Google, only months after raising $2 million. It too took an SDK approach, releasing a platform for real-time photo and video editing on mobile, dubbed Fabby. This was built upon a neural network-based AI platform. But Prokopenya has much bolder plans for Banuba.
In early 2017, he and Banuba launched a “technology-for-equity” program to enroll app developers and publishers across the world. This signed up Inventain, another startup from Belarus, to develop AR-based mobile games.
Prokopenya says the technologies associated with AR will be “leveraged by virtually every kind of app. Any app can recognize its user through the camera: male or female, age, ethnicity, level of stress, etc.” He says the app could then respond to the user in any number of ways. Literally, your apps could be watching you.
So, for instance, a fitness app could see how much weight you’d lost just by using the Banuba SDK to look at your face. Games apps could personalize the game based on what it knows about your face, such as reading your facial cues.
Back in his London office, overlooking a small park, Prokopenya waxes lyrical about the “incredible concentration of diversity, energy and opportunity” of London. “Living in London is fantastic,” he says. “The only thing I am upset about, however, is the uncertainty surrounding Brexit and what it might mean for business in the U.K. in the future.”
London may be great (and will always be), but sitting on his desk is a laptop with direct links back to Minsk, a place where the facial recognition technologies of the future are only now just emerging.
Amazon assembles video streaming apps to fight with Netflix and Disney in India – TechCrunch
Amazon has launched Prime Video Channels in India, allowing its customers to subscribe to eight streaming services including Discovery+ and Mubi from one hub (Prime Video website or app), the latest in a series of efforts by the U.S. giant to win customers in the South Asian nation.
The new offering makes it easier for users to login and pay for additional streaming services, the company said. To make things more enticing, Amazon said the services are available at discounted prices for the first year. Discovery+ costs $4 per year, Mubi $27, Hoichoi ($8.2), DocuBay $6.8, ErosNow $4, Lionsgate Play $9.5, manoramaMax $9.5, and ShortsTV $4.
The company, which is currently at the centre of several controversies in India, did not share how much of the revenue it is taking from the streaming services and the duration of the partnerships. India is the 12th market where Amazon has launched Prime Video Channels.
Friday’s launch will likely be more beneficial to the third-party streaming services, all of which are struggling to make inroads in India, than Amazon itself. Discovery+ has amassed fewer than 4 monthly active users in India on mobile, Mubi has fewer than 100,000, DocuBay has fewer than 50,000, ErosNow has fewer than half a million, Lionsgate Play and manoramaMax each has fewer than 750,000, according to mobile insight firm App Annie (the data of which an industry executive shared with TechCrunch).
“With the launch of Prime Video Channels, we now take the next big step in our journey to entertain India by creating a video entertainment marketplace – first of its kind in India – which will not only delight our customers by giving them even more entertainment choices, but also benefit the OTT Channel partners who collaborate with us to leverage Prime Video’s distribution, reach and tech infrastructure,” said Gaurav Gandhi, Country Manager of Amazon Prime Video India, in a statement.
Amazon’s Prime Video, which has amassed over 55 million monthly active users in India, competes with Netflix, Disney’s Hotstar and Times Internet’s MX Player in the country. Both Hotstar and MX Player have established larger reach than Prime Video in India, and Netflix has courted most elite customers in the country.
The e-commerce group, on its part, has attempted to push for more reach by securing some rights to cricket matches — though the vast majority of those rights are still owned by Hotstar — and has also launched a free, ad-supported streaming service.
Livestream video shopping app NTWRK raises $50M from Goldman Sachs, Kering – TechCrunch
NTWRK, a video shopping app that has helped to popularize the idea of livestreamed commerce in the U.S., announced today it has closed on $50 million in new funding led by Goldman Sachs Asset Management and global luxury group Kering, owners of luxury brands Gucci, Yves Saint Laurent, Bottega Veneta and others. The company has been working to capitalize on the growing interest in live commerce and creator content, and shifted into live virtual events and festivals as COVID-19 ravaged the U.S. last year. Now it says it will invest in furthering its growth and working to establish a more global footprint.
Other participants in the new round include LionTree Partners and Tenere Capital. They join the company’s prior backers: Main Street Advisors (whose investors include Jimmy Iovine, Drake and LeBron James), Live Nation, Foot Locker and others. Allison Berardo, a vice president within the Growth Equity business within Goldman Sachs Asset Management, will join NTWRK’s board of directors.
Aimed at a younger crowd of Gen Z and millennial consumers, NTWRK offers tools that allow creators to interact with viewers and sell products in real time, in what’s been called a mix of QVC, Twitter and Twitch. The experience blends commerce with entertainment, as viewers watch and chat with other viewers and hosts in real time, as they shop for streetwear, shoes, collectibles and other items. The company has also created its own exclusive content where it has featured hosts like Billie Eilish, Juice WRLD, DJ Khaled, Odell Beckham Jr., Eddie Huang, Blake Griffin, Alexander Wang, FaZe Clan, Nadeshot, Jonah Hill, Gary Vee, A$AP Ferg, Wu-Tang Clan, Doja Cat and others. (It even invested into FaZe Clan last year, we should note.)
Its business model extends beyond just offering live video shopping that you can tune into at any time, as it also regularly features product drops that work to build up anticipation and excitement. This sort of feature is only now making its way to larger social media platforms, like Instagram, which introduced drops just this spring.
NTWRK has also embraced live events and virtual festivals as another way to engage audiences. Last year, for instance, it ran TRANSFER, which featured 30 brands and artists, panels, interviews, DJ sets and musical performances. It also ran BEYOND THE STREETS, a virtual art fair that attracted over 250,000 attendees. Earlier this year it ran a two-day designer toy and collectibles festival, Unboxed, as the first of a slate of digital events which have subsequently run throughout the year, including Surface Festival, a virtual food festival and a virtual home goods festival. It’s now gearing up for the return of its flagship event, TRANSFER.
This summer, NTWRK also embraced the world of digital goods with the launch of NTWRK NFT, its own curated shop for unique crypto art from creators like BADBOI, Imaginary Foundation, MILKMAN, Young & Sick, Fafi, KidEight, MGOGLKTKO and Eddie Gangland.
Livestream shopping is already a popular activity overseas, but is still gaining ground here in the U.S. The company, in announcing its news today, noted that livestream shopping in China reached $150 billion in 2020 and is expected to grow to $300 billion this year. But in the U.S. it’s expected to reach $11 billion by the end of 2021 and $25 billion by 2025, leaving much room for growth.
“Our vision is to become the biggest, most culturally relevant, livestream shopping marketplace for Gen-Z and Millennial audiences who are obsessed with pop culture,” said NTWRK CEO Aaron Levant, in a statement. “It’s exciting for NTWRK to have Goldman Sachs and Kering sign on for the future of livestream shopping.”
NTWRK had previously raised a $10 million Series A, according to Crunchbase data.
Google powers up assistive tech in Android with facial gesture-powered shortcuts and switches – TechCrunch
Making smartphones more accessible is always a good idea, and Google’s latest features bring quick actions and navigation to people whose expressions are their primary means of interacting with the world. Project Activate and Camera Switches let users perform tasks like speaking a custom phrase, or navigating using a switch interface, through facial gestures alone.
The new features rely on the smartphone’s front-facing camera, which can watch the user’s face in real time for one of six expressions: a smile, raised eyebrows, opened mouth, and looking left, right, or up. It relies entirely on local computing and no image data is saved, nor is it doing what is generally understood as “facial recognition” — this type of machine learning can focus specifically on, for example, identifying the eyebrows and sending a signal whenever they move past a certain, customizable threshold.
Each expression can be assigned a different role. Camera Switches integrates with Android’s existing switch compatibility, by which people who use assistive tech like a joystick or blow tube can navigate the phone’s OS. Now that can be done without any peripheral device at all, and users can pick various facial gestures for iterating through selections, confirming a choice, backing out, and so on.
Using Project Activate, expressions can be tied to self-contained actions like speaking a phrase. Many people with disabilities rely on caretakers for a variety of reasons, but one thing you can’t ask a caretaker to do is get your caretaker’s attention! So one useful application might be assigning (say) an extended eyebrow raise to have the device speak the phrase “hey!” or “I need help with something,” or “thanks!”
The gestures can also be used to play an audio file, or text or call a predetermined number. More expressions and more capabilities are on the way, as well as more languages — of course faces don’t have languages, but the app and support documentation do, so Project Activate will start with English-speaking countries and move out from there. Camera Switches, on the other hand, will be available in 80 languages from the start.
Note that these can’t both be used at once, since they both require access to the camera and expression recognition thing. So users should make sure they have a backup method for navigation. Both should run on pretty much any Android phone from the last five years or so.
Lastly, an update to Google’s Lookout app, which reads labels for people with visual impairments, adds the ability to scan and read out handwritten content the way it can with printed things. That’s useful for sticky notes, “gone fishing” type signs on the doors of stores, and things like greeting cards with notes from the sender. The app has seen big increases in usage over the last year so they’re building it out. (Support for identifying euro and Indian rupee banknotes should help fuel that growth too.)
All the new stuff should be available later this week free of charge.
NFL Sunday Ticket games may be coming to Apple’s streaming platform
As the NFL’s 8-year Sunday Ticket deal with DirecTV nears its end, the football league is looking to Apple as...
As Florida punishes schools, study finds masks cut school COVID outbreaks 3.5X
Enlarge / A second-grade teacher talks to her class during the first day of school at Tustin Ranch Elementary School...
Real-life house behind ‘The Conjuring’ is on the market again
Just in time for Halloween is an old, very expensive house with one notable feature: it is allegedly haunted. The...
With iPhone 13 in hand, I miss Touch ID more than ever
I’ve been traveling this past week, and the iPhone 13 Pro Max has been my trusty sidekick. Whether it’s boarding...
Yahoo has built a new calendar app called Day, and it’s recruited the co-founder of Sunrise to design it – TechCrunch
When it comes to online calendars and calendar apps, services like Google Calendar and Outlook from Microsoft rule the roost...
Social2 years ago
CrashPlan for Small Business Review
Gadgets3 years ago
A fictional Facebook Portal videochat with Mark Zuckerberg – TechCrunch
Cars3 years ago
What’s the best cloud storage for you?
Social3 years ago
iPhone XS priciest yet in South Korea
Mobile3 years ago
Memory raises $5M to bring AI to time tracking – TechCrunch
Security3 years ago
Google latest cloud to be Australian government certified
Cars3 years ago
SK Telecom and Samsung to collaborate on 5G for enterprise
Social3 years ago
Apple’s new iPad Pro aims to keep enterprise momentum