Live Caption, Google’s automatic captioning system first introduced at its I/O developer conference this May, is now officially available, alongside the launch of the new Pixel 4. But unlike some of the other technologies highlighted at the company’s Pixel hardware event yesterday, Live Caption won’t be limited to Google’s new smartphone alone. After the initial debut on Pixel 4, the automatic captioning technology will roll out to Pixel 3, Pixel 3 XL, Pixel 3a and Pixel 3a XL before the year-end, says Google, and will become more broadly available in 2020.
The company has already offered automatic captions on YouTube for a decade, but that same sort of experience isn’t available across the wider web and mobile devices. For example, Google explains, you can’t read captions for things like the audio messages sent by your friends, on trending videos published elsewhere on social media, and on the content you record yourself.
There’s a significant accessibility issue with the lack of captions in all these places, but there’s a convenience issue, as well.
If you’re in a loud environment, like a commuter train, or trying to watch content privately and forgot your headphones, you may need to just use the captions. Or maybe you don’t want to blare the audio, which disturbs others around you. Or perhaps, you want to see the words appear because you’re having trouble understanding the audio, or just want to be sure to catch every word.
With the launch of the Pixel 4, Live Caption is also available for the first time to the general public.
The technology will capture and automatically caption videos and spoken audio on your device, except for phone and video calls. This captioning all happens in real-time and on your device — not in the cloud. That means it works even if your device lacks a cell signal or access to Wi-Fi. The captions also stay private and don’t leave your phone.
This is similar to how the Pixel 4’s new Recorder app functions. It, too, will do its speech-to-text processing all on your device, in order to give you real-time transcriptions of your meetings, interviews, lectures, or anything else you want to record, without compromising your privacy.
You can launch the Live Captions feature with a tap from the volume slider that appears, then reposition the caption box anything on your screen so it doesn’t get in the way of what you’re viewing.
Currently, the feature supports English only. But Google says it’s working to add more languages in the future.
After today’s launch on Pixel 4 and the rollout to the rest of the modern Pixel line of smartphones this year, it will start to show up in other new Android phones. Google says it’s working with other manufacturers to make the technology available to more people as soon as next year.
A successful phishing attack at SMS services company Twilio may have exposed the phone numbers of roughly 1,900 users of the secure messaging app Signal—but that’s about the extent of the breach, says Signal, noting that no further user data could be accessed.
In a Twitter thread and support document, Signal states that a recent successful (and deeply resourced) phishing attack on Twilio allowed access to the phone numbers linked with 1,900 users. That’s “a very small percentage of Signal’s total users,” Signal writes, and all 1,900 affected users will be notified (via SMS) to re-register their devices. Signal, like many app companies, uses Twilio to send SMS verification codes to users registering their Signal app.
With momentary access to Twilio’s customer support console, attackers could have potentially used the verification codes sent by Twilio to activate Signal on another device and thereby send or receive new Signal messages. Or an attacker could confirm that these 1,900 phone numbers were actually registered to Signal devices.
No other data could be accessed, in large part because of Signal’s design. Message history is stored entirely on user devices. Contact and block lists, profile details, and other user data require a Signal PIN to access. And Signal is asking users to enable registration lock, which prevents Signal access on new devices until the user’s PIN is correctly entered.
“The kind of telecom attack suffered by Twilio is a vulnerability that Signal developed features like registration lock and Signal PINs to protect against,” Signal’s support document reads. The messaging app notes that while Signal doesn’t “have the ability to directly fix the issues affecting the telecom ecosystem,” it will work with Twilio and other providers “to tighten up their security where it matters for our users.”
Signal PINs were introduced in May 2020, in part to de-emphasize the reliance on phone numbers as a primary user ID. This latest incident may provide another nudge to de-couple Signal’s strong security from the SMS ecosystem, where cheap, effective spoofing and broad network hacks remain all too common.
If you’re using Zoom on a Mac, it’s time for a manual update. The video conferencing software’s latest update fixes an auto-update vulnerability that could have allowed malicious programs to use its elevated installing powers, granting escalated privileges and control of the system.
The vulnerability was first discovered by Patrick Wardle, founder of the Objective-See Foundation, a nonprofit Mac OS security group. Wardle detailed in a talk at Def Con last week how Zoom’s installer asks for a user password when installing or uninstalling, but its auto-update function, enabled by default, doesn’t need one. Wardle found that Zoom’s updater is owned and runs as the root user.
It seemed secure, as only Zoom clients could connect to the privileged daemon, and only packages signed by Zoom could be extracted. The problem is that by simply passing the verification checker the name of the package it was looking for (“Zoom Video ... Certification Authority Apple Root CA.pkg“), this check could be bypassed. That meant malicious actors could force Zoom to downgrade to a buggier, less-secure version or even pass it an entirely different package that could give them root access to the system.
Some of Wardle’s findings had been patched in a prior update, but key root access was still available as of Wardle’s talk on Saturday. Zoom issued a security bulletin the same day, and a patch for version Zoom 5.11.5 (9788) followed soon after. You can download the update directly from Zoom or click on your menu bar options to “Check for updates.” We wouldn’t suggest waiting for an automatic update, for multiple reasons.
Zoom’s software security record is spotty—and at times, downright scary. The company settled with the FTC in 2020 after admitting that it lied for years about offering end-to-end encryption. Wardle previously revealed a Zoom vulnerability that let attackers steal Windows credentials by sending a string of text. Prior to that, Zoom was caught running an entire undocumented web server on Macs, causing Apple to issue its own silent update to kill the server.
Last May, a Zoom vulnerability that enabled a zero-click remote code execution used a similar downgrade and signature-check bypass. Ars’ Dan Goodin noted that his Zoom client didn’t actually update when the fix for that issue arrived, requiring a manual download of an intermediate version first. Hackers can take advantage of exposed Zoom vulnerabilities quickly, Goodin noted, if Zoom users aren’t updated right away. Minus the root access, of course.
Farmers around the world have turned to tractor hacking so they can bypass the digital locks that manufacturers impose on their vehicles. Like insulin pump “looping” and iPhone jailbreaking, this allows farmers to modify and repair the expensive equipment that’s vital to their work, the way they could with analog tractors. At the DefCon security conference in Las Vegas on Saturday, the hacker known as Sick Codes is presenting a new jailbreak for John Deere & Co. tractors that allows him to take control of multiple models through their touchscreens.
The finding underscores the security implications of the right-to-repair movement. The tractor exploitation that Sick Codes uncovered isn’t a remote attack, but the vulnerabilities involved represent fundamental insecurities in the devices that could be exploited by malicious actors or potentially chained with other vulnerabilities. Securing the agriculture industry and food supply chain is crucial, as incidents like the 2021 JBS Meat ransomware attack have shown. At the same time, though, vulnerabilities like the ones that Sick Codes found help farmers do what they need to do with their own equipment.
John Deere did not respond to WIRED’s request for comment about the research.
Sick Codes, an Australian who lives in Asia, presented at DefCon in 2021 about tractor application programming interfaces and operating system bugs. After he made his research public, tractor companies, including John Deere, started fixing some of the flaws. “The right-to-repair side was a little bit opposed to what I was trying to do,” he tells WIRED. “I heard from some farmers; one guy emailed me and was like ‘You’re fucking up all of our stuff!’ So I figured I would put my money where my mouth is and actually prove to farmers that they can root the devices.”
This year, Sick Codes says that while he is primarily concerned about world food security and the exposure that comes from vulnerable farming equipment, he also sees important value in letting farmers fully control their own equipment. “Liberate the tractors!” he says.
After years of controversy in the US over the “right to repair” the equipment one purchases, the movement seems to have reached a turning point. The White House issued an executive order last year directing the Federal Trade Commission to increase enforcement efforts over practices like voiding warranties for outside repair. That, combined with New York state passing its own right-to-repair law and creative activist pressure, has generated unprecedented momentum for the movement.
Facing mounting pressure, John Deere announced in March that it would make more of its repair software available to equipment owners. The company also said at the time that it will release an “enhanced customer solution” next year so customers and mechanics can download and apply official software updates for Deere equipment themselves, rather than having John Deere unilaterally apply the patches remotely or force farmers to bring products to authorized dealerships.
“Farmers prefer the older equipment simply because they want reliability. They don’t want stuff to go wrong at the most important part of the year when they have to pull stuff out of the ground,” Sick Codes says. “So that’s what we should all want too. We want farmers to be able to repair their stuff for when things go wrong, and now that means being able to repair or make decisions about the software in their tractors.”
To develop his jailbreak, Sick Codes got his hands on numerous generations of John Deere tractor control touchscreen consoles. But ultimately he focused on a few models, including the widely deployed 2630 and 4240 models, for the exploit he is presenting. It took experimentation on a number of touchscreen circuit boards over many months to find bypasses to John Deere’s dealer authentication requirements, but eventually Sick Codes was able to game a reboot check to restore the device as if it were being accessed by a certified dealer.
He found that when the system thought it was in such an environment, it would offer more than 1.5 GB worth of logs that were meant to help authorized service providers diagnose problems. The logs also revealed the path to another potential timing attack that might grant deeper access. Sick Codes soldered controllers directly onto the circuit board and eventually got his attack to bypass the system’s protections.
“I launched the attack, and two minutes later a terminal pops up,” Sick Codes says of the program used to access a computer’s command-line interface. “I had root access, which is rare in Deere land.”
The approach requires physical access to the circuit board, but Sick Codes says it would be possible to develop a tool based on the vulnerabilities to more easily execute the jailbreak. Mostly he says he is curious to see how John Deere will react. He’s unsure how comprehensively the company can patch the flaws without implementing full disk encryption, an addition that would mean a significant system overhaul in new tractor designs and likely wouldn’t be deployed in existing equipment.
The first priority? Running custom farm-themed Doom on the tractor, of course.