Connect with us


Someone used neural networks to upscale a famous 1896 video to 4k quality (Updated)



Note (February 5): In the original version of this story I was comparing a low-quality copy of the 1896 film to the upscaled version. Shiryaev actually started with a higher-quality scan of the film, and many of the differences I observed actually reflected his better source material, not the upscaling algorithm. I’ve updated the first video below to the one Shiryaev used, but I left the text of the story as-is.

Arrival of a Train at La Ciotat is one of the most famous films in cinema history. Shot by French filmmakers Auguste and Louis Lumière, it achieved an unprecedented level of quality for its time. Some people regard its commercial exhibition in 1896 as the birth of the film industry. An urban legend—likely apocryphal—says that viewers found the footage so realistic that they screamed and ran to the back of the room as the train approached. I’ve embedded a video of the original film above.

Of course, humanity’s standards for realism have risen dramatically over the last 125 years. Today, the Lumière brothers’ masterpiece looks grainy, murky, and basically ancient. But a man named Denis Shiryaev used modern machine-learning techniques to upscale the classic film to 21st-century video standards.

The result is remarkable. Watching the upscaled version makes the world of our great-great-great-grandparents come to life. Formerly murky details of the train, the clothing, and the faces of the passengers now stand out clearly.

How did Shiryaev do it? He says he used commercial image-editing software called Gigapixel AI. Created by Topaz Labs, the package allows customers to upscale images by up to 600 percent. Using sophisticated neural networks, Gigapixel AI adds realistic details into an image to avoid making it look blurry as it’s scaled up.

As the name implies, neural networks are networks of artificial neurons—mathematical functions that transform a set of input values into an output value. The key feature of neural networks is that they can be trained: if you have a bunch of example inputs whose “correct” outputs are known, you can tune the parameters of the network to make it more likely to produce correct answers. The hope is that this training will generalize—that once you’ve trained it to produce the right answer for inputs the network has seen before, it will produce good answers for inputs it hasn’t seen, too.

To train a network, you need to have a database of examples where the right answer is already known. Sometimes AI researchers have to hire human beings to produce these right answers by hand. But for image upscaling, there’s a convenient shortcut: you start with high-resolution images and downsample them. The low-resolution images become your inputs, and the high-resolution originals serve as the “correct” answer the network is aiming to produce.

“A neural network analyzes thousands of photo pairs to learn how details usually get lost,” Topaz Labs explains on their product page for Gigapixel AI. “The algorithm learns to ‘fill in’ information in new images based on what it has learned, effectively adding new detail to your photo.”

Show the neural network a low-resolution image of a face and it will figure out that it’s a face and fill in the right details for the subject’s eyes, nose, and mouth. Show the neural network a low-resolution brick building and it will add a suitable brick pattern in the high-res version.

Timothy B. Lee / Colorize Images / Denis Shiryaev

An obvious next step would be to colorize the video. Neural networks can do that, too, using the same basic technique: start with a bunch of color photos, convert them to black and white, and then train a neural network to reconstruct the color originals.

I dropped a frame from Shiryaev’s video into the Colorize Images app for Android, which uses machine learning to automatically colorize images. As you can see, it does a pretty good job, correctly inferring that trees should be green, gravel should be a brownish color, and that men’s coats should be black. I would love to see someone with more time and better tools colorize Shiryaev’s upscaled version of the Lumière Brothers’ classic.

Continue Reading


After ruining 75M J&J doses, Emergent gets FDA clearance for 25M doses



Enlarge / The Emergent BioSolutions plant, a manufacturing partner for Johnson & Johnson’s COVID-19 vaccine, in Baltimore, Maryland, on April 9, 2021.

The US Food and Drug Administration is making progress in its efforts to sort out the fiasco at Emergent BioSolutions’ Baltimore facility, which, at this point, has ruined more than 75 million doses of COVID-19 vaccines stemming from what the regulator identified as significant quality control failures.

In March, news leaked that Emergent ruined 15 million doses of Johnson & Johnson’s vaccine as well as millions more doses of AstraZeneca’s vaccine. The spoilage happened when Emergent cross-contaminated batches of the two vaccines with ingredients from the other.

Last week, the FDA told Emergent to trash about 60 million more doses of Johnson & Johnson’s vaccine due to similar contamination concerns, The New York Times reported.

But at the same time, the agency cleared 10 million doses of Johnson & Johnson’s vaccine for use—with the catch that the doses must carry a warning saying that the FDA cannot guarantee Emergent followed good manufacturing practices while making them. And on Tuesday, the FDA cleared an additional 15 million doses of Johnson & Johnson’s vaccine, bringing the total number of acceptable doses to just 25 million, according to The Wall Street Journal.

Still, more than 100 million finished doses of Johnson & Johnson’s and AstaZeneca’s vaccines are still in limbo at the facility, awaiting FDA review. All of the doses at the facility were made prior to April 16, when the FDA shut down production after an investigation found sweeping and significant quality control failures and manufacturing violations.

Some lawmakers say the issues were clear before the investigation; Emergent has a long track record of such problems, as well as trouble fulfilling contracts.

Troubled past

Still, the manufacturer was contracted during the pandemic to produce both the Johnson & Johnson’s one-dose vaccine and AstraZeneca’s vaccine, which use similar adenovirus-based vaccine platforms. Emergent had also been awarded millions of dollars in federal grants to help respond to the pandemic swiftly, including $27-million monthly “reservation” payments to keep its facility at the ready to produce large amounts of vaccine under proper manufacturing standards and practices.

But the FDA’s nine-day inspection of the Baltimore facility, which began April 12, revealed that Emergent wasn’t putting that money to good use. FDA inspectors logged a long list of problems, including unsanitary conditions, paint peeling off of the walls and floors, black and brown residue on surfaces, improperly trained staff, and numerous opportunities for vaccine products to be contaminated. For instance, inspectors witnessed Emergent employees dragging unsealed, non-decontaminated bags of medical waste across different areas of the facility. In some cases, employees tossed unsealed bags of medical waste in an elevator.

Though Emergent had already scrapped the initial 15 million contaminated vaccine doses at the time, FDA inspectors concluded that “there is no assurance that other batches have not been subject to cross contamination,” the inspectors wrote.

The FDA shut down production April 16 and has been sorting through the premade doses ever since. For the most part, Emergent’s failures have not had a significant impact on vaccination efforts in the US. All of the doses of Johnson & Johnson vaccine administered in the US were made in the Netherlands. And demand for the one-shot vaccine has slipped amid slowed vaccination rates and concern over an extremely rare but life-threatening blood-clotting condition. In fact, US regulators recently extended the expiration data on millions of doses that have gone unused. AstraZeneca’s vaccine, meanwhile, is not yet authorized for use in the US.

However, Emergent’s failures have global effects—many of the doses have been earmarked to be donated to other countries in need of vaccine supplies. The contamination problem has held up the export of potentially usable doses.

In a statement Tuesday after the FDA cleared the additional 15 million doses, Emergent said:

We welcome the approval of an additional batch of J&J vaccine made at Emergent. We remain committed to addressing the FDA’s observations in order to resume production as soon as possible and look forward to continuing our work to end this pandemic.

Federal officials stripped Emergent of its control of the Baltimore facility back in April, putting Johnson & Johnson in charge and telling AstraZeneca to find another manufacturer. Federal lawmakers, meanwhile, opened a multipronged investigation into whether Emergent used ties to the Trump administration to improperly obtain lucrative government contracts.

Continue Reading


Cold-War-era missile launches three modern-day spy satellites



Enlarge / A Minotaur rocket launches the NROL-111 mission on Tuesday.

Trevor Mahlmann

For the first time in nearly eight years, a Minotaur 1 rocket launched into space Tuesday from NASA’s Wallops Flight Facility in Virginia. The rocket, which is derived from Cold-War-era surplus missiles, carried three classified satellites into orbit for the US National Reconnaissance Office.

This was the first launch of the four-stage Minotaur 1 rocket since a demonstration mission for the Air Force in 2013, which also orbited 23 CubeSats. Although the current mission was delayed for more than two hours by poor weather on Tuesday morning, it successfully launched at 9:35 am ET (13:35 UTC).

The Minotaur 1, which has the capacity to launch a little more than 500 kg into low Earth orbit, is a mix of decades-old technology and modern avionics. The vehicle’s first and second stages are taken from a repurposed Minuteman I missile, the first generation of land-based, solid-fuel intercontinental ballistic missiles. These missiles were in service from 1962 to 1965 before they were phased out in favor of the Minuteman II and Minuteman III missiles. The latter ICBMs are still in silos today.

To configure the Minotaur 1 rocket for satellite launches, engineers added two additional stages based on Orion solid rocket motors. These orbital rockets are now built and launched by Northrop Grumman. In addition to the Minotaur 1 vehicle, the company also supports the larger Minotaur C and Minotaur IV launch vehicles based on Peacekeeper missiles.

The small rockets are not cheap. This Minotaur I launch cost the Air Force $29.2 million when it procured the rocket for the National Reconnaissance Office in 2016. By contrast, Relativity Space, Firefly, and ABL Space are all developing rockets more capable than the Minotaur 1, with about 1 metric ton of lift capacity, at a fraction of its cost.

However, the Minotaur line of vehicles has a perfect record across 28 missions, having launched from Alaska, California, Florida, and Virginia with 100 percent success. The US military values this kind of reliability and the operational readiness of a solid-motor rocket.

Lt. Col. Ryan Rose, chief of the Space and Missile Systems Center Launch Enterprise’s Small Launch and Targets Division, said in a statement that she is looking forward to future launches from Northrop Grumman: “This success continues to reinforce that the Launch Enterprise has multiple paths to rapidly acquire agile launch services for small satellites and will continue to take advantage of the latest in small launch technologies.”

As for the top-secret payloads launched Tuesday, it’s a good bet they are spy satellites of some sort. The National Reconnaissance Office is charged with a “mission of providing critical information to every member of the Intelligence Community, two dozen domestic agencies, our nation’s military, lawmakers, and decision makers.” So they’re probably reading this article—from space.

Continue Reading


A cold spot and a stellar burp led to strange dimming of Betelgeuse



Thanks to a new study conducted with ESO telescopes, we now know that Betelgeuse’s dip in brightness was the result of a “dusty veil” that formed from material that emerged from the star. Credit: ESO/L. Calçada.

In December 2019, astronomers noticed a strange, dramatic dimming in the light from Betelgeuse, a bright red star in the Orion constellation. They puzzled over the phenomenon and wondered whether it was a sign that the star was about to go supernova. Several months later, they had narrowed the most likely explanations to two: a short-lived cold patch on the star’s southern surface (akin to a sun spot), or a clump of dust making the star seem dimmer to observers on Earth. We now have our answer, according to a new paper published in the journal Nature. Dust is the primary culprit, but it is linked to the brief emergence of a cold spot.

As Ars’ John Timmer reported last year, Betelgeuse is one of the closest massive stars to Earth, about 700 light years away. It’s an old star that has reached the stage where it glows a dull red and expands, with the hot core only having a tenuous gravitational grip on its outer layers. The star has something akin to a heartbeat, albeit an extremely slow and irregular one. Over time, the star cycles through periods when its surface expands and then contracts.

One of these cycles is fairly regular, taking a bit over five years to complete. Layered on that is a shorter, more irregular cycle that takes anywhere from under a year to 1.5 years to complete. While they’re easy to track with ground-based telescopes, these shifts don’t cause the sort of radical changes in the star’s light that would account for the changes seen during the dimming event.

In late 2019, Betelgeuse dimmed so much that the difference was visible to the naked eye. The dimming persisted, decreasing in brightness by 35 percent in mid-February, before brightening again in April 2020.

Telescopes pointed at the giant were able to determine that—rather than a tidy, uniform drop in luminance—Betelgeuse’s dimming was unevenly distributed, giving the star an odd, squished shape when viewed from Earth. That raised lots of questions about what was going on with the giant, with some experts speculating that because of Betelgeuse’s size and advanced age, the strange behavior was a sign of a supernova in the making.

By mid-2020, astronomers had changed their tune. An international team of observers happened to have the Hubble Space Telescope pointed at Betelgeuse before, during, and after the dimming event. Combined with some timely ground observations, this UV data indicated that a big burp that formed a cloud of dust near the star may have caused the star to get darker.

“With Hubble, we could see the material as it left the star’s surface and moved out through the atmosphere, before the dust formed that caused the star to appear to dim,” said Andrea Dupree, an astronomer at the Harvard-Smithsonian Center for Astrophysics who made those observations. She is also a co-author on the new paper.

Enlarge / These images, taken with the SPHERE instrument on ESO’s Very Large Telescope, show the surface of the red supergiant star Betelgeuse during its unprecedented dimming. The image on the far left, taken in January 2019, shows the star at its normal brightness. The remaining images, from December 2019, January 2020, and March 2020, were all taken when the star’s brightness had noticeably dropped.

ESO/M. Montargès et al.

The findings last year showed that an outer layer of the star, called the photosphere, had begun unevenly accelerating outward right before Betelgeuse began to dim. At its peak, the photosphere was moving at around 7 kilometers per second, reversing the outward push as the dimming of the star became more dramatic.

Dupree and her colleagues suggested that as the star expanded in one of its usual cycles, a portion of the surface accelerated much more rapidly, thanks to a convection cell that had traveled from the interior of the star to its surface. Those two events combined pushed out sufficient material far enough from the star that it cooled down, forming stardust. That dust could account for the dimming.

The new Nature paper expands on those earlier observations due to images captured by the European Southern Observatory’s (ESO) Very Large Telescope (VLT) in January and March 2020. “For once, we were seeing the appearance of a star changing in real time on a scale of weeks,” said co-author Miguel Montargès, from the Observatoire de Paris, France, and KU Leuven, Belgium.

Those images, combined with earlier observations in January and December 2019, allowed astronomers to directly witness the stardust formation, matching the observations of Dupree and her colleagues last year. The ESO team concluded that a gas bubble was ejected and pushed further out by the star’s outward pulsation. When a convection-driven cold patch appeared on the surface, the local temperature decrease was sufficient to condense the heavier elements (like silicon) into solid dust, forming a dusty veil that obscured the star’s brightness in its southern hemisphere. The astronomers speculate that a similar expelling of dust from cool stars could end up becoming building blocks of planets.

The ESO team found no evidence to support the impending supernova hypothesis. “The lack of an explosive conclusion might seem disappointing, but [these] results go beyond explaining one brief wink of a nearby star,” University of Washington astronomer Emily Levesque (who is not a co-author) wrote in an accompanying Nature commentary. She raises the prospect of other red supergiants also showing signs of dimming. “Next-generation facilities focused on monitoring stellar brightness over time, or on studying the signature of dust in the infrared spectra of stars, could prove invaluable for expanding the lessons learned here.”

One of those next-generation facilities is ESO’s Extremely Large Telescope (ELT), slated to achieve first light in 2026. “With the ability to reach unparalleled spatial resolutions, the ELT will enable us to directly image Betelgeuse in remarkable detail,” said co-author Emily Cannon of KU Leuven. “It will also significantly expand the sample of red supergiants for which we can resolve the surface through direct imaging, further helping us to unravel the mysteries behind the winds of these massive stars.”

DOI: Nature, 2021. 10.1038/s41586-021-03546-8 (About DOIs).

This animation combines four real images of the red supergiant star Betelgeuse, the first taken in January 2019 and the others taken in December 2019, January 2020, and March 2020, during the star’s unprecedented dimming.

Listing image by ESO/M. Montargès et al.

Continue Reading