Connect with us

Gadgets

How fast are your disks? Find out the open source way, with fio

Published

on

Enlarge / Ars Technica does not recommend removing the protective cover from your hard disk or setting it on fire in production settings.

Storage benchmarking—much like Wi-Fi benchmarking—is a widely misunderstood black art. Admins and enthusiasts have for decades been tempted to just “get the big number” by reading or writing a large amount of data to a disk, getting a figure in MB/sec, and calling it a day. Unfortunately, the actual workload of a typical disk doesn’t look like that—and that “simple speed test” doesn’t reproduce a lot of the bottlenecks that slow down disk access in real-world systems.

The most realistic way to test and benchmark disks is, of course, to just use them and see what happens. Unfortunately, that’s neither very repeatable, nor is it simple to analyze. So we do want an artificial benchmarking tool—but we want one that we can use intelligently to test storage systems across realistic scenarios that model our day-to-day usage well. Fortunately, we don’t have to invent such a tool—there’s already a free and open source software tool called fio, and it’s even cross-platform!

We’re going to walk you through some simple but effective uses of fio on Windows, Mac, and Linux computers—but before we do that, let’s talk a little bit about storage systems from a more basic perspective.

Throughput, latency, IOPS and cache

Throughput

Throughput, measured most commonly in storage systems in MB/sec, is the most commonly used way to talk about storage performance. There are several choke points in a storage system for throughput—first and foremost, there’s the speed of the physical medium itself. If you’ve got a single head on a conventional rust disk spinning at 7200RPM, the rate you can get data on or off that disk will be limited by the number of physical sectors/blocks passing beneath the head. You’re also limited by the bandwidth of your controller and cabling—for example, modern SATA links typically operate at 6Gbps, while modern SAS links can operate up to 22.5Gbps.

Things get a little extra complicated here, because we’re mixing units—notice the big B in MB/sec, and the small b in Gbps. That’s the difference between bytes and bits. You divide Gbps by 8 to get GB/sec, then multiply by 1024 to get MB/sec. So a SATA-3 6Gbps link can theoretically move up to 768MB/sec. You can’t actually move data across the SATA or SAS bus at the full theoretical link speed, but you can get fairly close. It’s also worth noting that most SATA controllers won’t move much more data than a single link can manage, even with many disks connected to the controller—so it’s common to see even lots of very fast solid state drives in an array bottlenecking at around 700 MB/sec.

Latency

Latency is the flip side of the same performance coin. Where throughput refers to how many bytes of data per second you can move on or off the disk, latency—most commonly measured in milliseconds—refers to the amount of time it takes to read or write a single block. Most of the worst storage bottlenecks are latency issues that affect throughput, not the other way around.

In conventional spinning rust disks, there are two major sources of latency: rotational latency, and seek latency. The seek latency is how long it takes to move the mechanical arm the disk head is mounted on to the correct track on disk. Once the head has moved to the correct track, the drive then has to wait for the correct sector to rotate beneath the head—that’s the rotational latency. The combination of seek and rotational latency usually adds up to somewhere between 15ms and 25ms.

You can see how latency affects throughput by thought experiment. If we have a reasonably fast spinning disk with a maximum throughput of 180MB/sec and a total access latency of 16ms, and we present it with a maximally fragmented workload—meaning that no two blocks have been written/are being written in sequential order—we can do a little math to come up with that throughput. Assuming 4KB physical blocks on disk, 4KB per seek divided by 0.016 seconds per seek = only 250KB/sec. Ouch!

IOPS

Short for Input/Output Operations Per Second, IOPS is the metric of measurement you’ll most commonly hear real storage engineers discussing. It means exactly what it sounds like—how many different operations can a disk service? In much the same way, “throughput” usually refers to the maximal throughput of a disk, with very large and possibly sequential reads or writes, IOPS usually refers to the maximal number of operations a disk can service on the low end—4K random reads and writes.

Solid state disks don’t suffer from seek or rotational latency, but 4K random Input/Output (I/O) does still present them with problems. Under the hood, a consumer SSD isn’t really a single “disk”—it’s a RAID array in a little sealed box, with its own complex controller inside the disk itself managing reads and writes. The SSD controller itself tries to stripe writes across multiple channels of physical flash media in parallel—and, if the user is lucky, the writes which got striped out evenly across those channels will also be read the same way, maximizing throughput.

When a solid state disk is presented with a 4K random I/O workload, if it can’t figure out some way to aggregate and parallelize the requests, it will end up bottlenecking at much lower speeds, dictated by how quickly a single cell of flash media can read or write a block of data. The impact isn’t as dramatic as it would be on a rust disk, but it’s still significant—where a rust disk capable of 180MB/sec of throughput might plummet to 250KB/sec of 4K random I/O, a SSD capable of 500MB/sec could drop to around 40MB/sec.

Although you can discuss throughput in terms of 4K random I/O, and IOPS in terms of sequential 1MB I/O, that’s not how each term is typically used. You should generally expect throughput to be discussed in terms of how much data a disk moves under optimal conditions, and IOPS in terms of the “low end grunt” the disk is capable of even under the worst workload. For typical desktop PC use, IOPS is far more important than throughput—because there’s lots of that slow 4K random I/O, and it slows the whole system down when it happens.

Cache

As we’ve seen above, non-optimized workloads hurt performance, and hurt them badly. Thankfully for users, decades of research and development have presented us with all manner of tricks to keep from exploring the worst performance characteristics of our storage—especially rust storage. Operating systems use both read caches and write buffers to minimize the number of seeks necessary in operation and avoid the need to keep reading frequently needed blocks from storage over and over.

Write buffers allow the operating system to store up lots of small I/O requests and commit them to disk in large batches. One megabyte is a very small amount of data, but it still comes out to 256 4KB blocks—and if you must write each of those blocks out with individual operations, you might tie up your disk’s entire service capacity for a full second. On the other hand, if you can aggregate those 256 blocks in a write buffer and then flush them out in a single operation, you avoid all that access latency, and the same amount of data can be saved in a hundredth of a second or less. This aggregation can also greatly help with read speeds later. If most of the same blocks need to be read as a group later, the drive can avoid seeking between them since they were all written as a group in the first place.

Read cache keeps the system from having to tie up storage with unnecessary repeated requests for the same blocks over and over again. If your operating system has plenty of RAM available, each time it reads data from disk, it keeps a copy of it lying around in memory. If another program asks for the same blocks later, the operating system can service that request directly from the cache—which keeps the drive’s limited resources available for either read or write requests, which must hit the actual disk.

Some models of SSD have an additional non-volatile write cache on the disk itself, made of a faster and more expensive type of flash media. For example, a TLC or QLC (Quad Layer Cell) SSD might have a few gigabytes of MLC (Multi-Layer Cell) media to use as a buffer for writes; this enables the SSD to keep up with the writes demanded by a typical desktop workload using the faster MLC buffer—but if presented with sustained heavy writes for too long a time, the fast MLC buffer fills, and throughput drops to what the slower TLC or QLC media can manage. This can frequently be a “fall off the cliff” type scenario, since the slower media will typically not only have to sustain ongoing writes, but do so while continuing to stream out the already-accepted writes from the fast MLC buffer.

Modeling storage access realistically

Now that we understand a little about the pain points in a storage system, it’s pretty obvious that we shouldn’t just use a simple tool like dd to read or write huge chunks of data—and generate huge numbers. Those huge numbers don’t really correlate very well with how each disk performs under more realistic workloads—so, we want to generate more realistic access patterns to test with.

This is where fio comes in. Fio is short for Flexible Input/Output tester and can be configured to model nearly any storage workload under the sun. Real storage engineers—at least, the ones who are doing their jobs right—will first analyze the actual storage access patterns of a server or service, then write fio scripts to model those exact patterns. In this way, they can test a disk or array not only for its general performance, but its performance as very specifically applicable to their exact workload.

We’re not going to be quite that specific here, but we will use fio to model and report on some key usage patterns common to desktop and server storage. The most important of these is 4K random I/O, which we discussed at length above. 4K random is where the pain lives—it’s the reason your nice fast computer with a conventional hard drive suddenly sounds like it’s grinding coffee and makes you want to defenestrate it in frustration.

Next, we look at 64K random I/O, in sixteen parallel processes. This is sort of a middle-of-the-road workload for a busy computer—there are a lot of requests for relatively small amounts of data, but there are also lots of parallel processes; on a modern system, that high number of parallel processes is good, because it potentially allows the OS to aggregate lots of small requests into a few larger requests. Although nowhere near as punishing as 4K random I/O, 64K random I/O is enough to significantly slow most storage systems down.

Finally, we look at high-end throughput—some of the biggest numbers you can expect to see out of the system—by way of 1MB random I/O. Technically, you could still get a slightly bigger number by asking fio to generate truly sequential requests—but in the real world, those are vanishingly rare. If your OS needs to write a couple of lines to a system log, or read a few KB of data from a system library, your “sequential” read or write immediately becomes, effectively, 1MB random I/O as it shares time with the other process.

Installing fio

Windows

You can find Windows installers for fio at https://bsdio.com/fio/. Note that you may get Smartscreen warnings when running one of these installers, since they are not digitally signed. These packages are provided by Rebecca Cran and are available without warranty.

Note that Windows has a limited selection of ioengines available, which will inform your selection of command line arguments later. For the most part, Windows users should use --ioengine=windowsaio (Asynchronous Input/Output) with their fio arguments.

Linux / FreeBSD

The instructions for users of Linux and BSD distributions are a little different from one to another, but fio is in nearly all main repositories—so it boils down to <package manager> install fio for the vast majority.

Debian or Ubuntu: sudo apt install fio

FreeBSD: sudo pkg install fio

CentOS (and Red Hat Enterprise Linux) have rather more limited main repositories than most distributions; if you haven’t already, you’ll need to add the EPEL repository to CentOS/RHEL to get fio.

CentOS/RHEL: sudo yum install epel-release -y ; sudo yum install fio

You get the idea.

MacOS

On a Mac, you’ll want to install fio via brew. If you don’t already have brew installed, at the Terminal, issue the following command:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

On the one hand, the above is abominable procedure; on the other hand, you can confirm that the script being pulled down tells you everything it’s going to do, before it does it, and pauses to allow you to consent to it. If you’re sufficiently paranoid, you may wish to download the file, inspect it, and then run it as separate steps instead. Note that the homebrew install script does not need sudo privileges—and will, in fact, refuse to run at all if you try to execute it with sudo.

With Brew installed, you can now install fio easily:

brew install fio

Using fio

Now you can use fio to benchmark storage. First, change directory to the location you actually want to test: if you run fio in your home directory, you’ll be testing your computer’s internal disk, and if you run it in a directory located on a USB portable disk, you’ll be benchmarking that portable disk. Once you’ve got a command prompt somewhere in the disk you want to test, you’re ready to actually run fio.

Baby’s first fio run

First, we’ll examine the syntax needed for a simple 4K random write test. (Windows users: substitute --ioengine=windowsaio for --ioengine=posixaio in both this and future commands.)

fio --name=random-write --ioengine=posixaio --rw=randwrite --bs=4k --numjobs=1 --size=4g --iodepth=1 --runtime=60 --time_based --end_fsync=1

Let’s break down what each argument does.

--name= is a required argument, but it’s basically human-friendly fluff—fio will create files based on that name to test with, inside the working directory you’re currently in.

--ioengine=posixaio sets the mode fio interacts with the filesystem. POSIX is a standard Windows, Macs, Linux, and BSD all understand, so it’s great for portability—although inside fio itself, Windows users need to invoke --ioengine=windowsaio, not --ioengine=posixaio, unfortunately. AIO stands for Asynchronous Input Output and means that we can queue up multiple operations to be completed in whatever order the OS decides to complete them. (In this particular example, later arguments effectively nullify this.)

--rw=randwrite means exactly what it looks like it means: we’re going to do random write operations to our test files in the current working directory. Other options include seqread, seqwrite, randread, and randrw, all of which should hopefully be fairly self-explanatory.

--bs=4k blocksize 4K. These are very small individual operations. This is where the pain lives; it’s hard on the disk, and it also means a ton of extra overhead in the SATA, USB, SAS, SMB, or whatever other command channel lies between us and the disks, since a separate operation has to be commanded for each 4K of data.

--size=4g our test file(s) will be 4GB in size apiece. (We’re only creating one, see next argument.)

--numjobs=1 we’re only creating a single file, and running a single process commanding operations within that file. If we wanted to simulate multiple parallel processes, we’d do, eg, --numjobs=16, which would create 16 separate test files of --size size, and 16 separate processes operating on them at the same time.

--iodepth=1 this is how deep we’re willing to try to stack commands in the OS’s queue. Since we set this to 1, this is effectively pretty much the same thing as the sync IO engine—we’re only asking for a single operation at a time, and the OS has to acknowledge receipt of every operation we ask for before we can ask for another. (It does not have to satisfy the request itself before we ask it to do more operations, it just has to acknowledge that we actually asked for it.)

--runtime=60 --time_based Run for sixty seconds—and even if we complete sooner, just start over again and keep going until 60 seconds is up.

--end_fsync=1 After all operations have been queued, keep the timer going until the OS reports that the very last one of them has been successfully completed—ie, actually written to disk.

Interpreting fio’s output

This is the entire output from the 4K random I/O run on my Ubuntu workstation:

root@banshee:/tmp# fio --name=random-write --ioengine=posixaio --rw=randwrite --bs=4k --size=4g --numjobs=1 --runtime=60 --time_based --end_fsync=1
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio
fio-3.12
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][eta 00m:00s]                          
random-write: (groupid=0, jobs=1): err= 0: pid=16109: Wed Feb  5 15:09:36 2020
  write: IOPS=32.5k, BW=127MiB/s (133MB/s)(8192MiB/64602msec); 0 zone resets
    slat (nsec): min=250, max=555439, avg=1388.31, stdev=833.19
    clat (nsec): min=90, max=20251k, avg=9642.34, stdev=179381.02
     lat (usec): min=3, max=20252, avg=11.03, stdev=179.39
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    4], 10.00th=[    4], 20.00th=[    5],
     | 30.00th=[    6], 40.00th=[    6], 50.00th=[    7], 60.00th=[    8],
     | 70.00th=[    9], 80.00th=[   10], 90.00th=[   11], 95.00th=[   12],
     | 99.00th=[   17], 99.50th=[   20], 99.90th=[   43], 99.95th=[   77],
     | 99.99th=[12387]
   bw (  KiB/s): min=22256, max=613312, per=100.00%, avg=335527.28, stdev=162778.06, samples=50
   iops        : min= 5564, max=153328, avg=83881.88, stdev=40694.66, samples=50
  lat (nsec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (usec)   : 2=0.01%, 4=13.96%, 10=68.85%, 20=16.68%, 50=0.41%
  lat (usec)   : 100=0.04%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=6.35%, sys=11.96%, ctx=2348924, majf=0, minf=48
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,2097153,0,1 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=127MiB/s (133MB/s), 127MiB/s-127MiB/s (133MB/s-133MB/s), io=8192MiB (8590MB), run=64602-64602msec

Disk stats (read/write):
    md0: ios=71/749877, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=351/737911, aggrmerge=0/12145, aggrticks=1875/260901, aggrin_queue=30910, aggrutil=83.73%
  sdb: ios=342/737392, merge=0/12663, ticks=1832/241034, in_queue=28672, util=83.35%
  sda: ios=361/738430, merge=0/11628, ticks=1918/280768, in_queue=33148, util=83.73%

This may seem like a lot. It is a lot! But there’s only one piece you’ll likely care about, in most cases—the line directly under “Run status group 0 (all jobs):” is the one with the aggregate throughput. Fio is capable of running as many wildly different jobs in parallel as you’d like to execute complex workload models. But since we’re only running one job group, we’ve only got one line of aggregates to look through.

Run status group 0 (all jobs):
  WRITE: bw=127MiB/s (133MB/s), 127MiB/s-127MiB/s (133MB/s-133MB/s), io=8192MiB (8590MB), run=64602-64602msec

First, we’re seeing output in both MiB/sec and MB/sec. MiB means “mebibytes”—measured in powers of two—where MB means “megabytes,” measured in powers of ten. Mebibytes—1024×1024 bytes—are what operating systems and filesystems actually measure data in, so that’s the reading you care about.

Run status group 0 (all jobs):
  WRITE: bw=127MiB/s (133MB/s), 127MiB/s-127MiB/s (133MB/s-133MB/s), io=8192MiB (8590MB), run=64602-64602msec

In addition to only having a single job group, we only have a single job in this test—we didn’t ask fio to, for example, run sixteen parallel 4K random write processes—so although the second bit shows minimum and maximum range, in this case it’s just a repeat of the overall aggregate. If we’d had multiple processes, we’d see the slowest process to the fastest process represented here.

Run status group 0 (all jobs):
  WRITE: bw=127MiB/s (133MB/s), 127MiB/s-127MiB/s (133MB/s-133MB/s), io=8192MiB (8590MB), run=64602-64602msec

Finally, we get the total I/O—8192MiB written to disk, in 64602 milliseconds. Divide 8192MiB by 64.602 seconds, and surprise surprise, you get 126.8MiB/sec—round that up to 127MiB/sec, and that’s just what fio told you in the first block of the line for aggregate throughput.

If you’re wondering why fio wrote 8192MiB instead of only 4096MiB in this run—despite our --size argument being 4g, and only having one process running—it’s because we used --time_based and --runtime=60. And since we’re testing on a fast storage medium, we managed to loop through the full write run twice before terminating.

You can cherry pick lots more interesting stats out of the full fio output, including utilization percentages, IOPS per process, and CPU utilization—but for our purposes, we’re just going to stick with the aggregate throughput from here on out.

Ars recommended tests

Single 4KiB random write process

fio --name=random-write --ioengine=posixaio --rw=randwrite --bs=4k --size=4g --numjobs=1 --iodepth=1 --runtime=60 --time_based --end_fsync=1

This is a single process doing random 4K writes. This is where the pain really, really lives; it’s basically the worst possible thing you can ask a disk to do. Where this happens most frequently in real life: copying home directories and dotfiles, manipulating email stuff, some database operations, source code trees.

When I ran this test against the high-performance SSDs in my Ubuntu workstation, they pushed 127MiB/sec. The server just beneath it in the rack only managed 33MiB/sec on its “high-performance” 7200RPM rust disks… but even then, the vast majority of that speed is because the data is being written asynchronously, allowing the operating system to batch it up into larger, more efficient write operations.

If we add the argument --fsync=1, forcing the operating system to perform synchronous writes (calling fsync after each block of data is written) the picture gets much more grim: 2.6MiB/sec on the high-performance SSDs but only 184KiB/sec on the “high-performance” rust. The SSDs were about four times faster than the rust when data was written asynchronously but a whopping fourteen times faster when reduced to the worst-case scenario.

16 parallel 64KiB random write processes

fio --name=random-write --ioengine=posixaio --rw=randwrite --bs=64k --size=256m --numjobs=16 --iodepth=16 --runtime=60 --time_based --end_fsync=1

This time, we’re creating 16 separate 256MB files (still totaling 4GB, when all put together) and we’re issuing 64KB blocksized random write operations. We’re doing it with sixteen separate processes running in parallel, and we’re queuing up to 16 simultaneous asynchronous ops before we pause and wait for the OS to start acknowledging their receipt.

This is a pretty decent approximation of a significantly busy system. It’s not doing any one particularly nasty thing—like running a database engine or copying tons of dotfiles from a user’s home directory—but it is coping with a bunch of applications doing moderately demanding stuff all at once.

This is also a pretty good, slightly pessimistic approximation of a busy, multi-user system like a NAS, which needs to handle multiple 1MB operations simultaneously for different users. If several people or processes are trying to read or write big files (photos, movies, whatever) at once, the OS tries to feed them all data simultaneously. This pretty quickly devolves down to a pattern of multiple random small block access. So in addition to “busy desktop with lots of apps,” think “busy fileserver with several people actively using it.”

You will see a lot more variation in speed as you watch this operation play out on the console. For example, the 4K single process test we tried first wrote a pretty consistent 11MiB/sec on my MacBook Air’s internal drive—but this 16-process job fluctuated between about 10MiB/sec and 300MiB/sec during the run, finishing with an average of 126MiB/sec.

Most of the variation you’re seeing here is due to the operating system and SSD firmware sometimes being able to aggregate multiple writes. When it manages to aggregate them helpfully, it can write them in a way that allows parallel writes to all the individual physical media stripes inside the SSD. Sometimes, it still ends up having to give up and write to only a single physical media stripe at a time—or a garbage collection or other maintenance operation at the SSD firmware level needs to run briefly in the background, slowing things down.

Single 1MiB random write process

fio --name=random-write --ioengine=posixaio --rw=randwrite --bs=1m --size=16g --numjobs=1 --iodepth=1 --runtime=60 --time_based --end_fsync=1

This is pretty close to the best-case scenario for a real-world system doing real-world things. No, it’s not quite as fast as a single, truly contiguous write… but the 1MiB blocksize is large enough that it’s quite close. Besides, if literally any other disk activity is requested simultaneously with a contiguous write, the “contiguous” write devolves to this level of performance pretty much instantly, so this is a much more realistic test of the upper end of storage performance on a typical system.

You’ll see some kooky fluctuations on SSDs when doing this test. This is largely due to the SSD’s firmware having better luck or worse luck at any given time, when it’s trying to queue operations so that it can write across all physical media stripes cleanly at once. Rust disks will tend to provide a much more consistent, though typically lower, throughput across the run.

You can also see SSD performance fall off a cliff here if you exhaust an onboard write cache—TLC and QLC drives tend to have small write cache areas made of much faster MLC or SLC media. Once those get exhausted, the disk has to drop to writing directly to the much slower TLC/QLC media where the data eventually lands. This is the major difference between, for example, Samsung EVO and Pro SSDs—the EVOs have slow TLC media with a fast MLC cache, where the Pros use the higher-performance, higher-longevity MLC media throughout the entire SSD.

If you have any doubt at all about a TLC or QLC disk’s ability to sustain heavy writes, you may want to experimentally extend your time duration here. If you watch the throughput live as the job progresses, you’ll see the impact immediately when you run out of cache—what had been a fairly steady, several-hundred-MiB/sec throughput will suddenly plummet to half the speed or less and get considerably less stable as well.

However, you might choose to take the opposite position—you might not expect to do sustained heavy writes very frequently, in which case you actually are more interested in the on-cache behavior. What’s important here is that you understand both what you want to test, and how to test it accurately.

Conclusions

Using fio is definitely an exercise for the true nerd (or professional). It won’t hold your hand, and although it provides incredibly detailed results, they’re not automatically made into pretty graphs for you.

If all of this feels like far too much work, you can also find simpler-to-use graphical tools, such as HD Tune Pro for Windows. HD Tune Pro costs $35, or there’s a limited-capability non-Pro version that is free for personal use. It’s a good tool, and it’ll make shiny graphs—but it’s considerably more limited for advanced users, and the price of the make-it-easy user interface is that you’re much further removed from the technical reality of what you’re doing.

Learning to use fio means really learning the difference between asynchronous and synchronous writes and knowing for absolute certain what it’s going to do at a very low level on an individual argument basis. You can’t be as certain of what tools like HD Tune Pro are actually doing under the hood—and having to deal with different tools on different operating systems means it’s more difficult to directly compare and contrast results as well.

Continue Reading

Gadgets

All the little things that add up to make iPadOS productivity a pain

Published

on

Rumor has it a new iPad Pro is around the corner, which means Apple is about to make another big pitch for the iPad as a productivity and content-creation device.

But while we’ve found in our iPadOS reviews that Apple has done a marvelous job with the big-picture changes to the OS aimed at making it real-work-friendly, there are still a bunch of minor annoyances or “nope, you can’t do that” limitations that sabotage Apple’s intentions.

For that reason, it makes sense to preempt that upcoming marketing push with a few key caveats—especially since Apple likely won’t announce a major iPadOS software update alongside new hardware in March. Significant new OS changes probably won’t be discussed until the company’s developer conference in June, and said updates probably won’t reach the public until September or October.

Most of these are tiny problems, but they add up. iPads won’t be a real laptop replacement for everyone until most of these issues are addressed.

Webcams and multitasking

It won’t take you long in current computing use cases to notice this one: the front-facing camera on the iPad shuts off when you swipe away from whatever app is using it, Zoom included. Reviewers have brought this up again and again when reviewing recent iPads—us included. But 12 iPadOS updates later, it’s still an issue.

Granted, some applications will show your camera view in a small, picture-in-picture window over other apps when you switch spaces. But you don’t always want to see that—screen real estate is at a serious premium on iPads—and not every app does it.

Where third-party apps don’t support the picture-in-picture view, Apple needs to find a way to incentivize them to do so. But better yet: allow users to enable background video capture on a per-app basis in Settings.

A lot of people are spending a great deal of time on video calls these days, for obvious reasons. It’d be great if Apple’s flagship mainstream dedicated computing product actually did that well.

Audio-source management

Obviously, the iPad does support background audio. Apps like Apple Music or Spotify can play in the background, as can some (but not all) video apps. The problem is that it’s all too easy for the currently active app to silence the one in the background, because two audio sources usually cannot play at the same time.

So for example, if you are watching a Twitch stream in the background but an autoplaying video with audio comes up on a webpage, your Twitch stream will stop. You’ll have to stop the video on the Web, then go back to the Twitch app to start it up again. And sometimes, websites or apps take over your audio even if they aren’t apparently making any sound at present.

At a minimum, the iPad should either not stop the first audio source when this happens or at least resume playing whatever was playing in the background once the new audio sources starts. But the ideal situation would be a panel for managing multiple audio sources at once by app, including their levels.

External monitors

When Apple first announced that the iPad Pro would be able to work with external monitors via USB-C as part of an overall pitch of the Pro as a heavy-duty productivity and content-creation device, many users expected something very different than what they got.

Yes, you can hook your iPad Pro up to an external USB-C monitor. But typically, all it does is mirror the iPad’s display. It doesn’t give you more spaces for apps, and it doesn’t even adopt the aspect ratio of the screen you’re sending the image to.

There is a very small number of iPad apps, like iMovie, that let you use the external monitor a little differently. But the vast majority don’t, making external monitor support essentially useless on Apple’s tablet.

The limitations of the USB-C port

The move to USB-C from Lightning in recent iPads is a welcome one, even if it means some people had to buy some new cables. The ecosystem of USB-C accessories—like external storage devices, monitors, music production tools, and so on—is quite robust compared to what we get on Lightning.

Enlarge / The iPad Pro has USB-C instead of Lightning as its one port.

So we’re not knocking USB-C here. We’re knocking how many USB-C ports there are. The iPad Pro only has one, and all too often, it doesn’t play nicely with external USB-C hubs that you might normally use with a Mac. Users complain of constant disconnects and inconsistent behavior. Some hubs just don’t work at all.

If Apple can’t rely on other companies like CalDigit to do this well, and if it really must insist on not adding at least one more port, then it needs to release its own USB-C dock that is guaranteed to work smoothly with the iPad.

It certainly didn’t help that Apple removed the headphone jack from recent iPads. Some of the advertising around the iPad Pro centered on music production, but good luck producing music when you can’t easily connect both an instrument and headphones at the same time.

You’ll need a dongle, which is expensive and a hassle, and a whole lot of them don’t work well.

Pro app support

A computer is only as good as the apps it can run, of course. And while the iPad has many excellent apps for consumers of content, many good productivity apps, and several good tools for hobbyists in various creative disciplines, users of apps that are popular in certain professional contexts face a significant gap between iPadOS and either macOS or Windows.

And it’s not just from third parties. Apple’s own Final Cut, Logic, and Xcode are not available on the iPad. There aren’t a ton of great options from other companies either. Yes, Adobe has been working on fairly robust versions of both Photoshop and Illustrator for the iPad. But we haven’t heard a word about Premiere, for example.

And there are numerous widely used pro apps from other companies that aren’t available. There’s no Maya, no Blender, no Unity, no Visual Studio. There is an AutoCAD app, but it’s minimally functional compared to the desktop version.

If Apple is going to keep calling the iPad a device for professional content creators, it needs to convince these third parties to release more functional iPad apps. And at least as importantly, it needs to adapt its own software for the device.

How likely are we to see these changes?

For years, Apple has moved further away from the idea of more ports, large feature sets, and so on—particularly on mobile devices like the iPad. So in the past, we wouldn’t have expected most of these things to actually happen.

However, the world of Apple devices looks noticeably different in the wake of reliable reports that new MacBook Pro laptops later this year will include SD cards or HDMI ports. Apple seems to be changing course to better court high-end and certain pro use-case customers. Well, at least as far as the Mac is concerned.

The company has also moved aggressively in other ways on the iPad front, at least in terms of software—just maybe not as fast as everyone would like. It seems plausible to us that multitasking issues (like those with webcams and audio sources) may be fixed in the future. And at this point, never say never to iPad versions of Logic or Final Cut.

We’re less bullish on the idea of a multiport iPad Pro, though, and there’s only so much Apple can do to attract third parties to make more robust apps for the platform.

Apple is expected to announce a new iPad Pro before the end of March, so we’ll have a hint at what’s to come soon enough.

Listing image by Samuel Axon

Continue Reading

Gadgets

Hard-coded key vulnerability in Logix PLCs has severity score of 10 out of 10

Published

on

Rockwell Automation

Hardware that is widely used to control equipment in factories and other industrial settings can be remotely commandeered by exploiting a newly disclosed vulnerability that has a severity score of 10 out of 10.

The vulnerability is found in programmable logic controllers from Rockwell Automation that are marketed under the Logix brand. These devices, which range from the size of a small toaster to a large bread box or even bigger, help control equipment and processes on assembly lines and in other manufacturing environments. Engineers program the PLCs using Rockwell software called Studio 5000 Logix Designer.

On Thursday, the US Cybersecurity & Infrastructure Security Administration warned of a critical vulnerability that could allow hackers to remotely connect to Logix controllers and from there alter their configuration or application code. The vulnerability requires a low skill level to be exploited, CISA said.

The vulnerability, which is tracked as CVE-2021-22681, is the result of the Studio 5000 Logix Designer software making it possible for hackers to extract a secret encryption key. This key is hard-coded into both Logix controllers and engineering stations and verifies communication between the two devices. A hacker who obtained the key could then mimic an engineering workstation and manipulate PLC code or configurations that directly impact a manufacturing process.

“Any affected Rockwell Logix controller that is exposed on the Internet is potentially vulnerable and exploitable,” said Sharon Brizinov, principal vulnerability researcher at Claroty, one of three organizations Rockwell credited with independently discovering the flaw. “To successfully exploit this vulnerability, an attacker must first obtain the secret key and have the knowledge of the cryptographic algorithm being used in the authentication process.”

Brizinov said that Claroty notified Rockwell of the vulnerability in 2019. Rockwell didn’t disclose it until Thursday. Rockwell also credited Kaspersky Lab and Soonchunhyang University researchers Eunseon Jeong, Youngho An, Junyoung Park, Insu Oh, and Kangbin Yim.

The vulnerability affects just about every Logix PLC Rockwell sells, including:

  • CompactLogix 1768
  • CompactLogix 1769
  • CompactLogix 5370
  • CompactLogix 5380
  • CompactLogix 5480
  • ControlLogix 5550
  • ControlLogix 5560
  • ControlLogix 5570
  • ControlLogix 5580
  • DriveLogix 5560
  • DriveLogix 5730
  • DriveLogix 1794-L34
  • Compact GuardLogix 5370
  • Compact GuardLogix 5380
  • GuardLogix 5570
  • GuardLogix 5580
  • SoftLogix 5800

Rockwell isn’t issuing a patch that directly addresses the problems stemming from the hard-coded key. Instead, the company is recommending that PLC users follow specific risk mitigation steps. The steps involve putting the controller mode switch into run, and if that’s not possible, following other recommendations that are specific to each PLC model.

Those steps are laid out in an advisory Rockwell is making available to customers, as well as in the above-linked CISA advisory. Rockwell and CISA also recommend PLC users follow standard security-in-depth security advice. Chief among the recommendations is ensuring that control system devices aren’t accessible from the Internet.

Security professionals universally admonish engineers to place critical industrial systems behind a firewall so they aren’t exposed to the Internet. Unfortunately, engineers struggling with high workloads and limited budgets often don’t heed the advice. The latest reminder of this came earlier this month when a municipal water treatment plant in Florida said that an intruder accessed a remote system and tried to lace drinking water with lye. Plant employees used the same TeamViewer password and didn’t put the system behind a firewall.

If Logix PLC users are segmenting industrial control networks and following other best practices, it’s unlikely that the risk posed by CVE-2021-22681 is minimal. And if people haven’t implemented these practices, hackers probably have easier ways to hijack the devices. That said, this vulnerability is serious enough that all Logix PLC users should pay attention to the CISA and Rockwell advisories.

Claroty has issued its own writeup here.

Continue Reading

Gadgets

Google’s Smart TV software will have a “dumb TV” mode

Published

on

The new Google TV is a fine smart TV interface, but when it gets integrated into some TV sets later this year, its best feature might be that you can turn it off. A report from 9to5Google details an upcoming “Basic TV” mode that will be built into Google TV, which turns off just about all the smart TV features. Right now, Google TV is only available in the new Chromecast, but Google TV will be built into upcoming TVs from Sony and TCL. Basic mode means we’ll get smart TVs with a “dumb TV” mode.

The rise of smart TVs has led to the extinction of dumb TVs—today, basically every TV has some kind of computer and operating system built into it. If you’re actually expecting to live with a TV for several years, the problem with smart TVs is that the dirt-cheap computers inside these TVs don’t last as long as the display does. When your smart TV is a few years old, you might still have a perfectly good display panel, but you’ll be forced to interact with it through a slow, old, possibly abandoned integrated computer. Companies should sell dumb TVs without any of this crap permanently integrated into them, but if they refuse, letting consumers turn off the software is the next best thing.

When the new feature rolls out, you’ll be asked to choose between “Basic TV” or “Google TV” at setup. 9to5Google says that with basic mode, “almost everything is stripped, leaving users with just HDMI inputs and Live TV if they have an antenna plugged directly into the TV. Casting support, too, is dropped.” The UI notes that you’ll be turning off all apps, the Google Assistant, and personalized recommendations.

9to5 found this feature via the ADT-3 development set-top box and the Android 12 developer preview, so it’s not entirely clear how it will work when it’s running on a real TV. It seems like basic mode will only show a minimal set of icons for things like input-switching and settings. There’s also a big banner advertising Google TV mode, which you’ll presumably just have to learn to ignore. A Google spokesperson told the site that this feature is destined to hit TVs sold with integrated Google TV in the future.

If you’re wondering what the difference is between “Android TV” and “Google TV,” Google TV is kind of like the next version of Android TV. Google TV is just the Android TV codebase with a new interface, which offers things like a unified search. The upgrade path for existing Android TV devices is Google TV, assuming your device manufacturer is actually shipping updates. By 2022, Google says TV manufacturers won’t be allowed to ship Android TV and will instead ship Google TV. There are some product lines that Google just loves to rebrand every few years, and Android TV/Google TV is one of them.

Google TV will be in Sony’s entire Bravia XR 2021 lineup and select TCL TVs later this year.

Continue Reading

Trending