Use the Force Touch
A great post from John Gruber. The actual reviews of the Vision Pro – including his – have been fascinating, for reasons you probably could have imagined beforehand. But his latest post hits on the type of insights that only trickle out after you start to settle into using such technology.
One generally underappreciated aspect of the iPhone is the accuracy of its finger-tracking for touch. From the beginning—the original iPhone on the day it launched—when you slide your finger on an iPhone to scroll, the scrolling tracks your finger movement exactly as though you were sliding a piece of paper. The delight began right from “Slide to Unlock”, because the knob you slid to unlock the iPhone tracked your finger precisely. It didn’t feel like you were issuing an indirect shortcut gesture; it felt (and to this day, of course, still feels) like you were directly manipulating whatever it was you were sliding or scrolling on screen. The bouncing when you scroll to the end of a view. The stretching when you pull down from the top of a view. It felt (and feels) real, to some degree.
I've talked about this generally before, but this "direct" manipulation, from your finger to the screen, is actually what had me walking out of an Apple store with an iPhone on day one. Not only had I not pre-ordered one, I was certain I wouldn't be getting one – it was way too expensive and big compared to my beloved Motorola RAZR (sound familiar?). But I was back in my hometown sort of wandering,1 so I figured I'd go to an Apple Store to at least see what all the fuss was about. It wasn't what I saw, but what I "felt" that led to the purchase.
You put your finger on the screen and it moves. Not the screen itself, but the pixels. Just rubbing your finger on glass allows this to happen instantly. From slide-to-unlock on forward, it felt like utter magic. At times, when you actually stop to admire it, it still does.
I had a suspicion since it was still just rumored that the Vision Pro hand and eye-tracking would completely change the interaction paradigm for all of these types of XR devices (even if Apple swears they're not the same thing). Just as multi-touch did for smartphones. As I wrote just over a year ago:
Much of the focus to date has been on the headset itself. But all of these devices need some sort of input. If it’s truly a mixture of advanced eye-tracking paired with finger gestures, this could be a legitimate breakthrough in the space. It sounds simple and that’s the point. While the Meta Quest 2 controllers are nice, they’re still controllers — something that makes using VR more like a game (even when not playing a game). That’s not likely to fly with “regular” folks. Apple’s entire history is basically creating (or re-purposing) interaction and input paradigms to move computing forward. This could be that again… if it works well, of course.
It seems to work quite well. Back to Gruber:
One comparison would be to a maestro conducting an orchestra. But that doesn’t convey the sense of precision in VisionOS — the sense of fine control through indirect manipulation. What it feels like is using the Force.
One of the first times we see any character use the Force in Star Wars is when Obi-Wan Kenobi uses the Jedi mind trick to convince the droid-hunting stormtroopers on the outskirts of Mos Eisley that R2-D2 and C-3PO are not, in fact, the droids they’re looking for. He even pinches his finger and thumb together while gesturing with his hand. It couldn’t be more apt an image.
It's funny that Gruber brings up the notion of a maestro, because that has been one of the unique types of apps I assumed would be perfect for the Vision Pro, given the hand tracking. But obviously all this Force business is just great. I've seen all the various versions of different Star Wars – from great to good to bad, and everything in between – dozens of times, if not more. Amazingly, I don't think I ever realized the finger-and-thumb element. Everyone knows the hand wave, but this is just a great, fun coincidence. Or perhaps not!
And, like the butterfly from Encounter Dinosaurs that some people can feel landing on their outstretched finger, I swear to you, I can almost feel the telekinetic connection with UI elements in VisionOS. It’s a hint, a whiff, of tension—between not just my hands and the virtual elements I’m manipulating, but between my mind and those elements. Just the vaguest sensation of tension emanating from my forehead, like a taut thread of ultrafine string connecting my mind to the window I’m moving, or button I’m pressing, or photo I’m stretching.
One other element that feels under-explored (at least in everything I've read) to date about the Vision Pro is related to the above. When a technology is this immersive, it certainly can start to play tricks on your mind. We've likely all played endless amounts of Tetris (or a number of other equivalently addictive, repetitive games) and later closed our eyes only to still see Tetris, right there, as if it was being projected on the inside of our eyelids. And that was on a 2.6" monochrome screen of a GameBoy. What is an infinite amazing technicolor screen literally being projected into our eyes going to do to our minds?
I don't mean this in a negative way, necessarily. I'm genuinely curious! "Feeling" butterflies tickle your fingers sounds wonderful. What else tickles what senses?
Setting aside potential trademark complaints from their friends at Disney, this is what Apple should have saved the term “Force Touch” for.
I've loved Force Touch since day one for some of the same reasons that Gruber puts into words here – it's technology that makes you literally feel, thanks to haptics, like you're doing something when you're not. Sadly, Apple has sort of squandered it. They totally should use it to rebrand this Vision Pro manipulation.
1 In a sort of weird, coincidental bit of timing, I'll be picking up the Vision Pro in Ohio -- which is exactly where I bought the iPhone. Even though I didn't actually live in Ohio either time. When the iPhone came out, I lived in San Diego. Now, I live in London. Still, I went back to Ohio...