I thought this new VisionOS 2 feature was just a gimmick – until it made me cry in Vision Pro

I always find it a little disconcerting when an article I’ve carefully planned doesn’t go as planned. It doesn’t happen often, but sometimes an article can take on a life of its own.

This was one such article. The mission was simple. In honor of VisionOS 2’s soft launch this week, I had to write about my thoughts on the best VisionOS 2 features available right now (and what’s overdue).

Basically, I was disappointed. The new hand gestures don’t seem to work if, like me, you have some accessibility features turned on.

Virtual Ultra Widescreen for Mac, which I still expect to be the killer app for Vision Pro, isn’t available until “later this year.”

Also: The best VR headsets of 2024

Sure, there’s mouse support now. The new Bora Bora environment is beautiful. And you can rearrange the icons in your home view. But, yawn.

A bad trick (or so I thought)

And then, I’ll have to mention the dumbest new feature of all, a completely stupid gimmick. This is the showcase feature of carnival where you can create spatial images from your 2D photos.

Except, well, that stupid trick is great at making me cry. Serious.

I’m struggling to write about this because this feature is the most non-essential use of advanced technology ever. And it’s certainly not a justification for the purchase a $3,500 helmet.

Also: Meta Quest 3 review: The VR headset most people should buy in 2024

But…after experiencing it yesterday and today, it might also be the first thing you want to show someone if you’re trying to show off your painfully overpriced brick.

I mean, this is some Marty McFly Darth Vader from the planet Vulcan, the Ghost of Christmas Past, mind-blowing stuff.

It’s a 30-second theme park ride that can knock you 20 years into the past, rip your guts out, and throw you headfirst into an amazingly surreal valley of your own mind.

I’m not exaggerating. This is not hyperbole. Once you’ve strapped your headset to your head, launched the VisionOS 2.0 Photos app and clicked the little magic button, get ready for a digital acid trip that Ken Kesey and the Merry Pranksters would be proud of.

Okay, let’s back up a bit and put this into context. VisionOS 2.0 includes a feature in the Photos app that uses artificial intelligence and machine learning to transform a flat photo into something that looks fully 3D.

Also: Stanford’s VR breakthrough could spell the end of clunky headsets — thanks to artificial intelligence

Unfortunately, nothing I could screenshot and show you here will do the experience justice. In fact, the only way you’ll be able to fully understand what Apple has done is to put on a Vision Pro and try it out.

What I can do is try to describe the process of what happens and how I felt after following the white rabbit down the rabbit hole.

All of this happens inside the updated Photos app. Once you open an image, you will see a small 3D cube button like this:

When you press the button, a light shadow flows across the image, the subject is highlighted, and in about five seconds, a 3D version of the image is created.

I want to be clear here. This is not a 3D scan that you can use to recreate the subject on a 3D printer. All it does is tweak the pixels a bit so that if you move your head, you see a sense of depth.

But that description doesn’t do justice to the results. When you have the right subject, pressing the little button makes what you’re looking at go from a photo to real. I’m not even sure how to describe it properly because the effect isn’t just 3D. It’s almost an achievement button. It somehow manifests the subject of the image so that you’re suddenly looking at the real thing.

When you tap the rendered image, it expands into the entire environment, surrounded by a ghostly fog. I only noticed one case of artifact, with a few streaks under my Pixel Cub’s right ear.

When I look at this image through Vision Pro, it looks like I’m really looking at the Pixel. But because the image doesn’t move, it’s almost like looking at an animal in a taxidermy shop. It wasn’t too bad because the pup was sitting on my lap while using the device.

But then I got the idea that I wanted to see Sammy again. Samantha was my cat. She lived in the late 1980s and 1990s and died in 2001. I adored her and couldn’t bring myself to look at the few snapshots I have of her because I missed her so much.

Also: We’ve used every iPhone 16 model and here are the best buying tips for 2024

So in some ways this was a very baaad idea. It hit me right in the feels.

But I wanted to see how an old printed film would respond to the 3D ID process or if it would work. I pulled out some scans I had previously made of my film negatives and imported them into the Photos app.

And then I opened the pictures on Vision Pro and made them.

Did you know that if you cry inside the Vision Pro, your tears pool inside? There is not much room for the liquid to drain.

It wasn’t moving, but it was real. She was There. And in Vision Pro, she also looked the size of a sofa.

Review: Apple Vision Pro: Fascinating, Flawed, and Needs to Fix 5 Things

I felt like I could reach out and touch her. It was a very trippy experience. I miss my little girl, but I didn’t expect this level of reaction.

I looked at three or four more pictures of her, all made ghostly by the extended effect, and I was floored. A big, strong, bearded guy crying over a kitten he hasn’t seen in over two decades. It’s embarrassing. But that’s the unexpected power of this feature.

What works and what doesn’t

I found some pictures of me from a few years ago. While the 3D effect was there, the shots of me as an adult weren’t terribly impressive. We also tried a number of images that had strong subjects, such as a close-up of an F-14 Tomcat, a tugboat, and a helicopter.

In addition, we tried a few scenic views to see how they would perform. They all had that cheap feel you get from one of those old View-Master discs. They weren’t that special.

Also: My biggest regret when I updated my iPhone to iOS 18 (and I’m not alone)

But photos of very sharply defined small subjects, such as pets and toys and stuffed animals, work extremely well. I imagine pictures of babies and possibly small children might stand out.

Actually, this picture of my little one did pretty well. The original scan was only 800×600. It was too small for 3D conversion, so the little cube button was missing. Actually, I wasn’t sure if the button was missing because the image was black and white or too small. To test, I brought the image into Photoshop and used Photoshop’s smart resizing features to enlarge it by 4X.

That worked. It wasn’t as real, but you could see that the “me” part of the image was in front of the house in the background (which, if I remember correctly, was my friend Jonathan’s).

After some testing, it’s clear that this feature works on any image (black and white or color) that has enough pixel density to give it something to work with.

A surprisingly strong impact

Overall, the VisionOS 2.0 update is a nice little update. The killer productivity feature, the ultrawide virtual desktop, isn’t available yet, which greatly reduces the value of the upgrade. However, any improvements to the original VisionOS are welcome.

But the standout feature was definitely the ability to convert existing images to 3D. I really had no respect for the idea. But after trying it out, I found it to be one of the most impressive showcases of what is or could be possible with immersive VR.

Also: How VR and digital twins are streamlining remote work in technical professions

I strongly recommend showing the 2D to 3D conversion feature to new people who are curious about Vision Pro. Seeing someone else’s immersive 3D experience is one thing. That is impressive. But seeing your long-gone, much-loved kitty appear as if she were real right in front of your face had a visceral impact that demo content can’t touch.

It’s not a feature anyone would use regularly. I’m not even sure it would be emotionally healthy to use it regularly. But it certainly shows the impact and power of immersive 3D unlike anything else we’ve tried. That’s because it’s something old, familiar and personal like an old photograph and it gives it life. Damn if it didn’t feel a little like magic.

But it also came at a price. Seeing Sammy there had an emotional impact I wasn’t expecting. I’m still shaking from her. I’m really glad I had the chance to experiment, but it’s not something I want to try again anytime soon.

Also: Who’s Afraid of VR? I was – until I tried Meta Quest 3

what do you think Do you have a Vision Pro? Have you upgraded to 2.0 yet? Have you tried any of the new features? Let us know in the comments below.

You can follow my daily project updates on social media. Be sure to subscribe to my weekly update newsletter and follow me on Twitter/X at @DavidGewirtzon Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz and on YouTube at YouTube.com/DavidGewirtzTV.