Getting Started with VR Development in 2024

Where do you get started if you want to make the next great Mixed/Augmented/Virtual Reality experience?

Ever wish you can go back to the early days of the internet and make something ambitious? Did you see all these solo and small team games go big over the last decade and think “well, they had it easy when games were simple”? Do you want to make slick new tech for productivity like never before? In 2024, VR development offers the kind of new opportunity most coders might consider “once in a decade” if not once in a lifetime.

For me, it’s like stepping back to the advent of 3D graphics as far as the jump in what is possible, with the advantage of modern design tools for making new experiences. If you’re wanting to give development for this new medium a try — and I highly encourage it for anyone who wants to understand what the decades ahead for user experience might be like — then take a look at these resources. They are loosely grouped by technology and the direction one wants to take with building apps.

Trying out the Quest 3, Apple Vision Pro, and Playstation VR2 at Fuse in Richland, WA

This is one part persuasive essay on why developers should learn VR and two-parts quick guide of resources I’ve found while trying to satiate my own curiosity about eXtended Reality (XR — the proper umbrella term for Mixed/Augumented/Virtual Reality experiences) and how to do it with modern devices that have been released recently.

This article was originally written as a resource guide for the VR Hackathon at the TriTown Get Down Music Festival 2024 in Tri-Cities, WA, but should be applicable for at least the rest of the year. Comment with more resources to keep expanding the learning for our community!

Platforms in VR Development

To start wrapping your head around concepts in XR, especially if you’re a student coming to XR, you might want to do some general reading on introductory topics. For instance, there are different platforms for making experiences:

• PCVR — PCVR encompasses the broad notion of Windows-based video games that run on VR headsets, many of which are exclusive and well worth playing such as Half-Life: Alyx, Skyrim VR, to a lesser extent (but totally en vogue due to the TV series) Fallout 4 VR. PCVR offers the most raw capabilities for a platform, but keep in mind, one needs a very powerful computer to run PCVR games and the hardware itself
• Meta Quest — The Quest 3 is definitely the most popular standalone headset for getting started content on the web right now and my personally recommended device for anyone who has yet to buy any hardware. The Quest 2 is now an ultra-affordable hardware option for aspiring developers who don’t want to make the plunge, but remember, it can’t do mixed/augmented reality. The headsets themselves can also connect to a computer to do PCVR development and has a WebXR-enabled browser out-of-the-box, so it’s a very flexible device for development.
• Apple Vision Pro — The newly released Apple Vision Pro (AVP) is, in my perception, a software development kit for aspiring developers rather than a consumer product at the moment. It’s so much more capable than anything else, but so much more expensive. If you are reading this and can code, you’re probably the only viable market beyond the earliest of early adopters out there. It cannot connect to a PC, but can run iOS and Vision OS apps — plus has nascent game engine support in Unity. Additionally, with some power user options turned on, its browser can run WebXR experiences.

The three platforms above will be workable platforms for the VR Hackathon at TriTown Get Down 2024 as we have borrowed hardware and clearly identified no cost developer tooling for them

I would mention that the PSVR2 is a great balance of affordability and performance compared as far as playing VR games. I would especially encourage it for a family with multiple potential users who are sensitive to the higher-risk tech addiction aspects of VR. There are no AI/VR girlfriend apps or immersive porn options— at least to my knowledge — and if it’s tethered to your living room, you maintain the most control as a parent. However, the fact one needs a Playstation Development Kit to get started means it’s likely a not starter for enthusiasts getting into coding since it’s the cost of the AVP to get going and a much smaller audience compared to other platforms.

VR Software Development Toolsets

For instance, the notion that you can take many different paths to making a game:

• Native Development — Such as making an Android in VR native app on Quest or a VisionOS app for Apple Vision Pro (AVP). These seem most suited to non-gaming, productivity-oriented application. AVP developers who want in on the return of people actually paying for apps for Apple devices thanks to the AVP or prepare for a future where some of the B2B software economy will be built in XR, one will likely want to try native development.
• Game Engine — Development using a ready-made toolset using popular professional engines like Unity, Unreal, or the open source Godot. Each of those has XR features built-in plus strong community support both from the creators and online tutorials from the ecosystem at large — though Unity seems like the most robust starting place for beginners and Unity has support for AVP thanks to a partnership with Apple. This is definitely the best path for someone serious about making media and entertainment products and delivering them to a real app store.
• WebXR Development — If you want ultimate cross-platform support with lightning fast development time, you might try using the budding WebXR standard and the ecosystem on top of it to make the browsers pre-loaded into all of these devices to make your first XR experience. A framework like Babylon.js or web game engines like Wonderland Engine can get you started quickly and virtually any VR device can be used for live testing. The capabilities of WebXR are less robust, but fast iteration and broad reach may outweigh raw features for your idea. This is an especially good idea if you just want to understand VR design and have a nice proof-of-concept demo for your portfolio without necessarily wanting to make a “real product” out of it.

For a video summary of some of these points of consideration, check out this video on YouTube from Valem Tutorials who has more learning content I will reference later in this article around Unity development on Meta Quest. It runs over the platforms and options, with some gentle introduction to terms throughout the video to start you on the basics:

VR Beginner Video from Valem Tutorials to help you understand some concepts in getting started with VR development at large

From here, let’s get into individual devices and how to approach building your first apps for them. Given the options above, one has at least nine combinations, but truly there are only a few likely paths worth true consideration.

Development Hardware

One does need a personal computer of some kind (Mac, PC, Linux) to do VR development. Most recent machines can do it, though it seems like laptops need discrete graphics cards. One can install the Oculus Desktop App and it will check if your machine can run the OculusXR runtime. If it supports it, you can likely do VR development on that machine. If it doesn’t support it, you might still be able to do WebXR development on the machine.

The Default Path: Why Meta Quest?

The popular narrative about Meta’s entrance into the VR market is definitely that Mark Zuckerberg has set billions of dollars on fire to build the Metaverse, but not yet with any fruits for Meta to show for it. To me, that simply means that the Meta Quest 3 in front of me isn’t worth the $499 of my money I might have paid for it. It’s worth $46 billion dollars of Mark’s money that he paid for it.

The Meta Quest is not only the arguable champion of price-to-features in the VR market right now, but it has the most dynamic software ecosystem. There is an estimated 20 million Quest 2 headsets sold at time of publishing and at least 1 million Quest 3 headsets sold thus far. More importantly, the collective Meta Quest software ecosystem is reported to have passed $2 billion cumulative sales; the kind of ecosystem Steam might have had back in 2008 or 2009 based on estimates, back when it was niche but about to go mainstream.

While that may seem small, the most important aspect is the growth of the platform. If I could go back in time fifteen years, taking the software development tools of the modern age with me, to sell my competently made indie game on Steam, I would undoubtedly have turned out to be one of the savvier game devs of my time going in against only the first wave of what has become the most exciting segment of gaming. Well, Meta is making the same opportunity for anyone willing to sit down and learn today.

Wait — why not PCVR as the default path??? I remain a PC gamer and know that PCVR technically has more headsets and more features; however, no one has been willing to light a pile of money on fire to make sustained growth in PCVR and the developer experience using the Quest as your platform does largely translate to learning PCVR, but with a much faster time-to-feedback on making a unique VR experience.

Apple has also notably burnt about $10 billion on making the first iteration of the Apple Vision Pro — so consider it is still worth more than you pay for it despite the steep price, as long as you can make something with it.

The Default Path: Getting Started on Meta Quest with Unity

The largest ecosystem at the intersection of hardware and software development kit (SDK) in VR at the moment appears to be Unity development on the Meta Quest, thus if you want a “default path” I would recommend starting there. Quick links to get started would be:

1. Research Unity VR Capabilities a bit to understand the capabilities
2. Download Unity onto Your Windows, Mac, or Linux machine
3. Bookmark and start reading the official Unity VR Development Docs
4. Find tutorials, articles or videos, online to follow for getting started content, paying special attention to when they were made to avoid potentially learning old techniques

In the Unity store, you’ll find the modern Meta XR All-In-One SDK that is the new starting point for Unity apps targeting Oculus apps for VisionOS 65 or later. Meta also offers the Building Blocks for starting points for key elements of VR/AR games in Unity.

Valem Tutorials video on using Meta Building Blocks to build a multiplayer games without any coding

Once you have tried some tutorials, the best way to find more advanced examples is open source projects from which you can come to understand more complex examples of experiences.

• Oculus Samples repo on GitHub which has native, Unreal, and Unity examples,
• Valem Tutorials list of best open source VR unity projects
• Open source apps on the SideQuest Oculus Game directory where you can find unpublished Quest apps, many of which have open source projects you can find as links from their SideQuest entries or by searching the web for the author or game by name

PCVR with Unity

If you would like to focus on the more general PCVR rather than Meta Quest specifically — you can just plug a Quest headset (or HTC Vive, Valve Index, etc) into a PC. You might try starting with the Valem Tutorial’s video on “Learning VR Development in 3 Hours — Unity VR Tutorial Complete Course”, which uses a slightly different configuration of packages in Unity to make a more general-purpose VR experience.

PCVR with Godot

If you want to make Richard Stallman proud and, more importantly, ensure you don’t rely on a large company that can change its fee structure at any time to ruin the business potential for your app or build on a platform by a tech giant known for privacy concerns, then making a PCVR app using Godot Engine would be the ideal starting point. The Build a VR Game in Godot video series seems like the most comprehensive starting point.

Apple Vision Pro with VisionOS

If you want to try building an Apple Vision Pro app, I would recommend :

• If you want to make AR apps for AVP, then check out the Apple VisionOS documentation then check out some VisionOS app tutorials online. Again, make sure you have an Apple MacBook running Apple Silicon (the M1, M2, etc series processors).
• If you want to make immersive experiences for Apple Vision Pro, check out the Unity PolySpatial features, which enables 3D experiences on Apple Vision Pro. As of this writing, it seems they have made it a pro feature, so you have only have 30-days of free trial to dig in if you don’t want to start paying.

Conclusion

As you might guess, I’m a big believer in XR as an important future media. Personally, I’ve replaced all of my gaming with VR where I don’t just have fun, but I also burn calories like never before. I have also witness interactive storytelling, live sports, and virtual concerts that have opened my eyes to what it possible when you bridge old and new media. Furthermore, I believe the tools to join this revolution are easier than ever to try for yourself.

Erik Ralston is Co-Founder & CTO at Soundbite.AI and is the lead for the STEMFusion at TriTown Get Down 2024 where he coordinated the first VR Hackathon in the history of Tri-Cities, WA. He is an innovator with 18 years of education and experience, having spent the last five years in leadership at the fastest-growing tech company in Australia. Erik is also co-founder of Fuse Accelerator in the emerging community of Tri-Cities, WA, where he works on connecting people and sharing knowledge to turn new ideas into growing startups. You can find him on LinkedIn, Twitter, or the next Fuse event.