[Resource] AR & VR Tools

Technology is enabling our world faster than ever. From my first Nintendo to mobile devices that can answer any question I have we are experiencing rapid changes to human behavior at a extremely fast pace. Augmented Reality and Virtual Reality are one the most exciting technologies to move us to a more interactive and resourceful future.

This page acts as a continually updated reference that can enable education for anyone who is looking to design, learn or get started with tools for AR and VR.

0_KNyyn2jHeWmstlJT.png
skylight_r6_mr_hero01.png
vr.jpg

How many realities do we have now?

At the moment I could separate them into four different ones:

  • VR — virtual reality

  • AV — augmented virtuality

  • AR — augmented reality

  • MR — mixed reality

They all different but basically about the same.

How to get started:

  1. Research and learn

  2. Decide what experience you want to create

  3. Illustrate the user flows

  4. Sketch various UI and HUD options

  5. Determine tooling

  6. Select frameworks

Below are various tools that can help jumpstart your AR and VR creation process.


Virtual reality (VR) is an interactive computer-generated experience taking place within a simulated environment. It incorporates mainly auditory and visual feedback, but may also allow other types of sensory feedback like haptic. This immersive environment can be similar to the real world or it can be fantastical. Augmented reality systems may also be considered a form of VR that layers virtual information over a live camera feed into a headset or through a smartphone or tablet device giving the user the ability to view three-dimensional images.

Augmented reality (AR) is an interactive experience of a real-world environment where the objects that reside in the real-world are "augmented" by computer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory, and olfactory.The overlaid sensory information can be constructive (i.e. additive to the natural environment) or destructive (i.e. masking of the natural environment) and is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment. In this way, augmented reality alters one's ongoing perception of a real-world environment, whereas virtual reality completely replaces the user's real-world environment with a simulated one.

 

VR Tools

AR Tools

VR Frameworks

There are three major frameworks to choose from when designing for VR: Mozilla A-Frame, Daydream VR, and Unity VR/Unreal SDK.

AR Frameworks

Below are a few tools that can help designers or developers get jump started into the AR space.

 
 

AR Resources

Designing for Augmented Reality
https://blog.prototypr.io/designing-for-ar-b276c8251c20

A Quick Guide to Designing for Augmented Reality on Mobile (Part 1)
https://medium.com/@goatsandbacon/a-quick-guide-to-designing-for-augmented-reality-on-mobile-part-1-c8ecaaf303d5

Unleash Your Design Skills in AR Using Torch, Sketch, and InVision
https://www.torch.app/blog/an-end-to-end-ar-prototyping-workflow-using-torch-sketch-and-invision

A Quick Guide to Designing for Augmented Reality on Mobile (Part 1)
https://medium.com/@goatsandbacon/a-quick-guide-to-designing-for-augmented-reality-on-mobile-part-1-c8ecaaf303d5

Unleash Your Design Skills in AR Using Torch, Sketch, and InVision
https://www.torch.app/blog/an-end-to-end-ar-prototyping-workflow-using-torch-sketch-and-invision

Augmented Reality in 10 Lines of HTML
https://medium.com/arjs/augmented-reality-in-10-lines-of-html-4e193ea9fdbf

The Gigantic List of Augmented Reality Use Cases
https://uploadvr.com/augmented-reality-use-cases-list/

 

VR Approach

  • “Small” approach: The easiest, least expensive VR experience requires the quick additional step of rendering a spherical panorama from within our BIM software. This panorama then can be displayed in a VR headset or on a smartphone—via an app like HOK VR—to give users a static view of being inside a rendering. Though the user cannot navigate through this static model, they can get a 360-degree view of the space. (View example.)

  • “Medium” approach: This approach takes a 3D-design model and exports it to powerful gaming-engine software. When coupled with a VR headset and joystick, such as the HTC Vive or Oculus Rift, this “medium-size” approach gives designers and clients the ability to navigate their own way through the model. The design team can also create guided steps through the model to aid in navigation.

  • “Large” approach: This method allows users to not only move throughout the model but also interact with it. They can use a virtual laser pointer or joystick to highlight design details and make annotations about design changes or edits.

  • Group tour: In addition to the small, medium, and large approaches described above, we also have the ability to generate what is essentially a virtual group tour of the proposed design. Using VR hosting platforms such as InsiteVR, multiple project stakeholders can don head-mounted displays and navigate as a group or independently through the model with an HOK designer as the guide.

 

AR Approach

  • These underscore most of the AR use cases in the list, so it’s helpful to understand them:

  • Sentiment analysis: scan a person, or a group of people, and run apps to analyze body language, micro-expressions, language, and behavior. Get real-time feedback on how that person or group appears to be feeling or reacting, and adjust accordingly.

  • Facial recognition: scan a face and match it to an existing identity database to learn a person’s name and background information just by looking at him or her.

  • Object identification: use computer vision to detect and identify objects and track their physical location. This includes tracking the AR user’s location in relation to objects.

  • Information augmentation and display: once an object or person is identified, automatically search for information about it and display it to the AR user.

  • Porting from mobile phones to AR headsets: generally, AR moves a user’s attention from downward-facing on a phone to forward-facing in a headset. Almost anything that can be done on a phone (or any display device, really) will be done in AR.

  • Processing, sensing, and scanning: AR devices will contain their own processing units. To start, they’ll be big, clunky, and may be housed externally and connected to the headset, but they’ll get much smaller until they’re an indistinguishable part of the AR glasses. They’ll also be able to track wearers’ movements and locations and take rapid 3D scans of wearers and environments to project them elsewhere.

https://uploadvr.com/augmented-reality-use-cases-list/

6 quick (and key) AR concepts

These underscore most of the AR use cases in the list, so it’s helpful to understand them:

  1. Sentiment analysis: scan a person, or a group of people, and run apps to analyze body language, micro-expressions, language, and behavior. Get real-time feedback on how that person or group appears to be feeling or reacting, and adjust accordingly.

  2. Facial recognition: scan a face and match it to an existing identity database to learn a person’s name and background information just by looking at him or her.

  3. Object identification: use computer vision to detect and identify objects and track their physical location. This includes tracking the AR user’s location in relation to objects.

  4. Information augmentation and display: once an object or person is identified, automatically search for information about it and display it to the AR user.

  5. Porting from mobile phones to AR headsets: generally, AR moves a user’s attention from downward-facing on a phone to forward-facing in a headset. Almost anything that can be done on a phone (or any display device, really) will be done in AR.

  6. Processing, sensing, and scanning: AR devices will contain their own processing units. To start, they’ll be big, clunky, and may be housed externally and connected to the headset, but they’ll get much smaller until they’re an indistinguishable part of the AR glasses. They’ll also be able to track wearers’ movements and locations and take rapid 3D scans of wearers and environments to project them elsewhere.


Post > AR Advertising Inspiration

As devices continue to improve technology is enabling AR and VR to become more consumable. Over the past few years I have experimented with AR and VR possibilities and the potential is huge especially in the advertising space. In this post I wanted to inspire a few studies on how these technologies could be applied.

https://davidbanthony.com/creative-notes/2019/1/31/ar-advertising

Post > Oculus Dash

Our not to distant future of UI interaction envisioned through the Facebook Oculus Dash. Dash is your new command center for Rift — multitask across your favorite PC desktop apps, VR library, and more with a wave of your hand using an intuitive, built-for-Touch interface.

https://davidbanthony.com/creative-notes/2017/10/14/oculus-dash

 

Research Questions Are Not Interview Questions

As teams focus on research, testing and prototyping ensuring the types of questions and words you are using is correct. I’ve been guilty of running interviews and realizing at a later time that the questions I was asking were not organized in a way I could get the response I was looking for.

Read More

Reflections from a designer turned product manager: 6 unexpected differences

I'm sharing this article as a good reference for the designers who do decide to make the transition into product management. But also to encourage designers to reach out for mentorship as well as always be vocal about your work which in turn enable trust to open a seat at the table with the projects you work on.

Read More

How to run a research session if you’re not a design researcher

When I began designing almost 20+ years ago, customers would come to me and say they need a website. I would follow up the request by asking questions to understand better the customer's needs such as business who, what, why, when, etc. At the time I just wanted to understand the need so I could better design for there vision and the end user.

Read More

Mental models in UX Design in examples

I wanted to share this article for designers, product managers and anyone in the software development cycle. The importance of taking into account what user expectations are - vs - what you think they will understand is enormous. This mistake in UX is becoming commonplace as UX designers become more available to creating experiences we interact with on a daily basis.

Read More

Voice User Interfaces (VUI)  —  The Ultimate Designer’s Guide

VUI known as Voice User Interfaces. A topic even the general public is quite aware of thanks to the marketing powers of the Amazon Alexa. In the US, Google Assistant, Cortana, Siri, Alexa, have all been interacted with at some point due to the integrations we see on Mobile, Home, and Cars.

Read More

Changing the Conversation about Product Management vs. UX

This is an interesting article for those who have worked in the UX field inside startups and agencies. I’ve experienced the overlap of Product Manager and User Experience designer many times and often questioned what exactly are the key components that separate our roles inside of a company. In most cases, the roles will vary by the type and size of the company.

Read More

Drop-Down Usability

A great little write up detailing how to focus in on designing usable drop downs correctly. I’ve experienced this many times when it comes to building out CMS applications ( and other applications ) that give customers the power to create the experiences for there needs. Without guidelines or rules, these dropdowns can turn into unusable interfaces leading to frustrating experiences.

Read More

The business value of design

I’ve worked as a designer for close to 20 years and I am still amazed that some businesses don’t understand the value of design. It’s possible the word design in itself is the problem and the capabilities and strategic values get lost in trying to understand what designers are. In any case here are a few interesting snippets from a recent article posted by Mckinsey.

Read More

How to remote user test

When your user base consists of millions of people from pretty much every country, there are many things to consider while conducting user tests. User behaviors and mindsets change drastically from city to city and country to country. This creates problems if you conduct user tests the old fashioned way, aka in person. That’s why most of the usability tests that we do at JotForm are remote user tests. We find it very effective and receive great results from them.

Read More