Pokemon Go & Augmented Reality – Now is the Time for AR

Presenter: Dana Hu - AR Engineering at Niantic Labs

In this talk on augmented reality Dana Hu, AR Engineer at Niantic Labs dispelled some myths, while exploring the challenges, and the progress being made in this cutting edge field.

The "Where's my jet pack moment" comes from the sci fi novel Rainbow's End, by Vernor Vinge. It refers to a person waking up in a future that does not have the expected robots and automation, but rather is filled with software and virtual reality.

She started off talking about the idea that the future blueprint of Augmented Reality may be Science Fiction. She noted that many novels envision dystopic worlds where people are segregated into different realities based on the lenses implanted in their eyes and other horrible Black Mirror style futures. Other fictional perspectives are like Ready Player One. However, she dismissed this saying, "Reality is a lot more complex then sci fi," and posited that fiction is not the roadmap for development.

A Little history of AR
1968 Sword of Damacles: The First AR, invented by Ivan Sutherland. This was a head mounted display system that was perceived as being somewhat scary and alien.
1998 NASA-X38: This was the first system that implemented an integrated augmented reality system. This was an extremely high level, multi million dollar system.
Present: There are many VR Headsets available on the market. However they are still quite expensive and beyond the means of the average consumer. They also have many issues like tiny view displays, and a small periphery view.

These companies are often valued quite high, with Magic Leap being a particular example of this. Despite their immense valuation they have been working for several years and have only just released their first product.

Pokemon Go
This was a global AR phenomenon in 2016 that broke multiple records for downloads and active users. It was an important way to popularize the technology because the barrier to entry was very low. It didn't require an expensive system or headsets. It was also a simple game that could be enjoyed by everyone from children to grandparents.

It made people aware of the possibilities.

This led to 2017's release of the AR Core by Google and the AR Kit by Apple. The limitation of those systems is that they could only be implemented on a single phone.

Escher Reality: this is the next step, that moves beyond implementation on a single phone to allow multiple devices to share the same reality. The demo included a video of two people playing ping pong on two separate devices.

It is possible that AR will be the next platform after desktop and laptop.

Solving the Challenges in Augmented Reality
1. You need to understand the world: Computers are good at sequential things that require exact answers. However, the real world is fuzzy. Programs are not good at distinguishing between different objects and context. For instance, it is very easy for a human to look at a picture and know someone is looking at their phone, even if it is very small. Computers are terrible at that.
"When I think about the fear of AI taking over, since working in this field, I feel it's more likely we'll go to Mars."

2. Needs to run in real-time: SLAM simultaneous localization and mapping. A real time and evolving map needs to be created and understood. This is particular important in the robotics field, as they need to be able to understand the world in order to navigate the world.

3. Need for visuals, field of graphics: The real world has many light sources and lighting is something that has to blend in a very complex way. Computing all of the different light sources in reality is very difficult. 3D designers are able to do it, but only because they have complete control over the lighting. This is the reason many AR objects look very unreal when displayed against reality.

4. Need to create: This involves developing 3D systems for navigation, and is something that will be pioneered by the gaming industry, which is already forced to think in three dimensions.

5. See through cameras: There is a limit to the ability to create large virtual screens with a wide range of periphery. This is partially a computational problem, but also an inherent physics issue that requires the pioneering of new science to produce more effective headsets.

6. Need to consider the human factors: Right now the applications being developed are very rudimentary. This is similar to the way things were  in the early days of smart phones, where you found hundreds of different flashlight apps.

All of this needs to be tied together in a coherent system in order to create viable AR systems. That is not an easy task. But it is possible.

"It's early and a great time to take a seat at the table in engineering, science, games, products, design and UX." - Dana Hu