Niantic, the company behind the extremely popular augmented reality mobile games Pokémon Go and Ingress, announced that it is using data collected by its millions of players to create an AI model that can navigate the physical world.
In a blog post published last week, first spotted by Garbage Day, Niantic says it is building a “Large Geospatial Model.” This name, the company explains, is a direct reference to Large Language Models (LLMs) Like OpenAI’s GPT, which are trained on vast quantities of text scraped from the internet in order to process and produce natural language. Niantic explains that a Large Geospatial Model, or LGM, aims to do the same for the physical world, a technology it says “will enable computers not only to perceive and understand physical spaces, but also to interact with them in new ways, forming a critical component of AR glasses and fields beyond, including robotics, content creation and autonomous systems. As we move from phones to wearable technology linked to the real world, spatial intelligence will become the world’s future operating system.”
By training an AI model on millions of geolocated images from around the world, the model will be able to predict its immediate environment in the same way an LLM is able to produce coherent and convincing sentences by statistically determining what word is likely to follow another.
“Large Geospatial Models will help computers perceive, comprehend, and navigate the physical world in a way that will seem equally advanced,” Niantic said.
I found the following explanation in Niantic’s blog of how the LGM works to be the clearest:
Niantic’s LGM builds upon its Lightship Visual Positioning System (VPS), which allows players to pin virtual items to physical locations in the world with “centimeter-level accuracy.” For example, Niantic recently introduced an experimental feature in Pokémon Go called Pokémon Playgrounds, where the user can place Pokémon at a specific location that will remain there for others to see and interact with. This feature, Niantic explains, is powered by massive amounts of data, and is unique because it is taken from a pedestrian perspective from locations inaccessible to cars.
“Today we have 10 million scanned locations around the world, and over 1 million of those are activated and available for use with our VPS service,” Niantic said. “We receive about 1 million fresh scans each week, each containing hundreds of discrete images.” This data, Niantic’s blog explains, is collected from its games and Scaniverse, Niantic’s app for 3D scanning objects and locations.
The AI gold rush of the last few years prompted a frenzied hunt for large datasets that can train generative AI models. We’ve seen companies scrape text from the internet, YouTube subtitles, YouTube videos, books, and more, with little consideration for the humans who created this data. In this case also, players of the incredibly viral Pokémon Go had no way of knowing that when they downloaded the game in 2016 that it would one day fuel this type of AI product.
It should come as no surprise that Niantic would now try to leverage its data in the AI space. As the company says, data from Google Street View and various self-driving companies means that there’s quite a bit of data from roads, but Niantic’s games have created a huge dataset of where only pedestrians can go. At the moment, the company says the data can be useful in a few ways, like other augmented reality products, but the way this data might help robots navigate the world is the most interesting since robots that navigate the real world do anything from deliver food to carry automatic rifles.
Niantic did not respond to a request to comment about whether it had any limitations on who and how it would allow people to leverage this data.
