Augmented reality devices are becoming increasingly capable of overlaying digital information onto the physical world, but most systems still operate reactively. They respond only after a user has already shifted their attention. New research from the Georgia Institute of Technology aims to change that by enabling AR devices to predict where users will look next in a three-dimensional environment, tells Tech Xplore.
The work, led by Ph.D. student Fiona Ryan in Georgia Tech’s School of Interactive Computing, focuses on forecasting a user’s gaze from a first-person perspective. Rather than simply tracking eye movements, the system analyzes how people interact with objects and move through their surroundings to anticipate future visual attention. This capability could allow smart glasses and other AR devices to prepare information in advance, reducing delays and creating a smoother user experience.
The research addresses a longstanding challenge in augmented reality. Existing systems often lag behind user actions because they must first detect where a person is looking before rendering relevant content. Ryan’s approach introduces a predictive layer, giving AR devices an opportunity to react before the user completes a movement or changes focus. By understanding both user behavior and environmental context, the technology helps bridge the gap between perception and response.
To develop the system, the researchers combined information about eye movements, body motion, object interactions, and the three-dimensional structure of a scene. This enables the software to build a more complete picture of a user’s intentions. In demonstrations, the system successfully predicted where a user would direct their attention after interacting with objects in the environment, such as approaching a table, handling a cup, and then turning toward another location.
Beyond improving responsiveness, the technology could help AR systems gain a deeper understanding of real-world environments and user intent. Potential applications include navigation assistance, workplace training, accessibility tools, and context-aware information delivery. By anticipating attention rather than merely reacting to it, future AR devices may become more intuitive and effective.
The research marks an important step toward AR systems that can proactively support users. As predictive models become more sophisticated, augmented reality could evolve into a technology that understands not only what users are seeing, but also what they are likely to do next.