Researchers at Meta Superintelligence Labs, with leadership from Georgia Gkioxari of Caltech, have unveiled SAM 3D, an open-source AI model that can reconstruct any object in three dimensions from a single 2D image, tells this article from Caltech.
SAM 3D builds on the earlier 2D segmentation model, Segment Anything Model (SAM). In its two variants, “SAM 3D Objects” and “SAM 3D Body,” it can recover full geometry, texture, and spatial position of objects or human figures captured in ordinary photos.
That’s more than a neat trick; it helps machines understand our three-dimensional world using only flat images. Gkioxari explains that this matters for robotics, augmented reality, digital content creation, or any application that needs real-world spatial reasoning and interaction.
Training such a model posed a challenge: 3D ground-truth data are rare and expensive to create. The SAM 3D team overcame this by deploying a “model-in-the-loop” annotation strategy. Human reviewers simply pick the best 3D reconstruction proposals generated by the model instead of manually sculpting models themselves. Over repeated rounds, the system improves, enabling scalable learning without deep 3D expertise.
Meta released SAM 3D openly, free to use. Alongside code, a demo lets anyone upload an image and see objects rendered in 3D space. Already, Meta has begun experimenting with using SAM 3D for 3D previews on its marketplace and for robotics demonstrations where machines interact with reconstructed environments.
SAM 3D marks a big step toward giving machines the ability to “see” the world in three dimensions from ordinary photos. By making 3D understanding scalable and accessible, it may pave the way for richer robotics, AR/VR, simulation, and design applications.