Source: venturebeat.com
Facebook AI Research (FAIR) today unveiled PyTorch3D, a library that enables researchers and developers to combine deep learning and 3D objects.
As part of the release, Facebook is also open-sourcing Mesh R-CNN, a model introduced last year capable of rendering 3D objects from 2D shapes in images of interior spaces. PyTorch3D was inspired by Mesh R-CNN and recent 3D work by Facebook AI Research, FAIR engineer Nikhila Ravi said.
Working in 3D is important for rendering 3D objects or scenes that appear in mixed reality or virtual reality. It can also be used to tackle AI challenges like robotic grasping or helping autonomous vehicles understand the position of nearby objects.
PyTorch3D comes with frequently used 3D operators and loss functions for 3D data and a differentiable mesh renderer for creating 3D objects. PyTorch3D also has a differentiable rendering API, some CUDA support, and heterogeneous batching capabilities unavailable in any existing 3D library, Ravi told VentureBeat in a phone interview.
“With PyTorch3D, researchers can input all these functions and use them with the existing deep learning system in PyTorch and it greatly reduces on the time to work on 3D planning research, which requires a lot of expertise in order to get started, and we want to try and reduce that ramp-up time,” she said.
yTorch3D uses meshes, a data format for interoperability of vertices and faces that make up 3D objects, and can use a patch tensor to collapse all vertices for meshes in a batch into a single tensor involved with batching, a common process for deep learning research.
The premiere of PyTorch3D follows the launch of the PyRobot robotics framework last year, and FAIR 3D research that extracts characters from real-world videos.