The National Science Foundation (NSF) recently awarded Assistant Professor of Computer Science Shuang Zhao $400,000 over three years for his grant, “Physics and Learning Integration Using Differentiable Rendering.”

“This project advocates combining the complementary advantages of physics-based simulation and machine learning to create better and faster physical acquisition pipelines for a large variety of applications,” says Zhao.

As explained in the grant abstract, image measurements are often used in science and engineering to understand and acquire properties of the physical world (for example, the shape or the reflectance of a surface). This is a critical capability in areas such as biomedicine, robotics and computer vision. Inferring unknown parameters from these image measurements typically requires using one of two types of inference algorithms: physics-based algorithms, which are generally accurate but require a lot of computation, or machine-learning-based algorithms, which are computationally efficient but not always accurate.

The goal of this project is to create general-purpose computational tools that are both efficient and accurate by combining the complementary advantages of physics-based and machine-learning-based techniques. Zhao is taking a three-step approach for this work, with the first step being to “create a new class of physically accurate simulators, specifically designed to be compatible with the algorithms used to learn data-driven algorithms.” Next, he and his team will “develop computational inference algorithms that synergistically combine machine learning models with physics-based simulators.”

The final step will be to integrate physics-based simulation and machine learning and to demonstrate its effectiveness in a variety of applications in autonomous sensing and navigation, material science and fabrication, and biomedical imaging.

— Shani Murray