Arbitrary Optics for Gaussian Splatting Using Space Warping
Author Information
Author(s): Jakob Nazarenus, Simin Kou, Fang-Lue Zhang, Reinhard Koch
Primary Institution: Department of Computer Science, Kiel University
Hypothesis
Can we extend 3D Gaussian splatting to work with arbitrary camera optics instead of just pinhole cameras?
Conclusion
The proposed method successfully enables accurate scene reconstruction from highly distorted images using arbitrary camera optics.
Supporting Evidence
- The method reduces the number of Gaussians needed for reconstruction, enhancing space efficiency.
- It processes fisheye images directly, eliminating the need for additional preprocessing steps.
- Extensive ablations supported the design decisions, such as the learned skybox and Jacobian distortion.
Takeaway
This study shows how to make 3D images look good even when taken with weird camera lenses, like fisheye lenses, by using a special technique.
Methodology
The study introduces a space-warping module that allows the Gaussian rasterizer to render views for arbitrary optics, including fisheye lenses.
Limitations
The method may still struggle with highly reflective surfaces and reconstruction artifacts in shaded areas.
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website