Methods¶
2D Gaussian Splatting¶
2D Gaussian Splatting for Geometrically Accurate Radiance Fields
- Authors:
- Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, Shenghua Gao
- Paper:
- Web:
- Licenses:
- ID:
- 2d-gaussian-splatting
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,points3D_xyz- Supported outputs:
color,depth,accumulation,normal
2DGS adopts 2D oriented disks as surface elements and allows high-quality rendering with Gaussian Splatting. In NerfBaselines, we fixed bug with cx,cy, added appearance embedding optimization, and added support for sampling masks.
3DGS-MCMC¶
3D Gaussian Splatting as Markov Chain Monte Carlo
- Authors:
- Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Yang-Che Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi
- Paper:
- Web:
- Licenses:
- ID:
- 3dgs-mcmc
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,points3D_xyz- Supported outputs:
color,depth,accumulation,normal
3DGS-MCMC reinterprets 3D Gaussian Splatting as MCMC sampling, introducing noise-based updates and removing heuristic cloning strategies, leading to improved rendering quality, efficient Gaussian use, and robustness to initialization. In NerfBaselines, we fixed bug with cx,cy, added appearance embedding optimization, and added support for sampling masks and web demos.
CamP¶
CamP: Camera Preconditioning for Neural Radiance Fields
- Authors:
- Keunhong Park, Philipp Henzler, Ben Mildenhall, Jonathan T. Barron, Ricardo Martin-Brualla
- Paper:
- Web:
- Licenses:
- ID:
- camp
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,opencv_fisheye- Required features:
color- Supported outputs:
color,depth,accumulation
CamP is an extension of Zip-NeRF which adds pose refinement to the training process.
COLMAP¶
Pixelwise View Selection for Unstructured Multi-View Stereo
- Authors:
- Johannes Lutz Schőnberger, Enliang Zheng, Marc Pollefeys, Jan-Michael Frahm
- Paper:
- Web:
- Licenses:
- ID:
- colmap
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,full_opencv,opencv_fisheye- Required features:
color,images_points3D_indices,points3D_xyz,points3D_rgb- Supported outputs:
color,depth
COLMAP Multi-View Stereo (MVS) is a general-purpose, end-to-end image-based 3D reconstruction pipeline. It uses the point cloud if available, otherwise it runs a sparse reconstruction to obtained. The reconstruction consists of a stereo matching step, followed by a multi-view stereo step to obtain a dense point cloud. Finally, either Delaunay or Poisson meshing is used to obtain a mesh from the point cloud.
Gaussian Opacity Fields¶
Gaussian Opacity Fields: Efficient and Compact Surface Reconstruction in Unbounded Scenes
- Authors:
- Zehao Yu, Torsten Sattler, Andreas Geiger
- Paper:
- Web:
- Licenses:
- ID:
- gaussian-opacity-fields
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,points3D_xyz- Supported outputs:
color,normal,depth,accumulation,distortion_map
Improved Mip-Splatting with better geometry.
Gaussian Splatting¶
3D Gaussian Splatting for Real-Time Radiance Field Rendering
- Authors:
- Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis
- Paper:
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/3d_gaussian_splatting_low.pdf
- Web:
- Licenses:
- ID:
- gaussian-splatting
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,points3D_xyz- Supported outputs:
color
Official Gaussian Splatting implementation extended to support distorted camera models. It is fast to train (1 hous) and render (200 FPS).
GS-W¶
Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections
- Authors:
- Dongbin Zhang, Chuming Wang, Weitao Wang, Peihao Li, Minghan Qin, Haoqian Wang
- Paper:
- Web:
- Licenses:
unknown
- ID:
- gaussian-splatting-wild
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,points3D_xyz- Supported outputs:
color
Official GS-W implementation - 3DGS modified to handle appearance changes and transient objects. A reference view used to provide appearance conditioning. Note, that the method uses huge appearance embeddings (per-Gaussian) and appearance modeling has a large memory footprint.
gsplat¶
gsplat: An Open-Source Library for Gaussian Splatting
- Authors:
- Vickie Ye, Ruilong Li, Justin Kerr, Matias Turkulainen, Brent Yi, Zhuoyang Pan, Otto Seiskari, Jianbo Ye, Jeffrey Hu, Matthew Tancik, Angjoo Kanazawa
- Paper:
- Web:
- Licenses:
- ID:
- gsplat
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,images_points3D_indices,points3D_xyz,points3D_rgb- Supported outputs:
color,depth,accumulation
gsplat is an open-source library for CUDA accelerated rasterization of gaussians with python bindings. It is inspired by the 3DGS paper, but it is faster, more memory efficient, and with a growing list of new features. In NerfBaselines, the method was modified to enable appearance optimization, to support sampling masks, and to support setting background color (which is required for the Blender dataset).
Instant NGP¶
Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
- Authors:
- Thomas Müller, Alex Evans, Christoph Schied, Alexander Keller
- Paper:
https://nvlabs.github.io/instant-ngp/assets/mueller2022instant.pdf
- Web:
- Licenses:
- ID:
- instant-ngp
- Backends:
- docker, conda, apptainer, python
- Camera models:
opencv,pinhole,opencv_fisheye- Required features:
color- Supported outputs:
color,accumulation
Instant-NGP is a method that uses hash-grid and a shallow MLP to accelerate training and rendering. This method trains very fast (~6 min) and renders also fast ~3 FPS.
K-Planes¶
K-Planes: Explicit Radiance Fields in Space, Time, and Appearance
- Authors:
- Sara Fridovich-Keil, Giacomo Meanti, Frederik Warburg, Benjamin Recht, Angjoo Kanazawa
- Paper:
- Web:
- Licenses:
- ID:
- kplanes
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,images_points3D_indices,points3D_xyz- Supported outputs:
color,depth
K-Planes is a NeRF-based method representing d-dimensional space using 2 planes allowing for a seamless way to go from static (d=3) to dynamic (d=4) scenes.
Mip-Splatting¶
Mip-Splatting: Alias-free 3D Gaussian Splatting
- Authors:
- Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, Andreas Geiger
- Paper:
- Web:
- Licenses:
- ID:
- mip-splatting
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,points3D_xyz- Supported outputs:
color
A modification of Gaussian Splatting designed to better handle aliasing artifacts.
Mip-NeRF 360¶
Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
- Authors:
- Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, Peter Hedman
- Paper:
- Web:
- Licenses:
- ID:
- mipnerf360
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,opencv_fisheye- Required features:
color- Supported outputs:
color,depth,accumulation
Official Mip-NeRF 360 implementation addapted to handle different camera distortion/intrinsic parameters. It was designed for unbounded object-centric 360-degree capture and handles anti-aliasing well. It is, however slower to train and render compared to other approaches.
NeRF¶
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
- Authors:
- Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng
- Paper:
- Web:
- Licenses:
- ID:
- nerf
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,full_opencv,opencv_fisheye- Required features:
color- Supported outputs:
color,depth,accumulation
Original NeRF method representing radiance field using a large MLP.
NerfStudio¶
Nerfstudio: A Modular Framework for Neural Radiance Field Development
- Authors:
- Matthew Tancik, Ethan Weber, Evonne Ng, Ruilong Li, Brent Yi, Justin Kerr, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, David McAllister, Angjoo Kanazawa
- Paper:
- Web:
- Licenses:
- ID:
- nerfacto
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,opencv_fisheye- Required features:
color- Supported outputs:
color,depth,accumulation
NerfStudio (Nerfacto) is a method based on Instant-NGP which combines several improvements from different papers to achieve good quality on real-world scenes captured under normal conditions. It is fast to train (12 min) and render speed is ~1 FPS.
NeRF On-the-go¶
NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild
- Authors:
- Weining Ren, Zihan Zhu, Boyang Sun, Julia Chen, Marc Pollefeys, Songyou Peng
- Paper:
- Web:
- ID:
- nerfonthego
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,opencv_fisheye- Required features:
color- Supported outputs:
color,depth,accumulation
NeRF On-the-go enables novel view synthesis in in-the-wild scenes from casually captured images.
NeRF-W (reimplementation)¶
- Licenses:
- ID:
- nerfw-reimpl
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,full_opencv,opencv_fisheye- Required features:
color,points3D_xyz- Supported outputs:
color,depth
Unofficial reimplementation of NeRF-W. Does not reach the performance reported in the original paper, but is widely used for benchmarking.
Scaffold-GS¶
Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering
- Authors:
- Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua LinBo Dai
- Paper:
- Web:
- Licenses:
- ID:
- scaffold-gs
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,points3D_xyz- Supported outputs:
color
Scaffold-GS uses anchor points to distribute local 3D Gaussians, and predicts their attributes on-the-fly based on viewing direction and distance within the view frustum. In NerfBaselines, we fixed bug with cx,cy, added appearance embedding optimization, and added support for sampling masks. Note, that we also implement a demo for the method, but it does not evaluate the MLP and the Gaussians are “baked” for specific viewing direction and appearance embedding (if enabled).
SeaThru-NeRF¶
SeaThru-NeRF: Neural Radiance Fields in Scattering Media
- Authors:
- Deborah Levy, Amit Peleg, Naama Pearl, Dan Rosenbaum, Derya Akkaynak, Tali Treibitz, Simon Korman
- Paper:
- Web:
- Licenses:
- ID:
- seathru-nerf
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,opencv_fisheye- Required features:
color- Supported outputs:
color,depth,accumulation,depth_mean,color_clean,color_backscatter
Official SeaThru-NeRF implementation. It is based on MipNeRF 360 and was disagned for underwater scenes.
TensoRF¶
TensoRF: Tensorial Radiance Fields
- Authors:
- Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, Hao Su
- Paper:
- Web:
- Licenses:
- ID:
- tensorf
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,full_opencv,opencv_fisheye- Required features:
color- Supported outputs:
color,depth
TensoRF factorizes the radiance field into a multiple compact low-rank tensor components. It was designed and tester primarily on Blender, LLFF, and NSVF datasets.
Tetra-NeRF¶
Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedra
- Authors:
- Jonas Kulhanek, Torsten Sattler
- Paper:
- Web:
- Licenses:
- ID:
- tetra-nerf
- Backends:
- docker, apptainer, python
- Camera models:
opencv,pinhole,opencv_fisheye- Required features:
color,points3D_xyz,points3D_rgb- Supported outputs:
color,depth,accumulation
Tetra-NeRF is a method that represents the scene as tetrahedral mesh obtained using Delaunay tetrahedralization. The input point cloud has to be provided (for COLMAP datasets the point cloud is automatically extracted). This is the official implementation from the paper.
WildGaussians¶
WildGaussians: 3D Gaussian Splatting in the Wild
- Paper:
- Web:
- Licenses:
- ID:
- wild-gaussians
- Backends:
- conda, docker, apptainer, python
- Camera models:
pinhole- Required features:
color,points3D_xyz- Supported outputs:
color,accumulation,depth
WildGaussians adopts 3DGS to handle appearance changes and transient objects. After fixing appearance, it can be baked back into 3DGS.
Zip-NeRF¶
Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields
- Authors:
- Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, Peter Hedman
- Paper:
- Web:
- Licenses:
- ID:
- zipnerf
- Backends:
- conda, docker, apptainer, python
- Camera models:
opencv,pinhole,opencv_fisheye- Required features:
color- Supported outputs:
color,depth,accumulation
Zip-NeRF is a radiance field method which addresses the aliasing problem in the case of hash-grid based methods (iNGP-based). Instead of sampling along the ray it samples along a spiral path - approximating integration along the frustum.