3DGUT

3DGUT replaces traditional EWA splatting with an Unscented Transform to support nonlinear camera models and secondary effects like reflections, enabling flexible, distortion-aware rendering while retaining the speed of rasterization.

Web: https://research.nvidia.com/labs/toronto-ai/3DGUT/
Paper: 3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting
Authors: Qi Wu, Janick Martinez Esturo, Ashkan Mirzaei, Nicolas Moenne-Loccoz, Zan Gojcic

Mip-NeRF 360

Mip-NeRF 360 is a collection of four indoor and five outdoor object-centric scenes. The camera trajectory is an orbit around the object with fixed elevation and radius. The test set takes each n-th frame of the trajectory as test views.

Scene PSNR SSIM LPIPS (VGG) Time GPU mem.
garden 27.08
Paper's PSNR: 26.9
0.858
Paper's SSIM: 0.851
0.136 32m 38s 13.99 GB
bicycle 24.90
Paper's PSNR: 24.21
0.759
Paper's SSIM: 0.741
0.236 33m 59s 15.47 GB
flowers 21.37
Paper's PSNR: 21.48
0.615
Paper's SSIM: 0.612
0.343 35m 13s 12.42 GB
treehill 22.40
Paper's PSNR: 22.15
0.632
Paper's SSIM: 0.623
0.358 36m 59s 15.51 GB
stump 26.50
Paper's PSNR: 26.51
0.774
Paper's SSIM: 0.768
0.248 28m 35s 13.13 GB
kitchen 30.21
Paper's PSNR: 31.23
0.922
Paper's SSIM: 0.926
0.159 44m 5s 16.10 GB
bonsai 31.62
Paper's PSNR: 32.17
0.935
Paper's SSIM: 0.941
0.252 31m 30s 16.02 GB
counter 28.70
Paper's PSNR: 29.03
0.904
Paper's SSIM: 0.908
0.252 33m 39s 14.31 GB
room 30.59
Paper's PSNR: 31.64
0.913
Paper's SSIM: 0.919
0.284 34m 31s 17.32 GB
Average 27.04
Paper's PSNR: 27.26
0.812
Paper's SSIM: 0.810
0.252 34m 34s 14.92 GB

Blender

Blender (nerf-synthetic) is a synthetic dataset used to benchmark NeRF methods. It consists of 8 scenes of an object placed on a white background. Cameras are placed on a semi-sphere around the object. Scenes are licensed under various CC licenses.

Scene PSNR SSIM LPIPS (VGG) Time GPU mem.
lego 36.12
Paper's PSNR: 36.47

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.983
Paper's SSIM: 0.984

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.018 7m 15s 4.62 GB
drums 25.98
Paper's PSNR: 25.99

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.952
Paper's SSIM: 0.953

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.047 8m 17s 4.79 GB
ficus 35.86
Paper's PSNR: 36.43

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.988
Paper's SSIM: 0.988

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.012 6m 17s 4.62 GB
hotdog 37.69
Paper's PSNR: 38.11

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.985
Paper's SSIM: 0.986

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.026 5m 47s 4.46 GB
materials 30.10
Paper's PSNR: 30.39

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.960
Paper's SSIM: 0.96

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.043 7m 51s 4.76 GB
mic 35.37
Paper's PSNR: 36.32

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.991
Paper's SSIM: 0.992

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.008 7m 30s 4.61 GB
ship 30.90
Paper's PSNR: 31.72

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.905
Paper's SSIM: 0.908

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.123 7m 55s 4.71 GB
chair 35.79
Paper's PSNR: 35.61

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.987
Paper's SSIM: 0.988

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.014 7m 17s 4.69 GB
Average 33.48
Paper's PSNR: 33.88

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.969
Paper's SSIM: 0.970

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.036 7m 16s 4.66 GB