Zip-NeRF

Zip-NeRF is a radiance field method which addresses the aliasing problem in the case of hash-grid based methods (iNGP-based). Instead of sampling along the ray it samples along a spiral path - approximating integration along the frustum.

Web: https://jonbarron.info/zipnerf/
Paper: Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields
Authors: Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, Peter Hedman

Mip-NeRF 360

Mip-NeRF 360 is a collection of four indoor and five outdoor object-centric scenes. The camera trajectory is an orbit around the object with fixed elevation and radius. The test set takes each n-th frame of the trajectory as test views.

Scene PSNR SSIM LPIPS (VGG) Time GPU mem.
garden 28.19
Paper's PSNR: 28.2
0.864
Paper's SSIM: 0.86
0.127
Paper's LPIPS (VGG): 0.118
5h 19m 14s 26.77 GB
bicycle 25.89
Paper's PSNR: 25.8
0.775
Paper's SSIM: 0.769
0.226
Paper's LPIPS (VGG): 0.208
5h 22m 47s 26.74 GB
flowers 22.33
Paper's PSNR: 22.4
0.637
Paper's SSIM: 0.642
0.310
Paper's LPIPS (VGG): 0.273
5h 16m 23s 26.75 GB
treehill 24.01
Paper's PSNR: 23.89
0.677
Paper's SSIM: 0.681
0.279
Paper's LPIPS (VGG): 0.242
5h 48m 6s 26.75 GB
stump 27.41
Paper's PSNR: 27.55
0.792
Paper's SSIM: 0.8
0.233
Paper's LPIPS (VGG): 0.193
5h 25m 16s 26.74 GB
kitchen 32.35
Paper's PSNR: 32.5
0.929
Paper's SSIM: 0.928
0.134
Paper's LPIPS (VGG): 0.116
5h 39m 34s 26.92 GB
bonsai 34.68
Paper's PSNR: 34.46
0.951
Paper's SSIM: 0.949
0.195
Paper's LPIPS (VGG): 0.173
5h 35m 52s 26.92 GB
counter 29.13
Paper's PSNR: 29.38
0.905
Paper's SSIM: 0.902
0.223
Paper's LPIPS (VGG): 0.185
5h 39m 53s 26.91 GB
room 32.99
Paper's PSNR: 32.65
0.928
Paper's SSIM: 0.925
0.237
Paper's LPIPS (VGG): 0.196
5h 25m 53s 26.91 GB
Average 28.55
Paper's PSNR: 28.54
0.829
Paper's SSIM: 0.828
0.218
Paper's LPIPS (VGG): 0.189
5h 30m 20s 26.82 GB

Blender

Blender (nerf-synthetic) is a synthetic dataset used to benchmark NeRF methods. It consists of 8 scenes of an object placed on a white background. Cameras are placed on a semi-sphere around the object. Scenes are licensed under various CC licenses.

Scene PSNR SSIM LPIPS (VGG) Time GPU mem.
lego 35.81
Paper's PSNR: 34.84
0.983
Paper's SSIM: 0.98
0.019
Paper's LPIPS (VGG): 0.019
5h 20m 18s 26.20 GB
drums 25.90
Paper's PSNR: 25.84
0.948
Paper's SSIM: 0.944
0.054
Paper's LPIPS (VGG): 0.05
5h 20m 22s 26.20 GB
ficus 34.73
Paper's PSNR: 33.9
0.987
Paper's SSIM: 0.985
0.014
Paper's LPIPS (VGG): 0.015
5h 25m 57s 26.20 GB
hotdog 37.98
Paper's PSNR: 37.14
0.987
Paper's SSIM: 0.984
0.023
Paper's LPIPS (VGG): 0.02
5h 19m 2s 26.20 GB
materials 30.98
Paper's PSNR: 31.66
0.967
Paper's SSIM: 0.969
0.040
Paper's LPIPS (VGG): 0.032
5h 23m 48s 26.20 GB
mic 35.90
Paper's PSNR: 35.15
0.992
Paper's SSIM: 0.991
0.008
Paper's LPIPS (VGG): 0.007
5h 26m 19s 26.20 GB
ship 32.30
Paper's PSNR: 31.38
0.937
Paper's SSIM: 0.929
0.114
Paper's LPIPS (VGG): 0.091
5h 14m 50s 26.20 GB
chair 35.75
Paper's PSNR: 34.84
0.987
Paper's SSIM: 0.983
0.017
Paper's LPIPS (VGG): 0.017
5h 24m 56s 26.20 GB
Average 33.67
Paper's PSNR: 33.09
0.973
Paper's SSIM: 0.971
0.036
Paper's LPIPS (VGG): 0.031
5h 21m 57s 26.20 GB

Tanks and Temples

Tanks and Temples is a benchmark for image-based 3D reconstruction. The benchmark sequences were acquired outside the lab, in realistic conditions. Ground-truth data was captured using an industrial laser scanner. The benchmark includes both outdoor scenes and indoor environments. The dataset is split into three subsets: training, intermediate, and advanced.

Scene PSNR SSIM LPIPS Time GPU mem.
auditorium 24.52 0.877 0.153 5h 24m 16s 26.61 GB
ballroom 25.45 0.835 0.113 5h 25m 1s 26.61 GB
courtroom 22.17 0.790 0.153 5h 17m 54s 26.61 GB
museum 19.34 0.746 0.159 5h 24m 17s 26.61 GB
palace 19.11 0.718 0.317 6h 42m 11s 26.61 GB
temple 20.58 0.805 0.183 5h 37m 11s 26.61 GB
family 27.10 0.889 0.067 5h 32m 15s 26.61 GB
francis 29.10 0.915 0.106 5h 34m 11s 26.61 GB
horse 26.82 0.897 0.069 5h 55m 60s 26.61 GB
lighthouse 23.07 0.849 0.131 5h 52m 55s 26.63 GB
m60 29.01 0.912 0.076 6h 3m 19s 26.63 GB
panther 28.76 0.909 0.081 5h 36m 45s 26.63 GB
playground 27.13 0.880 0.095 5h 29m 4s 26.61 GB
train 22.19 0.814 0.119 6h 14m 13s 26.61 GB
barn 29.26 0.884 0.083 5h 28m 43s 26.61 GB
caterpillar 23.94 0.802 0.152 5h 53m 16s 26.61 GB
church 23.14 0.807 0.153 6h 4m 32s 26.61 GB
courthouse 22.88 0.780 0.218 6h 4m 6s 26.61 GB
ignatius 22.61 0.789 0.127 5h 48m 32s 26.61 GB
meetingroom 25.93 0.875 0.121 5h 21m 9s 26.61 GB
truck 25.09 0.864 0.081 5h 37m 11s 26.61 GB
Average 24.63 0.840 0.131 5h 44m 9s 26.61 GB

Mip-NeRF 360 Sparse

Modified Mip-NeRF 360 dataset with small train set (12 or 24) views. The dataset is used to evaluate sparse-view NVS methods.

Scene PSNR SSIM LPIPS (VGG) Time GPU mem.
garden n12 19.69 0.521 0.372 5h 22m 55s 29.28 GB
bicycle n12 14.23 0.183 0.644 5h 18m 11s 29.08 GB
flowers n12 12.11 0.133 0.692 5h 54m 33s 29.00 GB
treehill n12 15.73 0.307 0.554 5h 31m 58s 29.19 GB
stump n12 17.11 0.239 0.623 5h 15m 30s 29.10 GB
kitchen n12 13.47 0.397 0.635 5h 56m 43s 30.69 GB
bonsai n12 14.37 0.474 0.623 5h 11m 15s 30.69 GB
counter n12 12.03 0.379 0.685 5h 23m 6s 30.69 GB
room n12 18.91 0.711 0.430 5h 56m 45s 30.69 GB
garden n24 24.47 0.754 0.209 5h 15m 40s 29.28 GB
bicycle n24 14.53 0.209 0.610 5h 14m 20s 29.08 GB
flowers n24 13.77 0.239 0.593 5h 14m 16s 29.00 GB
treehill n24 17.73 0.360 0.480 5h 41m 51s 29.19 GB
stump n24 17.29 0.239 0.607 5h 13m 15s 29.10 GB
kitchen n24 14.56 0.516 0.541 5h 22m 22s 30.69 GB
bonsai n24 17.48 0.718 0.425 5h 24m 17s 30.69 GB
counter n24 14.61 0.517 0.584 5h 27m 33s 30.69 GB
room n24 24.28 0.816 0.332 5h 24m 56s 30.69 GB
Average 16.47 0.428 0.535 5h 27m 11s 29.82 GB