NerfBaselines

Nerf
Baselines

GitHub Paper Docs

NerfBaselines is a framework for evaluating and comparing existing NeRF methods. Currently, most official implementations use different dataset loaders, evaluation protocols, and metrics which renders the comparison of methods difficult. Therefore, this project aims to provide a unified interface for running and evaluating methods on different datasets in a consistent way using the same metrics. But instead of reimplementing the methods, we use the official implementations and wrap them so that they can be run easily using the same interface.

Mip-NeRF 360 

Mip-NeRF 360 is a collection of four indoor and five outdoor object-centric scenes. The camera trajectory is an orbit around the object with fixed elevation and radius. The test set takes each n-th frame of the trajectory as test views.

Method PSNR SSIM LPIPS (VGG) Time GPU mem.
COLMAP 16.67 0.445 0.590 2h 52m 55s 0.00 MB
garden 18.87 0.468 0.477 1h 35m 12s 0.00 MB
bicycle 18.29 0.352 0.644 3h 19m 55s 0.00 MB
flowers 14.50 0.257 0.634 1h 51m 21s 0.00 MB
treehill 15.73 0.340 0.726 1h 6m 41s 0.00 MB
stump 19.65 0.366 0.646 3h 6m 50s 0.00 MB
kitchen 17.35 0.497 0.575 3h 31m 29s 0.00 MB
bonsai 14.06 0.548 0.586 3h 45m 27s 0.00 MB
counter 15.04 0.549 0.520 3h 45m 10s 0.00 MB
room 16.55 0.628 0.505 3h 54m 6s 0.00 MB
Instant NGP 25.51 0.684 0.398 3m 54s 7.79 GB
garden 24.78 0.654 0.346 3m 57s 6.86 GB
bicycle 23.00 0.526 0.489 3m 37s 6.29 GB
flowers 20.54 0.462 0.478 4m 2s 6.26 GB
treehill 22.36 0.528 0.526 3m 40s 6.51 GB
stump 23.84 0.590 0.439 3m 52s 6.07 GB
kitchen 29.02 0.844 0.255 4m 19s 9.52 GB
bonsai 30.30 0.890 0.295 3m 33s 9.61 GB
counter 26.56 0.812 0.373 4m 10s 9.25 GB
room 29.16 0.850 0.383 3m 59s 9.73 GB
NerfStudio 26.39 0.731 0.343 19m 30s 5.86 GB
garden 25.89
Paper's PSNR: 26.47
0.752
Paper's SSIM: 0.774
0.254
Paper's LPIPS (VGG): 0.235
19m 31s 5.30 GB
bicycle 23.58
Paper's PSNR: 24.08
0.567
Paper's SSIM: 0.599
0.456
Paper's LPIPS (VGG): 0.422
19m 12s 5.15 GB
flowers 21.16 0.511 0.434 19m 6s 5.22 GB
treehill 22.85 0.549 0.488 19m 31s 5.24 GB
stump 25.81
Paper's PSNR: 24.78
0.697
Paper's SSIM: 0.662
0.353
Paper's LPIPS (VGG): 0.38
18m 53s 5.15 GB
kitchen 29.92
Paper's PSNR: 30.29
0.883
Paper's SSIM: 0.89
0.200
Paper's LPIPS (VGG): 0.19
20m 5s 6.66 GB
bonsai 30.59
Paper's PSNR: 32.16
0.907
Paper's SSIM: 0.933
0.249
Paper's LPIPS (VGG): 0.197
19m 46s 6.66 GB
counter 27.09
Paper's PSNR: 27.2
0.830
Paper's SSIM: 0.843
0.336
Paper's LPIPS (VGG): 0.314
19m 28s 6.66 GB
room 30.61
Paper's PSNR: 30.89
0.880
Paper's SSIM: 0.896
0.315
Paper's LPIPS (VGG): 0.296
19m 53s 6.66 GB
2D Gaussian Splatting 26.81
Paper's PSNR: 27.04

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.796
Paper's SSIM: 0.805

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.297 31m 10s 13.16 GB
garden 26.69
Paper's PSNR: 26.95

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.843
Paper's SSIM: 0.852

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.166 27m 56s 8.58 GB
bicycle 24.77
Paper's PSNR: 24.87

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.733
Paper's SSIM: 0.752

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.302 33m 18s 11.96 GB
flowers 21.14
Paper's PSNR: 21.15

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.572
Paper's SSIM: 0.588

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.403 26m 31s 7.64 GB
treehill 22.36
Paper's PSNR: 22.27

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.616
Paper's SSIM: 0.627

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.433 28m 44s 8.81 GB
stump 26.20
Paper's PSNR: 26.47

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.758
Paper's SSIM: 0.765

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.299 29m 12s 28.56 GB
kitchen 30.41
Paper's PSNR: 30.5

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.916
Paper's SSIM: 0.919

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.179 35m 31s 15.79 GB
bonsai 31.30
Paper's PSNR: 31.52

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.931
Paper's SSIM: 0.933

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.280 33m 16s 15.39 GB
counter 28.10
Paper's PSNR: 28.55

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.892
Paper's SSIM: 0.9

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.292 32m 38s 9.95 GB
room 30.37
Paper's PSNR: 31.06

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.906
Paper's SSIM: 0.912

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.317 33m 20s 11.74 GB
H3DGS 26.93 0.790 0.269 55m 29s 9.08 GB
garden 26.44 0.820 0.191 52m 35s 8.97 GB
bicycle 24.03 0.717 0.284 57m 19s 11.46 GB
flowers 20.53 0.571 0.344 59m 11s 11.99 GB
treehill 22.09 0.598 0.384 54m 52s 10.48 GB
stump 25.37 0.728 0.287 50m 50s 9.07 GB
kitchen 31.06 0.918 0.163 53m 54s 6.70 GB
bonsai 31.86 0.937 0.243 56m 6s 6.97 GB
counter 29.07 0.906 0.243 58m 13s 7.64 GB
room 31.89 0.918 0.284 56m 22s 8.41 GB
3DGUT 27.04
Paper's PSNR: 27.26
0.812
Paper's SSIM: 0.810
0.252 34m 34s 14.92 GB
garden 27.08
Paper's PSNR: 26.9
0.858
Paper's SSIM: 0.851
0.136 32m 38s 13.99 GB
bicycle 24.90
Paper's PSNR: 24.21
0.759
Paper's SSIM: 0.741
0.236 33m 59s 15.47 GB
flowers 21.37
Paper's PSNR: 21.48
0.615
Paper's SSIM: 0.612
0.343 35m 13s 12.42 GB
treehill 22.40
Paper's PSNR: 22.15
0.632
Paper's SSIM: 0.623
0.358 36m 59s 15.51 GB
stump 26.50
Paper's PSNR: 26.51
0.774
Paper's SSIM: 0.768
0.248 28m 35s 13.13 GB
kitchen 30.21
Paper's PSNR: 31.23
0.922
Paper's SSIM: 0.926
0.159 44m 5s 16.10 GB
bonsai 31.62
Paper's PSNR: 32.17
0.935
Paper's SSIM: 0.941
0.252 31m 30s 16.02 GB
counter 28.70
Paper's PSNR: 29.03
0.904
Paper's SSIM: 0.908
0.252 33m 39s 14.31 GB
room 30.59
Paper's PSNR: 31.64
0.913
Paper's SSIM: 0.919
0.284 34m 31s 17.32 GB
PGSR 27.20 0.819 0.233 39m 58s 14.28 GB
garden 27.44 0.870 0.116 42m 48s 13.71 GB
bicycle 25.28 0.780 0.204 50m 8s 16.71 GB
flowers 21.27 0.618 0.304 42m 10s 14.01 GB
treehill 21.91 0.623 0.322 44m 41s 14.46 GB
stump 26.85 0.787 0.225 40m 42s 12.83 GB
kitchen 30.94 0.923 0.161 36m 2s 13.74 GB
bonsai 31.89 0.941 0.244 33m 23s 14.36 GB
counter 28.64 0.910 0.246 35m 6s 13.87 GB
room 30.58 0.917 0.277 34m 38s 14.85 GB
Taming 3DGS 27.22
Paper's PSNR: 27.29

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.793 0.305 5m 59s 8.69 GB
garden 27.42
Paper's PSNR: 27.45

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.858 0.145 8m 20s 8.10 GB
bicycle 24.75
Paper's PSNR: 24.97

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.712 0.336 5m 16s 9.15 GB
flowers 21.10
Paper's PSNR: 21.18

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.551 0.438 4m 51s 6.18 GB
treehill 23.02
Paper's PSNR: 23.04

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.623 0.443 5m 18s 6.58 GB
stump 25.95
Paper's PSNR: 26.04

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.734 0.336 4m 12s 5.35 GB
kitchen 30.92
Paper's PSNR: 31.14

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.921 0.172 7m 30s 10.91 GB
bonsai 31.86
Paper's PSNR: 31.8

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.935 0.271 6m 7s 10.80 GB
counter 28.53
Paper's PSNR: 28.59

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.897 0.283 6m 37s 9.88 GB
room 31.41
Paper's PSNR: 31.39

Experiments use the default 'ours' configuration. There is also 'big' configuration which uses more resources and achieves better results.

 0.906 0.321 5m 40s 11.28 GB
Octree-GS 27.40 0.811 0.264 28m 3s 8.52 GB
garden 27.72 0.869 0.124 30m 15s 7.65 GB
bicycle 25.03 0.759 0.251 24m 55s 7.50 GB
flowers 21.43 0.600 0.374 22m 57s 6.53 GB
treehill 23.01 0.643 0.360 24m 41s 6.44 GB
stump 26.48 0.763 0.276 20m 38s 5.79 GB
kitchen 30.52 0.917 0.172 35m 35s 10.41 GB
bonsai 30.88 0.923 0.280 27m 5s 10.86 GB
counter 29.49 0.907 0.262 34m 58s 9.85 GB
room 32.01 0.922 0.275 31m 21s 11.66 GB
gsplat 27.41 0.815 0.256 29m 19s 8.30 GB
garden 27.49
Paper's PSNR: 27.39
0.868
Paper's SSIM: 0.87
0.120 39m 56s 12.52 GB
bicycle 25.26
Paper's PSNR: 25.29
0.766
Paper's SSIM: 0.77
0.237 35m 41s 11.75 GB
flowers 21.59 0.603 0.368 26m 2s 8.64 GB
treehill 22.57 0.635 0.377 24m 28s 7.79 GB
stump 26.57
Paper's PSNR: 26.51
0.772
Paper's SSIM: 0.77
0.248 26m 40s 9.63 GB
kitchen 30.86
Paper's PSNR: 31.37
0.923
Paper's SSIM: 0.93
0.158 30m 54s 6.79 GB
bonsai 32.13
Paper's PSNR: 32.21
0.941
Paper's SSIM: 0.94
0.254 23m 13s 5.69 GB
counter 28.94
Paper's PSNR: 29.01
0.906
Paper's SSIM: 0.91
0.257 28m 24s 5.79 GB
room 31.30
Paper's PSNR: 31.23
0.916
Paper's SSIM: 0.92
0.286 28m 30s 6.11 GB
Gaussian Opacity Fields 27.42 0.826 0.234 1h 3m 54s 28.44 GB
garden 27.55 0.874 0.115 1h 13m 32s 34.59 GB
bicycle 25.52 0.792 0.197 1h 11m 12s 37.49 GB
flowers 21.71 0.639 0.308 53m 26s 33.63 GB
treehill 22.44 0.643 0.320 57m 48s 33.73 GB
stump 26.99 0.798 0.223 52m 14s 33.82 GB
kitchen 31.23 0.925 0.158 1h 12m 30s 22.96 GB
bonsai 31.94 0.940 0.247 59m 18s 23.38 GB
counter 28.83 0.907 0.258 1h 5m 60s 12.64 GB
room 30.59 0.913 0.282 1h 9m 1s 23.70 GB
Gaussian Splatting 27.43
Paper's PSNR: 27.20
0.814
Paper's SSIM: 0.815
0.257
Paper's LPIPS (VGG): 0.214
23m 25s 11.14 GB
garden 27.34
Paper's PSNR: 27.41
0.866
Paper's SSIM: 0.868
0.124
Paper's LPIPS (VGG): 0.103
28m 58s 14.75 GB
bicycle 25.21
Paper's PSNR: 25.246
0.764
Paper's SSIM: 0.771
0.240
Paper's LPIPS (VGG): 0.205
28m 25s 14.90 GB
flowers 21.60
Paper's PSNR: 21.52
0.604
Paper's SSIM: 0.605
0.367
Paper's LPIPS (VGG): 0.336
20m 53s 10.01 GB
treehill 22.44
Paper's PSNR: 22.49
0.631
Paper's SSIM: 0.638
0.377
Paper's LPIPS (VGG): 0.317
20m 51s 11.36 GB
stump 26.58
Paper's PSNR: 26.55
0.771
Paper's SSIM: 0.775
0.250
Paper's LPIPS (VGG): 0.21
22m 30s 12.53 GB
kitchen 31.14
Paper's PSNR: 30.317
0.926
Paper's SSIM: 0.922
0.155
Paper's LPIPS (VGG): 0.129
25m 35s 9.75 GB
bonsai 32.20
Paper's PSNR: 31.98
0.941
Paper's SSIM: 0.938
0.254
Paper's LPIPS (VGG): 0.205
19m 15s 8.35 GB
counter 28.96
Paper's PSNR: 28.7
0.907
Paper's SSIM: 0.905
0.258
Paper's LPIPS (VGG): 0.204
21m 53s 8.55 GB
room 31.43
Paper's PSNR: 30.632
0.917
Paper's SSIM: 0.914
0.287
Paper's LPIPS (VGG): 0.22
22m 23s 10.05 GB
Mip-Splatting 27.49
Paper's PSNR: 27.79

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.815
Paper's SSIM: 0.827

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.258
Paper's LPIPS (VGG): 0.203

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 25m 37s 11.01 GB
garden 27.47
Paper's PSNR: 27.76

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.869
Paper's SSIM: 0.875

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.124
Paper's LPIPS (VGG): 0.103

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 30m 6s 13.45 GB
bicycle 25.25
Paper's PSNR: 25.72

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.765
Paper's SSIM: 0.78

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.243
Paper's LPIPS (VGG): 0.206

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 33m 19s 15.07 GB
flowers 21.60
Paper's PSNR: 21.93

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.605
Paper's SSIM: 0.623

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.371
Paper's LPIPS (VGG): 0.331

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 22m 47s 9.90 GB
treehill 22.65
Paper's PSNR: 22.98

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.633
Paper's SSIM: 0.655

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.381
Paper's LPIPS (VGG): 0.32

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 22m 49s 9.79 GB
stump 26.64
Paper's PSNR: 26.94

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.774
Paper's SSIM: 0.786

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.251
Paper's LPIPS (VGG): 0.209

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 25m 8s 12.49 GB
kitchen 31.25
Paper's PSNR: 31.55

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.926
Paper's SSIM: 0.933

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.155
Paper's LPIPS (VGG): 0.113

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 27m 47s 9.50 GB
bonsai 31.96
Paper's PSNR: 32.31

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.941
Paper's SSIM: 0.948

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.254
Paper's LPIPS (VGG): 0.173

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 20m 49s 8.98 GB
counter 29.04
Paper's PSNR: 29.16

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.907
Paper's SSIM: 0.916

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.258
Paper's LPIPS (VGG): 0.179

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 23m 21s 9.03 GB
room 31.54
Paper's PSNR: 31.74

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.918
Paper's SSIM: 0.928

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.286
Paper's LPIPS (VGG): 0.192

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 24m 25s 10.85 GB
Mip-NeRF 360 27.68
Paper's PSNR: 27.69
0.792
Paper's SSIM: 0.792
0.272
Paper's LPIPS (VGG): 0.237
30h 14m 36s 33.61 GB
garden 27.00
Paper's PSNR: 26.98
0.814
Paper's SSIM: 0.813
0.189
Paper's LPIPS (VGG): 0.17
30h 13m 41s 33.06 GB
bicycle 24.28
Paper's PSNR: 24.37
0.686
Paper's SSIM: 0.685
0.329
Paper's LPIPS (VGG): 0.301
30h 14m 19s 32.86 GB
flowers 21.74
Paper's PSNR: 21.73
0.583
Paper's SSIM: 0.583
0.371
Paper's LPIPS (VGG): 0.344
30h 13m 53s 32.78 GB
treehill 22.89
Paper's PSNR: 22.87
0.633
Paper's SSIM: 0.632
0.378
Paper's LPIPS (VGG): 0.339
30h 13m 9s 32.98 GB
stump 26.50
Paper's PSNR: 26.4
0.749
Paper's SSIM: 0.744
0.298
Paper's LPIPS (VGG): 0.261
30h 15m 5s 32.88 GB
kitchen 32.10
Paper's PSNR: 32.23
0.919
Paper's SSIM: 0.92
0.155
Paper's LPIPS (VGG): 0.127
30h 15m 15s 34.48 GB
bonsai 33.48
Paper's PSNR: 33.46
0.940
Paper's SSIM: 0.941
0.211
Paper's LPIPS (VGG): 0.176
30h 13m 6s 34.48 GB
counter 29.51
Paper's PSNR: 29.55
0.894
Paper's SSIM: 0.894
0.252
Paper's LPIPS (VGG): 0.204
30h 14m 4s 34.47 GB
room 31.64
Paper's PSNR: 31.63
0.912
Paper's SSIM: 0.913
0.267
Paper's LPIPS (VGG): 0.211
30h 18m 49s 34.47 GB
Scaffold-GS 27.71 0.813 0.262 23m 28s 8.72 GB
garden 27.50
Paper's PSNR: 27.17
0.863
Paper's SSIM: 0.842
0.136 21m 46s 7.91 GB
bicycle 25.19
Paper's PSNR: 24.5
0.759
Paper's SSIM: 0.705
0.259 20m 38s 7.37 GB
flowers 21.44 0.592 0.382 20m 23s 6.90 GB
treehill 23.15 0.640 0.373 20m 17s 6.48 GB
stump 26.59
Paper's PSNR: 26.27
0.766
Paper's SSIM: 0.784
0.277 17m 1s 6.22 GB
kitchen 31.59
Paper's PSNR: 31.3
0.927
Paper's SSIM: 0.928
0.156 33m 25s 10.97 GB
bonsai 32.58
Paper's PSNR: 32.7
0.943
Paper's SSIM: 0.946
0.249 24m 33s 10.94 GB
counter 29.48
Paper's PSNR: 29.34
0.910
Paper's SSIM: 0.914
0.256 28m 34s 10.38 GB
room 31.89
Paper's PSNR: 31.93
0.922
Paper's SSIM: 0.925
0.275 24m 37s 11.28 GB
3DGS-MCMC 27.98 0.835 0.224 41m 11s 28.86 GB
garden 27.81
Paper's PSNR: 28.16

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.877
Paper's SSIM: 0.89

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.109 48m 5s 35.23 GB
bicycle 25.69
Paper's PSNR: 26.15

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.799
Paper's SSIM: 0.81

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.194 49m 28s 38.87 GB
flowers 22.05 0.644 0.316 37m 29s 38.42 GB
treehill 22.97 0.658 0.315 37m 33s 38.15 GB
stump 27.38
Paper's PSNR: 27.8

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.811
Paper's SSIM: 0.82

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.200 40m 48s 34.43 GB
kitchen 31.91
Paper's PSNR: 32.27

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.933
Paper's SSIM: 0.94

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.148 42m 32s 20.35 GB
bonsai 32.66
Paper's PSNR: 32.88

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.947
Paper's SSIM: 0.95

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.236 35m 14s 14.88 GB
counter 29.32
Paper's PSNR: 29.51

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.916
Paper's SSIM: 0.92

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.238 38m 44s 20.42 GB
room 32.05
Paper's PSNR: 32.48

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.927
Paper's SSIM: 0.94

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.262 40m 43s 18.96 GB
Zip-NeRF 28.55
Paper's PSNR: 28.54
0.829
Paper's SSIM: 0.828
0.218
Paper's LPIPS (VGG): 0.189
5h 30m 20s 26.82 GB
garden 28.19
Paper's PSNR: 28.2
0.864
Paper's SSIM: 0.86
0.127
Paper's LPIPS (VGG): 0.118
5h 19m 14s 26.77 GB
bicycle 25.89
Paper's PSNR: 25.8
0.775
Paper's SSIM: 0.769
0.226
Paper's LPIPS (VGG): 0.208
5h 22m 47s 26.74 GB
flowers 22.33
Paper's PSNR: 22.4
0.637
Paper's SSIM: 0.642
0.310
Paper's LPIPS (VGG): 0.273
5h 16m 23s 26.75 GB
treehill 24.01
Paper's PSNR: 23.89
0.677
Paper's SSIM: 0.681
0.279
Paper's LPIPS (VGG): 0.242
5h 48m 6s 26.75 GB
stump 27.41
Paper's PSNR: 27.55
0.792
Paper's SSIM: 0.8
0.233
Paper's LPIPS (VGG): 0.193
5h 25m 16s 26.74 GB
kitchen 32.35
Paper's PSNR: 32.5
0.929
Paper's SSIM: 0.928
0.134
Paper's LPIPS (VGG): 0.116
5h 39m 34s 26.92 GB
bonsai 34.68
Paper's PSNR: 34.46
0.951
Paper's SSIM: 0.949
0.195
Paper's LPIPS (VGG): 0.173
5h 35m 52s 26.92 GB
counter 29.13
Paper's PSNR: 29.38
0.905
Paper's SSIM: 0.902
0.223
Paper's LPIPS (VGG): 0.185
5h 39m 53s 26.91 GB
room 32.99
Paper's PSNR: 32.65
0.928
Paper's SSIM: 0.925
0.237
Paper's LPIPS (VGG): 0.196
5h 25m 53s 26.91 GB

Blender 

Blender (nerf-synthetic) is a synthetic dataset used to benchmark NeRF methods. It consists of 8 scenes of an object placed on a white background. Cameras are placed on a semi-sphere around the object. Scenes are licensed under various CC licenses.

Method PSNR SSIM LPIPS (VGG) Time GPU mem.
COLMAP 12.12 0.766 0.214 1h 20m 34s 0.00 MB
lego 12.31 0.776 0.220 2h 5m 25s 0.00 MB
drums 8.06 0.657 0.296 2h 14m 38s 0.00 MB
ficus 15.12 0.838 0.143 39m 23s 0.00 MB
hotdog 12.21 0.831 0.195 1h 13m 26s 0.00 MB
materials 14.56 0.802 0.193 1h 42m 37s 0.00 MB
mic 9.29 0.765 0.182 1h 13m 24s 0.00 MB
ship 10.02 0.616 0.335 48m 3s 0.00 MB
chair 15.42 0.847 0.148 47m 35s 0.00 MB
NeRF 28.72
Paper's PSNR: 31.00
0.936
Paper's SSIM: 0.947
0.092
Paper's LPIPS (VGG): 0.081
23h 26m 30s 10.25 GB
lego 32.86
Paper's PSNR: 32.54
0.964
Paper's SSIM: 0.961
0.049
Paper's LPIPS (VGG): 0.05
23h 32m 41s 10.25 GB
drums 25.12
Paper's PSNR: 25.01
0.926
Paper's SSIM: 0.925
0.087
Paper's LPIPS (VGG): 0.091
22h 56m 29s 10.25 GB
ficus 30.31
Paper's PSNR: 30.13
0.964
Paper's SSIM: 0.964
0.042
Paper's LPIPS (VGG): 0.044
23h 38m 35s 10.25 GB
hotdog 36.39
Paper's PSNR: 36.18
0.975
Paper's SSIM: 0.974
0.095
Paper's LPIPS (VGG): 0.121
23h 14m 14s 10.25 GB
materials 29.84
Paper's PSNR: 29.62
0.950
Paper's SSIM: 0.949
0.062
Paper's LPIPS (VGG): 0.063
23h 58m 42s 10.25 GB
mic 13.03
Paper's PSNR: 32.91
0.881
Paper's SSIM: 0.98
0.154
Paper's LPIPS (VGG): 0.028
23h 21m 46s 10.25 GB
ship 28.80
Paper's PSNR: 28.65
0.859
Paper's SSIM: 0.856
0.210
Paper's LPIPS (VGG): 0.206
23h 30m 51s 10.25 GB
chair 33.43
Paper's PSNR: 33.0
0.970
Paper's SSIM: 0.967
0.041
Paper's LPIPS (VGG): 0.046
23h 18m 37s 10.25 GB
NerfStudio 29.19 0.941 0.095 9m 38s 3.65 GB
lego 31.37 0.967 0.069 9m 44s 3.65 GB
drums 22.48 0.897 0.139 9m 12s 3.65 GB
ficus 27.82 0.957 0.087 9m 22s 3.65 GB
hotdog 31.09 0.963 0.104 9m 44s 3.65 GB
materials 25.38 0.903 0.121 9m 23s 3.65 GB
mic 33.74 0.984 0.029 9m 47s 3.65 GB
ship 28.71 0.881 0.166 10m 2s 3.64 GB
chair 32.94 0.977 0.044 9m 48s 3.65 GB
Mip-NeRF 360 30.34 0.951 0.060 3h 29m 39s 114.80 GB
lego 33.20 0.975 0.028 2h 43m 18s 126.91 GB
drums 24.36 0.923 0.083 8h 51m 32s 30.02 GB
ficus 26.66 0.952 0.048 2h 43m 27s 126.91 GB
hotdog 36.44 0.979 0.039 2h 43m 11s 126.91 GB
materials 27.91 0.944 0.067 2h 43m 22s 126.91 GB
mic 31.50 0.984 0.021 2h 43m 9s 126.91 GB
ship 28.66 0.875 0.164 2h 45m 55s 126.91 GB
chair 34.01 0.977 0.032 2h 43m 17s 126.91 GB
gsplat 31.47 0.966 0.054 14m 45s 2.80 GB
lego 32.13 0.977 0.043 13m 31s 2.94 GB
drums 24.50 0.948 0.076 11m 15s 2.74 GB
ficus 34.44 0.986 0.014 11m 48s 2.75 GB
hotdog 35.39 0.983 0.041 20m 32s 2.67 GB
materials 29.93 0.959 0.044 11m 37s 2.76 GB
mic 33.73 0.991 0.023 11m 30s 2.84 GB
ship 28.77 0.899 0.155 17m 56s 2.86 GB
chair 32.88 0.986 0.034 19m 53s 2.82 GB
Tetra-NeRF 31.95
Paper's PSNR: 32.52
0.957
Paper's SSIM: 0.982
0.056 6h 53m 20s 29.57 GB
lego 33.93
Paper's PSNR: 34.75
0.972
Paper's SSIM: 0.987
0.036 7h 8m 12s 25.21 GB
drums 24.99
Paper's PSNR: 25.01
0.927
Paper's SSIM: 0.947
0.087 8h 34m 5s 36.26 GB
ficus 32.37
Paper's PSNR: 33.31
0.977
Paper's SSIM: 0.989
0.032 4h 19m 24s 21.84 GB
hotdog 35.80
Paper's PSNR: 36.16
0.978
Paper's SSIM: 0.989
0.040 6h 41m 21s 23.06 GB
materials 28.75
Paper's PSNR: 29.3
0.941
Paper's SSIM: 0.968
0.076 6h 23m 31s 37.98 GB
mic 34.54
Paper's PSNR: 35.49
0.987
Paper's SSIM: 0.993
0.022 6h 51m 58s 31.04 GB
ship 31.06
Paper's PSNR: 31.13
0.896
Paper's SSIM: 0.994
0.129 9h 4m 53s 27.19 GB
chair 34.17
Paper's PSNR: 35.05
0.977
Paper's SSIM: 0.99
0.029 6h 3m 14s 34.01 GB
Instant NGP 32.20
Paper's PSNR: 33.18

Instant-NGP trained and evaluated on black background instead of white.

 0.959 0.055 2m 23s 2.57 GB
lego 35.65
Paper's PSNR: 36.39

Instant-NGP trained and evaluated on black background instead of white.

 0.981 0.020 2m 41s 2.59 GB
drums 24.57
Paper's PSNR: 26.02

Instant-NGP trained and evaluated on black background instead of white.

 0.930 0.109 2m 6s 2.58 GB
ficus 30.29
Paper's PSNR: 33.51

Instant-NGP trained and evaluated on black background instead of white.

 0.972 0.031 1m 58s 2.58 GB
hotdog 37.02
Paper's PSNR: 37.4

Instant-NGP trained and evaluated on black background instead of white.

 0.982 0.037 2m 33s 2.58 GB
materials 28.96
Paper's PSNR: 29.78

Instant-NGP trained and evaluated on black background instead of white.

 0.944 0.069 2m 11s 2.58 GB
mic 35.41
Paper's PSNR: 36.22

Instant-NGP trained and evaluated on black background instead of white.

 0.989 0.016 2m 25s 2.47 GB
ship 30.61
Paper's PSNR: 31.1

Instant-NGP trained and evaluated on black background instead of white.

 0.892 0.136 2m 24s 2.58 GB
chair 35.07
Paper's PSNR: 35.0

Instant-NGP trained and evaluated on black background instead of white.

 0.984 0.023 2m 49s 2.57 GB
K-Planes 32.27 0.961 0.062 23m 58s 4.64 GB
lego 35.73 0.981 0.047 23m 54s 4.64 GB
drums 25.68 0.938 0.096 24m 10s 4.64 GB
ficus 31.31 0.974 0.052 23m 37s 4.64 GB
hotdog 36.46 0.981 0.033 23m 38s 4.64 GB
materials 29.41 0.949 0.070 24m 1s 4.64 GB
mic 33.95 0.988 0.019 24m 6s 4.64 GB
ship 30.72 0.897 0.141 24m 8s 4.64 GB
chair 34.85 0.983 0.036 24m 9s 4.64 GB
Scaffold-GS 33.08
Paper's PSNR: 33.68
0.966 0.048 7m 4s 3.72 GB
lego 34.96
Paper's PSNR: 35.69
0.980 0.024 6m 40s 3.72 GB
drums 26.28
Paper's PSNR: 26.44
0.949 0.054 6m 46s 3.73 GB
ficus 34.38
Paper's PSNR: 35.21
0.986 0.015 6m 9s 3.69 GB
hotdog 37.62
Paper's PSNR: 37.73
0.983 0.075 8m 26s 3.71 GB
materials 30.29
Paper's PSNR: 30.65
0.962 0.045 6m 49s 3.73 GB
mic 35.99
Paper's PSNR: 37.25
0.991 0.010 6m 31s 3.71 GB
ship 29.97
Paper's PSNR: 31.17
0.895 0.139 8m 30s 3.74 GB
chair 35.16
Paper's PSNR: 35.28
0.984 0.019 6m 38s 3.71 GB
TensoRF 33.17
Paper's PSNR: 33.14
0.963
Paper's SSIM: 0.963
0.051
Paper's LPIPS (VGG): 0.047
10m 47s 16.37 GB
lego 36.49
Paper's PSNR: 36.46
0.983
Paper's SSIM: 0.983
0.022
Paper's LPIPS (VGG): 0.018
10m 1s 20.53 GB
drums 26.01
Paper's PSNR: 26.01
0.936
Paper's SSIM: 0.937
0.076
Paper's LPIPS (VGG): 0.073
8m 56s 19.67 GB
ficus 34.06
Paper's PSNR: 33.99
0.982
Paper's SSIM: 0.982
0.029
Paper's LPIPS (VGG): 0.022
10m 45s 7.28 GB
hotdog 37.49
Paper's PSNR: 37.41
0.982
Paper's SSIM: 0.982
0.033
Paper's LPIPS (VGG): 0.032
11m 18s 19.77 GB
materials 30.08
Paper's PSNR: 30.12
0.952
Paper's SSIM: 0.952
0.059
Paper's LPIPS (VGG): 0.058
14m 55s 21.09 GB
mic 34.85
Paper's PSNR: 34.61
0.988
Paper's SSIM: 0.988
0.021
Paper's LPIPS (VGG): 0.015
8m 8s 11.21 GB
ship 30.69
Paper's PSNR: 30.77
0.894
Paper's SSIM: 0.895
0.141
Paper's LPIPS (VGG): 0.138
13m 19s 18.80 GB
chair 35.72
Paper's PSNR: 35.76
0.984
Paper's SSIM: 0.985
0.027
Paper's LPIPS (VGG): 0.022
8m 56s 12.60 GB
PGSR 33.27 0.968 0.039 8m 20s 3.94 GB
lego 35.12 0.980 0.023 8m 8s 3.95 GB
drums 26.08 0.953 0.045 8m 19s 4.01 GB
ficus 34.75 0.987 0.013 7m 18s 3.86 GB
hotdog 37.42 0.984 0.031 8m 43s 3.84 GB
materials 29.93 0.959 0.044 7m 57s 3.96 GB
mic 35.66 0.992 0.008 7m 52s 3.93 GB
ship 31.44 0.906 0.129 10m 27s 4.03 GB
chair 35.79 0.987 0.017 7m 54s 3.94 GB
Gaussian Splatting 33.31
Paper's PSNR: 33.31
0.969 0.037 6m 6s 3.08 GB
lego 35.70
Paper's PSNR: 35.78
0.982 0.019 5m 59s 3.07 GB
drums 26.15
Paper's PSNR: 26.15
0.953 0.044 6m 3s 3.07 GB
ficus 34.79
Paper's PSNR: 34.87
0.987 0.013 5m 22s 2.82 GB
hotdog 37.64
Paper's PSNR: 37.72
0.985 0.026 5m 49s 2.85 GB
materials 30.01
Paper's PSNR: 30.0
0.959 0.043 5m 28s 2.91 GB
mic 35.49
Paper's PSNR: 35.36
0.991 0.008 6m 11s 3.02 GB
ship 30.85
Paper's PSNR: 30.8
0.904 0.130 8m 13s 3.99 GB
chair 35.84
Paper's PSNR: 35.83
0.987 0.015 5m 45s 2.89 GB
Mip-Splatting 33.33 0.969 0.039 6m 49s 2.65 GB
lego 35.45 0.982 0.021 6m 30s 2.51 GB
drums 26.14 0.953 0.046 6m 36s 2.53 GB
ficus 35.12 0.988 0.013 5m 8s 2.43 GB
hotdog 37.78 0.985 0.028 6m 44s 2.58 GB
materials 30.12 0.960 0.044 5m 49s 2.46 GB
mic 35.55 0.991 0.008 8m 8s 2.75 GB
ship 30.78 0.904 0.132 9m 19s 3.54 GB
chair 35.70 0.986 0.018 6m 16s 2.40 GB
Gaussian Opacity Fields 33.45 0.969 0.038 18m 26s 3.15 GB
lego 35.56 0.982 0.021 19m 4s 2.88 GB
drums 26.17 0.955 0.045 17m 50s 2.84 GB
ficus 35.19 0.988 0.013 11m 10s 2.71 GB
hotdog 37.46 0.985 0.028 17m 52s 2.61 GB
materials 30.20 0.961 0.043 14m 48s 2.72 GB
mic 36.06 0.992 0.008 21m 56s 2.99 GB
ship 30.68 0.901 0.131 29m 60s 5.64 GB
chair 36.28 0.988 0.017 14m 46s 2.79 GB
3DGUT 33.48
Paper's PSNR: 33.88

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.969
Paper's SSIM: 0.970

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.036 7m 16s 4.66 GB
lego 36.12
Paper's PSNR: 36.47

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.983
Paper's SSIM: 0.984

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.018 7m 15s 4.62 GB
drums 25.98
Paper's PSNR: 25.99

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.952
Paper's SSIM: 0.953

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.047 8m 17s 4.79 GB
ficus 35.86
Paper's PSNR: 36.43

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.988
Paper's SSIM: 0.988

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.012 6m 17s 4.62 GB
hotdog 37.69
Paper's PSNR: 38.11

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.985
Paper's SSIM: 0.986

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.026 5m 47s 4.46 GB
materials 30.10
Paper's PSNR: 30.39

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.960
Paper's SSIM: 0.96

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.043 7m 51s 4.76 GB
mic 35.37
Paper's PSNR: 36.32

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.991
Paper's SSIM: 0.992

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.008 7m 30s 4.61 GB
ship 30.90
Paper's PSNR: 31.72

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.905
Paper's SSIM: 0.908

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.123 7m 55s 4.71 GB
chair 35.79
Paper's PSNR: 35.61

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.987
Paper's SSIM: 0.988

The method was trained and evaluated on black background instead of white, which is the default in NerfBaselines. The results are not part of the paper (only results for `Ours (sorted)` are reported), but are provided in the GitHub repository.

 0.014 7m 17s 4.69 GB
3DGS-MCMC 33.64
Paper's PSNR: 33.80

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.971
Paper's SSIM: 0.970

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.036 9m 8s 4.51 GB
lego 35.75
Paper's PSNR: 36.01

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.984
Paper's SSIM: 0.98

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.019 8m 51s 4.50 GB
drums 26.22
Paper's PSNR: 26.29

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.954
Paper's SSIM: 0.95

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.045 9m 3s 4.51 GB
ficus 34.88
Paper's PSNR: 35.07

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.988
Paper's SSIM: 0.99

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.013 8m 2s 4.54 GB
hotdog 37.34
Paper's PSNR: 37.82

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.986
Paper's SSIM: 0.99

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.025 9m 55s 4.50 GB
materials 30.56
Paper's PSNR: 30.59

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.964
Paper's SSIM: 0.96

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.039 9m 8s 4.35 GB
mic 37.28
Paper's PSNR: 37.29

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.994
Paper's SSIM: 0.99

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.007 9m 36s 4.47 GB
ship 30.76
Paper's PSNR: 30.82

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.907
Paper's SSIM: 0.91

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.124 10m 38s 4.87 GB
chair 36.32
Paper's PSNR: 36.51

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.989
Paper's SSIM: 0.99

Exact hyperparameters for Blender dataset are not provided in the released source code. The default parameters were used in NerfBaselines likely leading to worse results.

 0.016 7m 54s 4.30 GB
Zip-NeRF 33.67
Paper's PSNR: 33.09
0.973
Paper's SSIM: 0.971
0.036
Paper's LPIPS (VGG): 0.031
5h 21m 57s 26.20 GB
lego 35.81
Paper's PSNR: 34.84
0.983
Paper's SSIM: 0.98
0.019
Paper's LPIPS (VGG): 0.019
5h 20m 18s 26.20 GB
drums 25.90
Paper's PSNR: 25.84
0.948
Paper's SSIM: 0.944
0.054
Paper's LPIPS (VGG): 0.05
5h 20m 22s 26.20 GB
ficus 34.73
Paper's PSNR: 33.9
0.987
Paper's SSIM: 0.985
0.014
Paper's LPIPS (VGG): 0.015
5h 25m 57s 26.20 GB
hotdog 37.98
Paper's PSNR: 37.14
0.987
Paper's SSIM: 0.984
0.023
Paper's LPIPS (VGG): 0.02
5h 19m 2s 26.20 GB
materials 30.98
Paper's PSNR: 31.66
0.967
Paper's SSIM: 0.969
0.040
Paper's LPIPS (VGG): 0.032
5h 23m 48s 26.20 GB
mic 35.90
Paper's PSNR: 35.15
0.992
Paper's SSIM: 0.991
0.008
Paper's LPIPS (VGG): 0.007
5h 26m 19s 26.20 GB
ship 32.30
Paper's PSNR: 31.38
0.937
Paper's SSIM: 0.929
0.114
Paper's LPIPS (VGG): 0.091
5h 14m 50s 26.20 GB
chair 35.75
Paper's PSNR: 34.84
0.987
Paper's SSIM: 0.983
0.017
Paper's LPIPS (VGG): 0.017
5h 24m 56s 26.20 GB

Tanks and Temples 

Tanks and Temples is a benchmark for image-based 3D reconstruction. The benchmark sequences were acquired outside the lab, in realistic conditions. Ground-truth data was captured using an industrial laser scanner. The benchmark includes both outdoor scenes and indoor environments. The dataset is split into three subsets: training, intermediate, and advanced.

Method PSNR SSIM LPIPS Time GPU mem.
COLMAP 11.92 0.436 0.606 5h 16m 11s 0.00 MB
auditorium 14.55 0.390 0.637 3h 17m 31s 0.00 MB
ballroom 14.09 0.408 0.466 3h 34m 34s 0.00 MB
courtroom 14.48 0.371 0.537 3h 24m 16s 0.00 MB
museum 14.16 0.418 0.495 3h 18m 47s 0.00 MB
palace 9.08 0.407 0.784 3h 8m 22s 0.00 MB
temple 7.32 0.408 0.617 3h 5m 14s 0.00 MB
family 10.28 0.440 0.550 2h 33m 22s 0.00 MB
francis 8.33 0.302 0.686 2h 40m 27s 0.00 MB
horse 6.35 0.404 0.563 2h 41m 33s 0.00 MB
lighthouse 10.68 0.514 0.665 2h 46m 8s 0.00 MB
m60 14.19 0.547 0.550 10h 54m 40s 0.00 MB
panther 14.78 0.594 0.524 10h 32m 2s 0.00 MB
playground 13.30 0.455 0.615 10h 52m 56s 0.00 MB
train 11.83 0.396 0.813 9h 18m 35s 0.00 MB
barn 8.97 0.479 0.520 8h 15m 20s 0.00 MB
caterpillar 12.67 0.376 0.771 3h 20m 10s 0.00 MB
church 15.67 0.425 0.514 9h 2m 32s 0.00 MB
courthouse 9.09 0.397 0.737 9h 30m 56s 0.00 MB
ignatius 12.74 0.400 0.551 2h 54m 26s 0.00 MB
meetingroom 14.38 0.509 0.591 2h 48m 23s 0.00 MB
truck 13.35 0.507 0.545 2h 39m 37s 0.00 MB
2D Gaussian Splatting 21.54 0.768 0.281 15m 47s 7.23 GB
auditorium 22.86 0.835 0.304 13m 45s 6.55 GB
ballroom 23.40 0.808 0.120 19m 53s 6.12 GB
courtroom 18.92 0.660 0.422 15m 12s 9.39 GB
museum 21.11 0.764 0.170 26m 10s 8.09 GB
palace 16.91 0.668 0.512 13m 40s 5.50 GB
temple 19.74 0.763 0.305 13m 46s 4.06 GB
family 24.14 0.854 0.111 16m 16s 4.32 GB
francis 26.50 0.884 0.202 14m 56s 9.51 GB
horse 23.21 0.867 0.122 15m 15s 7.43 GB
lighthouse 18.00 0.707 0.438 13m 3s 8.47 GB
m60 22.83 0.825 0.249 15m 2s 8.98 GB
panther 21.70 0.771 0.358 13m 40s 6.39 GB
playground 21.45 0.745 0.316 16m 51s 7.16 GB
train 16.41 0.583 0.529 11m 43s 4.77 GB
barn 25.42
Paper's PSNR: 28.79

2DGS used different data pre-processing and train/test split for Tanks and Temples. It sets specific hyperparameters for each scene which may not be suitable with the public Tanks and Temples released by NerfBaselines. The results are not directly comparable and a hyperparameter tuning is needed to improve the results.

 0.816 0.213 13m 36s 6.29 GB
caterpillar 22.24
Paper's PSNR: 24.23

2DGS used different data pre-processing and train/test split for Tanks and Temples. It sets specific hyperparameters for each scene which may not be suitable with the public Tanks and Temples released by NerfBaselines. The results are not directly comparable and a hyperparameter tuning is needed to improve the results.

 0.761 0.227 14m 3s 7.15 GB
church 18.50 0.658 0.443 16m 11s 14.63 GB
courthouse 19.48
Paper's PSNR: 23.51

2DGS used different data pre-processing and train/test split for Tanks and Temples. It sets specific hyperparameters for each scene which may not be suitable with the public Tanks and Temples released by NerfBaselines. The results are not directly comparable and a hyperparameter tuning is needed to improve the results.

 0.707 0.398 13m 18s 8.98 GB
ignatius 21.34
Paper's PSNR: 23.82

2DGS used different data pre-processing and train/test split for Tanks and Temples. It sets specific hyperparameters for each scene which may not be suitable with the public Tanks and Temples released by NerfBaselines. The results are not directly comparable and a hyperparameter tuning is needed to improve the results.

 0.762 0.176 18m 57s 5.54 GB
meetingroom 24.26
Paper's PSNR: 26.15

2DGS used different data pre-processing and train/test split for Tanks and Temples. It sets specific hyperparameters for each scene which may not be suitable with the public Tanks and Temples released by NerfBaselines. The results are not directly comparable and a hyperparameter tuning is needed to improve the results.

 0.849 0.168 20m 5s 7.41 GB
truck 23.83
Paper's PSNR: 26.85

2DGS used different data pre-processing and train/test split for Tanks and Temples. It sets specific hyperparameters for each scene which may not be suitable with the public Tanks and Temples released by NerfBaselines. The results are not directly comparable and a hyperparameter tuning is needed to improve the results.

 0.843 0.123 15m 54s 4.99 GB
Instant NGP 21.62 0.712 0.340 4m 27s 4.13 GB
auditorium 20.67 0.761 0.429 4m 31s 3.92 GB
ballroom 21.62 0.652 0.352 4m 5s 4.02 GB
courtroom 19.44 0.640 0.448 4m 35s 3.93 GB
museum 15.19 0.471 0.606 5m 53s 3.93 GB
palace 19.09 0.668 0.440 4m 46s 4.64 GB
temple 17.84 0.689 0.424 4m 40s 3.94 GB
family 22.59 0.761 0.235 3m 42s 3.42 GB
francis 24.38 0.824 0.265 4m 10s 3.94 GB
horse 21.82 0.784 0.225 3m 47s 3.42 GB
lighthouse 21.65 0.765 0.281 4m 35s 4.02 GB
m60 25.82 0.832 0.202 4m 34s 4.04 GB
panther 26.81 0.844 0.208 4m 29s 4.04 GB
playground 23.33 0.696 0.344 4m 14s 3.97 GB
train 20.01 0.658 0.334 4m 39s 3.94 GB
barn 25.90 0.772 0.271 4m 41s 4.31 GB
caterpillar 21.72 0.633 0.360 4m 30s 4.22 GB
church 19.92 0.650 0.419 4m 21s 4.64 GB
courthouse 20.80 0.681 0.414 4m 55s 6.72 GB
ignatius 19.40 0.613 0.343 3m 58s 3.81 GB
meetingroom 23.24 0.783 0.326 3m 59s 4.18 GB
truck 22.85 0.770 0.216 4m 20s 3.77 GB
NerfStudio 22.04 0.743 0.270 19m 27s 3.74 GB
auditorium 20.77 0.771 0.330 19m 46s 3.88 GB
ballroom 22.68 0.705 0.261 19m 44s 3.88 GB
courtroom 20.24 0.673 0.336 19m 17s 3.88 GB
museum 17.84 0.648 0.311 18m 49s 3.88 GB
palace 17.68 0.640 0.452 20m 9s 3.64 GB
temple 17.06 0.678 0.392 19m 37s 3.88 GB
family 24.32 0.822 0.158 18m 54s 3.63 GB
francis 24.60 0.851 0.190 18m 48s 3.63 GB
horse 24.31 0.847 0.139 18m 53s 3.88 GB
lighthouse 20.85 0.768 0.245 19m 33s 3.89 GB
m60 26.54 0.843 0.179 19m 47s 3.64 GB
panther 27.57 0.858 0.174 20m 10s 3.89 GB
playground 24.69 0.755 0.249 19m 33s 3.64 GB
train 20.43 0.693 0.261 19m 42s 3.88 GB
barn 26.40 0.794 0.215 19m 16s 3.63 GB
caterpillar 21.71 0.666 0.302 19m 52s 3.63 GB
church 20.06 0.671 0.338 19m 29s 3.63 GB
courthouse 18.11 0.632 0.465 19m 37s 3.63 GB
ignatius 20.44 0.689 0.251 19m 5s 3.63 GB
meetingroom 23.21 0.793 0.261 19m 3s 3.63 GB
truck 23.37 0.797 0.167 19m 28s 3.63 GB
Gaussian Opacity Fields 22.39 0.825 0.172 41m 21s 24.07 GB
auditorium 23.20 0.871 0.194 35m 17s 8.44 GB
ballroom 22.84 0.818 0.107 1h 0m 31s 38.17 GB
courtroom 21.15 0.781 0.168 47m 1s 31.55 GB
museum 19.92 0.761 0.152 50m 33s 34.90 GB
palace 16.46 0.683 0.443 35m 6s 10.30 GB
temple 20.29 0.794 0.234 34m 55s 7.02 GB
family 22.31 0.875 0.084 41m 32s 35.16 GB
francis 24.76 0.901 0.158 35m 52s 12.30 GB
horse 23.73 0.881 0.092 38m 29s 24.41 GB
lighthouse 21.80 0.833 0.181 40m 8s 7.85 GB
m60 28.04 0.906 0.104 40m 9s 26.18 GB
panther 28.47 0.910 0.102 40m 13s 30.99 GB
playground 23.89 0.869 0.142 45m 37s 37.62 GB
train 19.69 0.796 0.164 37m 11s 19.04 GB
barn 25.72 0.866 0.140 37m 4s 15.67 GB
caterpillar 21.78 0.791 0.187 36m 21s 23.78 GB
church 19.65 0.775 0.208 48m 2s 36.45 GB
courthouse 19.60 0.726 0.346 35m 12s 12.82 GB
ignatius 20.34 0.769 0.162 43m 50s 34.97 GB
meetingroom 24.31 0.862 0.140 40m 29s 23.30 GB
truck 22.33 0.860 0.099 44m 56s 34.46 GB
PGSR 23.21 0.832 0.146 18m 59s 10.44 GB
auditorium 23.65 0.868 0.198 16m 56s 8.54 GB
ballroom 23.69 0.817 0.099 22m 29s 10.59 GB
courtroom 21.50 0.783 0.156 24m 60s 11.56 GB
museum 20.59 0.759 0.151 31m 10s 14.12 GB
palace 19.45 0.736 0.291 14m 46s 11.51 GB
temple 21.28 0.813 0.172 16m 24s 8.49 GB
family 24.68 0.861 0.090 20m 33s 7.49 GB
francis 24.84 0.897 0.158 14m 58s 7.97 GB
horse 24.05 0.874 0.097 16m 59s 6.25 GB
lighthouse 21.64 0.843 0.137 15m 11s 8.53 GB
m60 27.61 0.905 0.098 17m 38s 9.14 GB
panther 28.34 0.908 0.102 16m 59s 9.24 GB
playground 24.56 0.873 0.122 20m 28s 10.22 GB
train 20.67 0.797 0.141 16m 53s 9.16 GB
barn 27.34 0.871 0.121 16m 7s 10.24 GB
caterpillar 22.14 0.790 0.173 17m 16s 10.48 GB
church 20.89 0.795 0.154 22m 38s 13.84 GB
courthouse 21.49 0.785 0.228 15m 8s 21.75 GB
ignatius 21.03 0.772 0.148 22m 40s 10.59 GB
meetingroom 23.82 0.857 0.132 17m 54s 10.39 GB
truck 24.13 0.857 0.096 20m 25s 9.21 GB
Gaussian Splatting 23.83 0.831 0.165 13m 48s 6.95 GB
auditorium 24.13 0.871 0.193 10m 54s 4.98 GB
ballroom 24.07 0.824 0.101 19m 55s 9.08 GB
courtroom 23.12 0.790 0.165 17m 12s 9.02 GB
museum 20.92 0.764 0.160 20m 55s 11.38 GB
palace 19.63 0.736 0.350 11m 21s 5.85 GB
temple 20.85 0.806 0.222 10m 56s 5.13 GB
family 24.43 0.865 0.095 14m 60s 6.31 GB
francis 27.22 0.897 0.169 10m 20s 4.28 GB
horse 23.82 0.875 0.103 11m 35s 4.50 GB
lighthouse 22.11 0.843 0.156 12m 3s 6.09 GB
m60 27.84 0.901 0.113 13m 18s 6.80 GB
panther 28.32 0.908 0.107 13m 19s 7.01 GB
playground 25.37 0.848 0.170 15m 33s 7.44 GB
train 21.67 0.791 0.171 11m 56s 5.39 GB
barn 27.51 0.852 0.160 11m 9s 6.22 GB
caterpillar 23.38 0.791 0.190 11m 32s 5.89 GB
church 22.79 0.811 0.177 17m 15s 8.36 GB
courthouse 22.22 0.779 0.266 11m 8s 10.34 GB
ignatius 21.53 0.776 0.153 16m 36s 8.82 GB
meetingroom 25.19 0.866 0.141 12m 39s 5.50 GB
truck 24.25 0.853 0.108 15m 18s 7.47 GB
Mip-Splatting 23.93 0.833 0.166 15m 56s 7.27 GB
auditorium 24.41 0.872 0.196 11m 47s 4.82 GB
ballroom 24.15 0.826 0.098 25m 4s 9.76 GB
courtroom 23.00 0.791 0.165 20m 46s 9.27 GB
museum 20.88 0.768 0.158 24m 55s 12.57 GB
palace 19.63 0.731 0.354 12m 53s 6.48 GB
temple 20.55 0.805 0.226 12m 3s 5.93 GB
family 24.55 0.872 0.095 15m 44s 6.99 GB
francis 27.61 0.899 0.172 11m 47s 4.40 GB
horse 23.94 0.879 0.104 12m 47s 4.58 GB
lighthouse 22.25 0.844 0.159 13m 47s 7.01 GB
m60 27.98 0.904 0.112 15m 13s 7.13 GB
panther 28.27 0.908 0.109 15m 43s 7.38 GB
playground 25.87 0.861 0.155 18m 26s 8.30 GB
train 21.82 0.795 0.172 13m 26s 5.65 GB
barn 27.75 0.855 0.161 12m 42s 6.12 GB
caterpillar 23.42 0.790 0.197 13m 16s 5.69 GB
church 22.76 0.812 0.176 19m 53s 8.37 GB
courthouse 22.15 0.779 0.265 13m 57s 10.28 GB
ignatius 21.73 0.780 0.159 18m 44s 8.46 GB
meetingroom 25.46 0.870 0.137 14m 13s 5.73 GB
truck 24.36 0.857 0.108 17m 32s 7.68 GB
Zip-NeRF 24.63 0.840 0.131 5h 44m 9s 26.61 GB
auditorium 24.52 0.877 0.153 5h 24m 16s 26.61 GB
ballroom 25.45 0.835 0.113 5h 25m 1s 26.61 GB
courtroom 22.17 0.790 0.153 5h 17m 54s 26.61 GB
museum 19.34 0.746 0.159 5h 24m 17s 26.61 GB
palace 19.11 0.718 0.317 6h 42m 11s 26.61 GB
temple 20.58 0.805 0.183 5h 37m 11s 26.61 GB
family 27.10 0.889 0.067 5h 32m 15s 26.61 GB
francis 29.10 0.915 0.106 5h 34m 11s 26.61 GB
horse 26.82 0.897 0.069 5h 55m 60s 26.61 GB
lighthouse 23.07 0.849 0.131 5h 52m 55s 26.63 GB
m60 29.01 0.912 0.076 6h 3m 19s 26.63 GB
panther 28.76 0.909 0.081 5h 36m 45s 26.63 GB
playground 27.13 0.880 0.095 5h 29m 4s 26.61 GB
train 22.19 0.814 0.119 6h 14m 13s 26.61 GB
barn 29.26 0.884 0.083 5h 28m 43s 26.61 GB
caterpillar 23.94 0.802 0.152 5h 53m 16s 26.61 GB
church 23.14 0.807 0.153 6h 4m 32s 26.61 GB
courthouse 22.88 0.780 0.218 6h 4m 6s 26.61 GB
ignatius 22.61 0.789 0.127 5h 48m 32s 26.61 GB
meetingroom 25.93 0.875 0.121 5h 21m 9s 26.61 GB
truck 25.09 0.864 0.081 5h 37m 11s 26.61 GB

LLFF 

LLFF is a dataset of forward-facing scenes with a small variation in camera pose. NeRF methods usually use NDC-space parametrization for the scene representation.

Method PSNR SSIM LPIPS (VGG) Time GPU mem.
Mip-NeRF 360 26.59 0.846 0.168 9h 55m 23s 104.98 GB
Fern 24.59 0.820 0.210 7h 29m 8s 129.06 GB
Flower 27.56 0.867 0.140 14h 2m 26s 64.85 GB
Fortress 31.34 0.900 0.117 14h 2m 6s 64.85 GB
Horns 28.51 0.909 0.124 7h 26m 41s 129.06 GB
Leaves 19.84 0.721 0.231 7h 26m 52s 129.06 GB
Orchids 19.51 0.660 0.246 7h 27m 12s 129.06 GB
Room 33.49 0.965 0.115 7h 26m 22s 129.06 GB
Trex 27.86 0.927 0.158 14h 2m 15s 64.85 GB
TensoRF 26.68
Paper's PSNR: 26.73
0.834
Paper's SSIM: 0.839
0.202
Paper's LPIPS (VGG): 0.204
31m 32s 20.38 GB
Fern 25.08
Paper's PSNR: 25.27
0.801
Paper's SSIM: 0.814
0.246
Paper's LPIPS (VGG): 0.237
36m 6s 18.28 GB
Flower 28.38
Paper's PSNR: 28.6
0.860
Paper's SSIM: 0.871
0.171
Paper's LPIPS (VGG): 0.169
33m 33s 23.74 GB
Fortress 31.48
Paper's PSNR: 31.36
0.898
Paper's SSIM: 0.897
0.141
Paper's LPIPS (VGG): 0.148
32m 41s 18.39 GB
Horns 28.33
Paper's PSNR: 28.14
0.882
Paper's SSIM: 0.877
0.180
Paper's LPIPS (VGG): 0.196
33m 26s 18.27 GB
Leaves 20.96
Paper's PSNR: 21.3
0.730
Paper's SSIM: 0.752
0.239
Paper's LPIPS (VGG): 0.217
28m 21s 20.11 GB
Orchids 19.85
Paper's PSNR: 19.87
0.644
Paper's SSIM: 0.649
0.279
Paper's LPIPS (VGG): 0.278
32m 3s 20.69 GB
Room 31.86
Paper's PSNR: 32.35
0.950
Paper's SSIM: 0.952
0.159
Paper's LPIPS (VGG): 0.167
27m 20s 16.38 GB
Trex 27.53
Paper's PSNR: 26.97
0.909
Paper's SSIM: 0.9
0.199
Paper's LPIPS (VGG): 0.221
28m 46s 27.20 GB

Mip-NeRF 360 Sparse 

Modified Mip-NeRF 360 dataset with small train set (12 or 24) views. The dataset is used to evaluate sparse-view NVS methods.

Method PSNR SSIM LPIPS (VGG) Time GPU mem.
Zip-NeRF 16.47 0.428 0.535 5h 27m 11s 29.82 GB
garden n12 19.69 0.521 0.372 5h 22m 55s 29.28 GB
bicycle n12 14.23 0.183 0.644 5h 18m 11s 29.08 GB
flowers n12 12.11 0.133 0.692 5h 54m 33s 29.00 GB
treehill n12 15.73 0.307 0.554 5h 31m 58s 29.19 GB
stump n12 17.11 0.239 0.623 5h 15m 30s 29.10 GB
kitchen n12 13.47 0.397 0.635 5h 56m 43s 30.69 GB
bonsai n12 14.37 0.474 0.623 5h 11m 15s 30.69 GB
counter n12 12.03 0.379 0.685 5h 23m 6s 30.69 GB
room n12 18.91 0.711 0.430 5h 56m 45s 30.69 GB
garden n24 24.47 0.754 0.209 5h 15m 40s 29.28 GB
bicycle n24 14.53 0.209 0.610 5h 14m 20s 29.08 GB
flowers n24 13.77 0.239 0.593 5h 14m 16s 29.00 GB
treehill n24 17.73 0.360 0.480 5h 41m 51s 29.19 GB
stump n24 17.29 0.239 0.607 5h 13m 15s 29.10 GB
kitchen n24 14.56 0.516 0.541 5h 22m 22s 30.69 GB
bonsai n24 17.48 0.718 0.425 5h 24m 17s 30.69 GB
counter n24 14.61 0.517 0.584 5h 27m 33s 30.69 GB
room n24 24.28 0.816 0.332 5h 24m 56s 30.69 GB
PGSR 17.20 0.491 0.468 26m 27s 6.95 GB
garden n12 18.01 0.499 0.370 23m 17s 5.90 GB
bicycle n12 15.58 0.242 0.562 22m 23s 5.35 GB
flowers n12 12.59 0.169 0.628 21m 43s 5.43 GB
treehill n12 13.78 0.277 0.578 23m 55s 5.70 GB
stump n12 17.22 0.266 0.597 18m 8s 5.54 GB
kitchen n12 16.64 0.607 0.429 28m 30s 7.36 GB
bonsai n12 13.65 0.536 0.577 24m 49s 7.36 GB
counter n12 15.14 0.544 0.511 26m 9s 7.05 GB
room n12 15.64 0.585 0.524 26m 7s 7.26 GB
garden n24 21.86 0.725 0.231 25m 21s 6.25 GB
bicycle n24 18.68 0.385 0.447 32m 11s 8.89 GB
flowers n24 15.23 0.281 0.497 29m 9s 8.07 GB
treehill n24 17.21 0.382 0.469 30m 26s 8.73 GB
stump n24 19.51 0.366 0.501 25m 3s 6.44 GB
kitchen n24 21.65 0.792 0.269 31m 58s 7.40 GB
bonsai n24 18.84 0.751 0.407 28m 8s 7.33 GB
counter n24 19.03 0.711 0.403 29m 47s 7.71 GB
room n24 19.31 0.718 0.416 28m 57s 7.23 GB
SparseGS 19.65 0.573 0.438 42m 5s 11.56 GB
garden n12 19.26 0.553 0.367 32m 50s 10.64 GB
bicycle n12 17.10 0.342 0.495 31m 41s 10.59 GB
flowers n12 14.08 0.241 0.582 32m 51s 10.45 GB
treehill n12 15.46 0.344 0.577 32m 57s 10.96 GB
stump n12 18.15 0.343 0.542 28m 11s 10.01 GB
kitchen n12 20.80 0.739 0.338 47m 38s 11.64 GB
bonsai n12 18.23 0.685 0.443 42m 58s 11.42 GB
counter n12 18.05 0.643 0.465 46m 29s 11.35 GB
room n12 20.40 0.733 0.443 45m 35s 11.41 GB
garden n24 23.74 0.744 0.243 41m 2s 12.36 GB
bicycle n24 19.80 0.479 0.424 39m 57s 12.54 GB
flowers n24 16.39 0.344 0.530 39m 35s 11.52 GB
treehill n24 18.81 0.444 0.529 38m 29s 11.79 GB
stump n24 20.03 0.443 0.481 36m 19s 12.22 GB
kitchen n24 24.20 0.839 0.258 57m 15s 12.31 GB
bonsai n24 22.76 0.818 0.381 52m 36s 12.52 GB
counter n24 22.64 0.778 0.382 55m 36s 12.13 GB
room n24 23.71 0.800 0.400 55m 29s 12.23 GB
Mip-Splatting 19.75 0.572 0.412 20m 19s 6.83 GB
garden n12 19.42 0.548 0.334 19m 10s 6.03 GB
bicycle n12 17.19 0.342 0.488 22m 26s 7.28 GB
flowers n12 14.02 0.237 0.559 18m 2s 6.21 GB
treehill n12 15.06 0.334 0.561 20m 8s 7.04 GB
stump n12 18.38 0.328 0.538 14m 28s 5.41 GB
kitchen n12 21.33 0.741 0.320 21m 49s 6.56 GB
bonsai n12 19.13 0.717 0.420 16m 4s 5.90 GB
counter n12 18.49 0.649 0.445 20m 45s 6.04 GB
room n12 20.42 0.722 0.425 21m 26s 6.41 GB
garden n24 23.80 0.748 0.199 22m 31s 7.53 GB
bicycle n24 19.82 0.475 0.395 26m 8s 9.93 GB
flowers n24 16.41 0.350 0.491 19m 38s 7.04 GB
treehill n24 17.99 0.427 0.482 19m 55s 7.65 GB
stump n24 19.76 0.413 0.479 17m 58s 7.17 GB
kitchen n24 24.22 0.840 0.231 23m 46s 6.84 GB
bonsai n24 23.00 0.836 0.338 17m 45s 6.46 GB
counter n24 23.06 0.786 0.346 20m 42s 6.30 GB
room n24 24.01 0.807 0.367 22m 57s 7.15 GB
Gaussian Splatting 19.76 0.568 0.412 19m 46s 7.16 GB
garden n12 19.51 0.541 0.335 19m 56s 6.72 GB
bicycle n12 17.21 0.334 0.484 20m 48s 7.21 GB
flowers n12 14.08 0.231 0.552 17m 53s 6.72 GB
treehill n12 15.08 0.327 0.557 20m 29s 7.75 GB
stump n12 18.36 0.324 0.535 13m 35s 5.28 GB
kitchen n12 21.38 0.738 0.324 21m 21s 6.37 GB
bonsai n12 18.97 0.710 0.423 15m 40s 6.11 GB
counter n12 18.43 0.647 0.445 19m 51s 5.97 GB
room n12 20.60 0.724 0.425 20m 38s 6.37 GB
garden n24 23.73 0.743 0.200 22m 21s 8.65 GB
bicycle n24 19.77 0.468 0.396 24m 49s 11.22 GB
flowers n24 16.49 0.345 0.489 19m 14s 7.90 GB
treehill n24 17.99 0.425 0.475 19m 54s 8.42 GB
stump n24 19.78 0.407 0.478 17m 1s 7.32 GB
kitchen n24 24.33 0.838 0.232 23m 11s 6.82 GB
bonsai n24 22.91 0.835 0.340 17m 1s 6.35 GB
counter n24 23.00 0.785 0.346 20m 2s 6.42 GB
room n24 24.02 0.803 0.370 22m 4s 7.35 GB
3DGS-MCMC 20.07 0.578 0.407 35m 60s 23.05 GB
garden n12 19.99 0.582 0.311 50m 18s 33.04 GB
bicycle n12 17.84 0.359 0.482 47m 16s 34.66 GB
flowers n12 14.49 0.236 0.542 34m 16s 38.85 GB
treehill n12 15.33 0.340 0.533 35m 24s 32.81 GB
stump n12 19.23 0.370 0.529 38m 47s 31.93 GB
kitchen n12 21.47 0.741 0.317 39m 17s 15.24 GB
bonsai n12 19.09 0.709 0.405 31m 33s 9.73 GB
counter n12 18.44 0.634 0.455 32m 53s 9.17 GB
room n12 20.28 0.720 0.411 32m 59s 9.67 GB
garden n24 23.81 0.709 0.361 10m 23s 4.38 GB
bicycle n24 19.80 0.480 0.393 44m 57s 39.34 GB
flowers n24 16.56 0.341 0.461 35m 2s 38.31 GB
treehill n24 18.47 0.434 0.444 34m 28s 32.98 GB
stump n24 20.59 0.450 0.463 37m 55s 33.02 GB
kitchen n24 24.49 0.852 0.219 39m 44s 16.78 GB
bonsai n24 23.83 0.852 0.311 32m 46s 9.96 GB
counter n24 23.10 0.784 0.338 34m 9s 11.81 GB
room n24 24.43 0.811 0.345 35m 57s 13.17 GB
Scaffold-GS 20.28 0.585 0.410 20m 41s 5.59 GB
garden n12 19.68 0.576 0.326 17m 21s 4.65 GB
bicycle n12 18.02 0.363 0.493 16m 60s 4.80 GB
flowers n12 14.52 0.240 0.553 15m 33s 4.73 GB
treehill n12 15.84 0.324 0.552 18m 55s 5.05 GB
stump n12 18.96 0.346 0.538 14m 9s 4.61 GB
kitchen n12 21.91 0.764 0.317 29m 31s 6.12 GB
bonsai n12 19.76 0.741 0.413 21m 55s 6.10 GB
counter n12 18.98 0.661 0.456 23m 7s 5.98 GB
room n12 20.91 0.740 0.415 20m 4s 6.13 GB
garden n24 23.11 0.719 0.236 19m 23s 5.13 GB
bicycle n24 20.29 0.495 0.396 19m 40s 5.66 GB
flowers n24 16.69 0.347 0.493 17m 14s 4.98 GB
treehill n24 18.92 0.453 0.468 19m 26s 5.67 GB
stump n24 20.32 0.426 0.483 17m 35s 5.43 GB
kitchen n24 24.98 0.861 0.217 31m 36s 6.79 GB
bonsai n24 24.37 0.850 0.320 23m 56s 6.29 GB
counter n24 23.04 0.794 0.348 24m 17s 6.12 GB
room n24 24.76 0.821 0.352 21m 38s 6.30 GB
DropGaussian 21.45 0.616 0.458 5m 21s 4.84 GB
garden n12 20.74 0.579 0.421 4m 36s 4.21 GB
bicycle n12 19.46 0.425 0.523 4m 21s 3.97 GB
flowers n12 16.39 0.296 0.615 4m 25s 3.96 GB
treehill n12 17.39 0.413 0.610 4m 23s 4.21 GB
stump n12 19.84 0.406 0.571 4m 5s 3.88 GB
kitchen n12 22.83 0.792 0.306 6m 60s 5.49 GB
bonsai n12 21.23 0.786 0.404 6m 14s 5.60 GB
counter n12 20.07 0.725 0.425 6m 25s 5.53 GB
room n12 22.78 0.797 0.422 6m 31s 5.49 GB
garden n24 24.26 0.694 0.369 4m 35s 4.46 GB
bicycle n24 21.27 0.496 0.496 4m 29s 4.42 GB
flowers n24 17.99 0.355 0.588 4m 35s 4.21 GB
treehill n24 20.31 0.487 0.576 4m 22s 4.33 GB
stump n24 21.09 0.466 0.556 4m 16s 4.21 GB
kitchen n24 25.85 0.863 0.248 6m 49s 5.74 GB
bonsai n24 25.12 0.863 0.351 6m 15s 5.78 GB
counter n24 24.27 0.817 0.360 6m 22s 5.82 GB
room n24 25.25 0.832 0.399 6m 37s 5.74 GB

Photo Tourism 

Photo Tourism is a dataset of images of famous landmarks, such as the Sacre Coeur, the Trevi Fountain, and the Brandenburg Gate. The images were captured by tourist at different times of the day and year, images have varying lighting conditions and occlusions. The evaluation protocol is based on NeRF-W, where the image appearance embeddings are optimized on the left side of the image and the metrics are computed on the right side of the image.

Method PSNR SSIM LPIPS Time GPU mem.
K-Planes 21.10 0.761 0.313 24m 37s 3.59 GB
Sacre Coeur 19.96 0.762 0.299 24m 26s 3.62 GB
Trevi Fountain 19.70 0.662 0.388 24m 44s 3.59 GB
Brandenburg Gate 23.65 0.859 0.253 24m 40s 3.56 GB
GS-W 21.38
Paper's PSNR: 24.70

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 0.817
Paper's SSIM: 0.865

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 0.213
Paper's LPIPS: 0.124

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 1h 13m 50s 21.93 GB
Sacre Coeur 19.73
Paper's PSNR: 23.24

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 0.824
Paper's SSIM: 0.8632

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 0.210
Paper's LPIPS: 0.13

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 1h 8m 56s 18.80 GB
Trevi Fountain 20.06
Paper's PSNR: 22.91

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 0.724
Paper's SSIM: 0.8014

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 0.272
Paper's LPIPS: 0.1563

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 1h 21m 34s 27.92 GB
Brandenburg Gate 24.35
Paper's PSNR: 27.96

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 0.903
Paper's SSIM: 0.9319

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 0.155
Paper's LPIPS: 0.0862

The original paper reports metrics for test images where the appearance embedding is estimated from the full test image, not just the left half as in the official evaluation protocol. The reported numbers are computed using the official evaluation protocol and are, therefore, lower than the numbers reported in the paper.

 1h 10m 60s 19.06 GB
NeRF-W (reimplementation) 21.75 0.790 0.268 44h 23m 46s 98.80 GB
Sacre Coeur 19.56 0.795 0.260 41h 41m 36s 98.80 GB
Trevi Fountain 21.48 0.693 0.331 49h 43m 22s 98.81 GB
Brandenburg Gate 24.22 0.884 0.213 41h 46m 21s 98.80 GB
Scaffold-GS 23.50 0.854 0.170 1h 27m 49s 18.34 GB
Sacre Coeur 21.85 0.871 0.157 1h 38m 45s 11.38 GB
Trevi Fountain 23.21 0.768 0.228 1h 21m 34s 21.84 GB
Brandenburg Gate 25.45 0.923 0.127 1h 23m 7s 21.80 GB
gsplat 23.66 0.857 0.162 1h 44m 24s 4.68 GB
Sacre Coeur 22.05 0.876 0.154 1h 30m 15s 3.33 GB
Trevi Fountain 22.56 0.765 0.213 2h 11m 60s 7.05 GB
Brandenburg Gate 26.36 0.931 0.118 1h 30m 56s 3.68 GB
WildGaussians 24.65 0.851 0.179 10h 18m 16s 18.24 GB
Sacre Coeur 22.56 0.859 0.177 8h 41m 57s 8.66 GB
Trevi Fountain 23.63 0.766 0.228 13h 14m 51s 38.25 GB
Brandenburg Gate 27.76 0.927 0.133 8h 57m 60s 7.83 GB

Hierarchical 3DGS 

Hierarchical 3DGS is a dataset released with H3DGS paper. We implement the two public single-chunks scenes (SmallCity, Campus) used for evaluation. To collect the dataset, authors used a bicycle helmet on which they mounted 6 GoPro HERO6 Black cameras (5 for the Campus scene). They collected SmallCity and BigCity captures on a bicycle, riding at around 6–7km/h, while Campus was captured on foot wearing the helmet. Poses were estimated using COLMAP with custom parameters and hierarchical mapper. Additinal per-chunk bundle adjustment was performed. It is recommended to use exposure modeling with this dataset

Method PSNR SSIM LPIPS (VGG) Time GPU mem.
H3DGS 25.53
Paper's PSNR: 25.39

Results in the paper were evaluated using different tau (0, 3, 6, 15), where tau=0 is the slowest, but highest quality. We choose tau=6 - consistent with most experiments in the paper, providing a good trade-off between quality and speed.

 0.803
Paper's SSIM: 0.806

Results in the paper were evaluated using different tau (0, 3, 6, 15), where tau=0 is the slowest, but highest quality. We choose tau=6 - consistent with most experiments in the paper, providing a good trade-off between quality and speed.

 0.364 1h 23m 4s 14.94 GB
SmallCity 26.42
Paper's PSNR: 26.29

Results in the paper were evaluated using different tau (0, 3, 6, 15), where tau=0 is the slowest, but highest quality. We choose tau=6 - consistent with most experiments in the paper, providing a good trade-off between quality and speed.

 0.807
Paper's SSIM: 0.81

Results in the paper were evaluated using different tau (0, 3, 6, 15), where tau=0 is the slowest, but highest quality. We choose tau=6 - consistent with most experiments in the paper, providing a good trade-off between quality and speed.

 0.331 1h 16m 1s 15.85 GB
Campus 24.63
Paper's PSNR: 24.5

Results in the paper were evaluated using different tau (0, 3, 6, 15), where tau=0 is the slowest, but highest quality. We choose tau=6 - consistent with most experiments in the paper, providing a good trade-off between quality and speed.

 0.798
Paper's SSIM: 0.801

Results in the paper were evaluated using different tau (0, 3, 6, 15), where tau=0 is the slowest, but highest quality. We choose tau=6 - consistent with most experiments in the paper, providing a good trade-off between quality and speed.

 0.396 1h 30m 7s 14.04 GB

SeaThru-NeRF 

SeaThru-NeRF dataset contains four underwater forward-facing scenes.

Method PSNR SSIM LPIPS Time GPU mem.
Gaussian Splatting 21.31 0.729 0.300 20m 9s 6.49 GB
Curasao 24.15 0.738 0.318 22m 26s 9.30 GB
Panama 23.68 0.749 0.250 22m 41s 8.43 GB
IUI3 18.86 0.677 0.390 18m 36s 4.26 GB
Japanese Gradens 18.54 0.752 0.242 16m 55s 3.96 GB
Mip-Splatting 21.33 0.730 0.304 20m 53s 6.19 GB
Curasao 24.41 0.739 0.316 23m 38s 9.54 GB
Panama 23.99 0.752 0.237 22m 42s 6.85 GB
IUI3 19.18 0.683 0.412 19m 32s 4.26 GB
Japanese Gradens 17.75 0.747 0.253 17m 43s 4.10 GB
SeaThru-NeRF 26.37 0.815 0.245 2h 49m 9s 127.55 GB
Curasao 30.00
Paper's PSNR: 30.48
0.870
Paper's SSIM: 0.87
0.215
Paper's LPIPS: 0.2
2h 48m 47s 127.66 GB
Panama 27.82
Paper's PSNR: 27.89
0.834
Paper's SSIM: 0.83
0.226
Paper's LPIPS: 0.22
2h 54m 57s 127.66 GB
IUI3 25.92 0.787 0.294 2h 46m 50s 127.44 GB
Japanese Gradens 21.73
Paper's PSNR: 21.83
0.768
Paper's SSIM: 0.77
0.246
Paper's LPIPS: 0.25
2h 46m 1s 127.44 GB

Zip-NeRF 

ZipNeRF is a dataset with four large scenes: Berlin, Alameda, London, and NYC, (1000-2000 photos each) captured using fisheye cameras. This implementation uses undistorted images which are provided with the dataset and the downsampled resolutions are between 1392 × 793 and 2000 × 1140 depending on scene. It is recommended to use exposure modeling with this dataset if available.

Method PSNR SSIM LPIPS (VGG) Time GPU mem.
Octree-GS 22.33 0.762 0.473 44m 56s 27.63 GB
Alameda 22.79 0.730 0.448 44m 25s 28.68 GB
Berlin 13.64 0.669 0.640 1h 6m 40s 34.85 GB
London 25.76 0.807 0.433 33m 44s 27.35 GB
NYC 27.13 0.841 0.372 34m 55s 19.63 GB

Citation

If you use this code in your research, please cite the following paper:

@inproceedings{kulhanek2025nerfbaselines,
  title={\{N}erf{B}aselines: Consistent and Reproducible Evaluation of Novel View Synthesis Methods},
  author={Jonas Kulhanek and Torsten Sattler},
  year={2025},
  booktitle={Proceedings of the 39th International Conference on Neural Information Processing Systems (NeurIPS 2025)},
}

Acknowledgements

We want to thank Brent Yi and the NerfStudio Team for helpful discussions regarding the NerfStudio codebase. This work was supported by the Czech Science Foundation (GAČR) EXPRO (grant no. 23-07973X), the Grant Agency of the Czech Technical University in Prague (grant no. SGS24/095/OHK3/2T/13), and by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90254).

License

The NerfBaselines project is licensed under the MIT license. Each implemented method is licensed under the license provided by the authors of the method. For the currently implemented methods, the following licenses apply: