Blender

Blender (nerf-synthetic) is a synthetic dataset used to benchmark NeRF methods. It consists of 8 scenes of an object placed on a white background. Cameras are placed on a semi-sphere around the object. Scenes are licensed under various CC licenses.

Web: https://www.matthewtancik.com/nerf
Paper: NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Authors: Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng
Method PSNR SSIM LPIPS (VGG) Time GPU mem.
COLMAP 12.12 0.766 0.214 1h 20m 34s 0.00 MB
lego 12.31 0.776 0.220 2h 5m 25s 0.00 MB
drums 8.06 0.657 0.296 2h 14m 38s 0.00 MB
ficus 15.12 0.838 0.143 39m 23s 0.00 MB
hotdog 12.21 0.831 0.195 1h 13m 26s 0.00 MB
materials 14.56 0.802 0.193 1h 42m 37s 0.00 MB
mic 9.29 0.765 0.182 1h 13m 24s 0.00 MB
ship 10.02 0.616 0.335 48m 3s 0.00 MB
chair 15.42 0.847 0.148 47m 35s 0.00 MB
NeRF 28.72
Paper's PSNR: 31.00
0.936
Paper's SSIM: 0.947
0.092
Paper's LPIPS (VGG): 0.081
23h 26m 30s 10.25 GB
lego 32.86
Paper's PSNR: 32.54
0.964
Paper's SSIM: 0.961
0.049
Paper's LPIPS (VGG): 0.05
23h 32m 41s 10.25 GB
drums 25.12
Paper's PSNR: 25.01
0.926
Paper's SSIM: 0.925
0.087
Paper's LPIPS (VGG): 0.091
22h 56m 29s 10.25 GB
ficus 30.31
Paper's PSNR: 30.13
0.964
Paper's SSIM: 0.964
0.042
Paper's LPIPS (VGG): 0.044
23h 38m 35s 10.25 GB
hotdog 36.39
Paper's PSNR: 36.18
0.975
Paper's SSIM: 0.974
0.095
Paper's LPIPS (VGG): 0.121
23h 14m 14s 10.25 GB
materials 29.84
Paper's PSNR: 29.62
0.950
Paper's SSIM: 0.949
0.062
Paper's LPIPS (VGG): 0.063
23h 58m 42s 10.25 GB
mic 13.03
Paper's PSNR: 32.91
0.881
Paper's SSIM: 0.98
0.154
Paper's LPIPS (VGG): 0.028
23h 21m 46s 10.25 GB
ship 28.80
Paper's PSNR: 28.65
0.859
Paper's SSIM: 0.856
0.210
Paper's LPIPS (VGG): 0.206
23h 30m 51s 10.25 GB
chair 33.43
Paper's PSNR: 33.0
0.970
Paper's SSIM: 0.967
0.041
Paper's LPIPS (VGG): 0.046
23h 18m 37s 10.25 GB
NerfStudio 29.19 0.941 0.095 9m 38s 3.65 GB
lego 31.37 0.967 0.069 9m 44s 3.65 GB
drums 22.48 0.897 0.139 9m 12s 3.65 GB
ficus 27.82 0.957 0.087 9m 22s 3.65 GB
hotdog 31.09 0.963 0.104 9m 44s 3.65 GB
materials 25.38 0.903 0.121 9m 23s 3.65 GB
mic 33.74 0.984 0.029 9m 47s 3.65 GB
ship 28.71 0.881 0.166 10m 2s 3.64 GB
chair 32.94 0.977 0.044 9m 48s 3.65 GB
Mip-NeRF 360 30.34 0.951 0.060 3h 29m 39s 114.80 GB
lego 33.20 0.975 0.028 2h 43m 18s 126.91 GB
drums 24.36 0.923 0.083 8h 51m 32s 30.02 GB
ficus 26.66 0.952 0.048 2h 43m 27s 126.91 GB
hotdog 36.44 0.979 0.039 2h 43m 11s 126.91 GB
materials 27.91 0.944 0.067 2h 43m 22s 126.91 GB
mic 31.50 0.984 0.021 2h 43m 9s 126.91 GB
ship 28.66 0.875 0.164 2h 45m 55s 126.91 GB
chair 34.01 0.977 0.032 2h 43m 17s 126.91 GB
gsplat 31.47 0.966 0.054 14m 45s 2.80 GB
lego 32.13 0.977 0.043 13m 31s 2.94 GB
drums 24.50 0.948 0.076 11m 15s 2.74 GB
ficus 34.44 0.986 0.014 11m 48s 2.75 GB
hotdog 35.39 0.983 0.041 20m 32s 2.67 GB
materials 29.93 0.959 0.044 11m 37s 2.76 GB
mic 33.73 0.991 0.023 11m 30s 2.84 GB
ship 28.77 0.899 0.155 17m 56s 2.86 GB
chair 32.88 0.986 0.034 19m 53s 2.82 GB
Tetra-NeRF 31.95
Paper's PSNR: 32.52
0.957
Paper's SSIM: 0.982
0.056 6h 53m 20s 29.57 GB
lego 33.93
Paper's PSNR: 34.75
0.972
Paper's SSIM: 0.987
0.036 7h 8m 12s 25.21 GB
drums 24.99
Paper's PSNR: 25.01
0.927
Paper's SSIM: 0.947
0.087 8h 34m 5s 36.26 GB
ficus 32.37
Paper's PSNR: 33.31
0.977
Paper's SSIM: 0.989
0.032 4h 19m 24s 21.84 GB
hotdog 35.80
Paper's PSNR: 36.16
0.978
Paper's SSIM: 0.989
0.040 6h 41m 21s 23.06 GB
materials 28.75
Paper's PSNR: 29.3
0.941
Paper's SSIM: 0.968
0.076 6h 23m 31s 37.98 GB
mic 34.54
Paper's PSNR: 35.49
0.987
Paper's SSIM: 0.993
0.022 6h 51m 58s 31.04 GB
ship 31.06
Paper's PSNR: 31.13
0.896
Paper's SSIM: 0.994
0.129 9h 4m 53s 27.19 GB
chair 34.17
Paper's PSNR: 35.05
0.977
Paper's SSIM: 0.99
0.029 6h 3m 14s 34.01 GB
Instant NGP 32.20
Paper's PSNR: 33.18

Instant-NGP trained and evaluated on black background instead of white.

 0.959 0.055 2m 23s 2.57 GB
lego 35.65
Paper's PSNR: 36.39

Instant-NGP trained and evaluated on black background instead of white.

 0.981 0.020 2m 41s 2.59 GB
drums 24.57
Paper's PSNR: 26.02

Instant-NGP trained and evaluated on black background instead of white.

 0.930 0.109 2m 6s 2.58 GB
ficus 30.29
Paper's PSNR: 33.51

Instant-NGP trained and evaluated on black background instead of white.

 0.972 0.031 1m 58s 2.58 GB
hotdog 37.02
Paper's PSNR: 37.4

Instant-NGP trained and evaluated on black background instead of white.

 0.982 0.037 2m 33s 2.58 GB
materials 28.96
Paper's PSNR: 29.78

Instant-NGP trained and evaluated on black background instead of white.

 0.944 0.069 2m 11s 2.58 GB
mic 35.41
Paper's PSNR: 36.22

Instant-NGP trained and evaluated on black background instead of white.

 0.989 0.016 2m 25s 2.47 GB
ship 30.61
Paper's PSNR: 31.1

Instant-NGP trained and evaluated on black background instead of white.

 0.892 0.136 2m 24s 2.58 GB
chair 35.07
Paper's PSNR: 35.0

Instant-NGP trained and evaluated on black background instead of white.

 0.984 0.023 2m 49s 2.57 GB
K-Planes 32.27 0.961 0.062 23m 58s 4.64 GB
lego 35.73 0.981 0.047 23m 54s 4.64 GB
drums 25.68 0.938 0.096 24m 10s 4.64 GB
ficus 31.31 0.974 0.052 23m 37s 4.64 GB
hotdog 36.46 0.981 0.033 23m 38s 4.64 GB
materials 29.41 0.949 0.070 24m 1s 4.64 GB
mic 33.95 0.988 0.019 24m 6s 4.64 GB
ship 30.72 0.897 0.141 24m 8s 4.64 GB
chair 34.85 0.983 0.036 24m 9s 4.64 GB
Scaffold-GS 33.08
Paper's PSNR: 33.68
0.966 0.048 7m 4s 3.72 GB
lego 34.96
Paper's PSNR: 35.69
0.980 0.024 6m 40s 3.72 GB
drums 26.28
Paper's PSNR: 26.44
0.949 0.054 6m 46s 3.73 GB
ficus 34.38
Paper's PSNR: 35.21
0.986 0.015 6m 9s 3.69 GB
hotdog 37.62
Paper's PSNR: 37.73
0.983 0.075 8m 26s 3.71 GB
materials 30.29
Paper's PSNR: 30.65
0.962 0.045 6m 49s 3.73 GB
mic 35.99
Paper's PSNR: 37.25
0.991 0.010 6m 31s 3.71 GB
ship 29.97
Paper's PSNR: 31.17
0.895 0.139 8m 30s 3.74 GB
chair 35.16
Paper's PSNR: 35.28
0.984 0.019 6m 38s 3.71 GB
TensoRF 33.17
Paper's PSNR: 33.14
0.963
Paper's SSIM: 0.963
0.051
Paper's LPIPS (VGG): 0.047
10m 47s 16.37 GB
lego 36.49
Paper's PSNR: 36.46
0.983
Paper's SSIM: 0.983
0.022
Paper's LPIPS (VGG): 0.018
10m 1s 20.53 GB
drums 26.01
Paper's PSNR: 26.01
0.936
Paper's SSIM: 0.937
0.076
Paper's LPIPS (VGG): 0.073
8m 56s 19.67 GB
ficus 34.06
Paper's PSNR: 33.99
0.982
Paper's SSIM: 0.982
0.029
Paper's LPIPS (VGG): 0.022
10m 45s 7.28 GB
hotdog 37.49
Paper's PSNR: 37.41
0.982
Paper's SSIM: 0.982
0.033
Paper's LPIPS (VGG): 0.032
11m 18s 19.77 GB
materials 30.08
Paper's PSNR: 30.12
0.952
Paper's SSIM: 0.952
0.059
Paper's LPIPS (VGG): 0.058
14m 55s 21.09 GB
mic 34.85
Paper's PSNR: 34.61
0.988
Paper's SSIM: 0.988
0.021
Paper's LPIPS (VGG): 0.015
8m 8s 11.21 GB
ship 30.69
Paper's PSNR: 30.77
0.894
Paper's SSIM: 0.895
0.141
Paper's LPIPS (VGG): 0.138
13m 19s 18.80 GB
chair 35.72
Paper's PSNR: 35.76
0.984
Paper's SSIM: 0.985
0.027
Paper's LPIPS (VGG): 0.022
8m 56s 12.60 GB
Gaussian Splatting 33.31
Paper's PSNR: 33.31
0.969 0.037 6m 6s 3.08 GB
lego 35.70
Paper's PSNR: 35.78
0.982 0.019 5m 59s 3.07 GB
drums 26.15
Paper's PSNR: 26.15
0.953 0.044 6m 3s 3.07 GB
ficus 34.79
Paper's PSNR: 34.87
0.987 0.013 5m 22s 2.82 GB
hotdog 37.64
Paper's PSNR: 37.72
0.985 0.026 5m 49s 2.85 GB
materials 30.01
Paper's PSNR: 30.0
0.959 0.043 5m 28s 2.91 GB
mic 35.49
Paper's PSNR: 35.36
0.991 0.008 6m 11s 3.02 GB
ship 30.85
Paper's PSNR: 30.8
0.904 0.130 8m 13s 3.99 GB
chair 35.84
Paper's PSNR: 35.83
0.987 0.015 5m 45s 2.89 GB
Mip-Splatting 33.33 0.969 0.039 6m 49s 2.65 GB
lego 35.45 0.982 0.021 6m 30s 2.51 GB
drums 26.14 0.953 0.046 6m 36s 2.53 GB
ficus 35.12 0.988 0.013 5m 8s 2.43 GB
hotdog 37.78 0.985 0.028 6m 44s 2.58 GB
materials 30.12 0.960 0.044 5m 49s 2.46 GB
mic 35.55 0.991 0.008 8m 8s 2.75 GB
ship 30.78 0.904 0.132 9m 19s 3.54 GB
chair 35.70 0.986 0.018 6m 16s 2.40 GB
Gaussian Opacity Fields 33.45 0.969 0.038 18m 26s 3.15 GB
lego 35.56 0.982 0.021 19m 4s 2.88 GB
drums 26.17 0.955 0.045 17m 50s 2.84 GB
ficus 35.19 0.988 0.013 11m 10s 2.71 GB
hotdog 37.46 0.985 0.028 17m 52s 2.61 GB
materials 30.20 0.961 0.043 14m 48s 2.72 GB
mic 36.06 0.992 0.008 21m 56s 2.99 GB
ship 30.68 0.901 0.131 29m 60s 5.64 GB
chair 36.28 0.988 0.017 14m 46s 2.79 GB
Zip-NeRF 33.67
Paper's PSNR: 33.09
0.973
Paper's SSIM: 0.971
0.036
Paper's LPIPS (VGG): 0.031
5h 21m 57s 26.20 GB
lego 35.81
Paper's PSNR: 34.84
0.983
Paper's SSIM: 0.98
0.019
Paper's LPIPS (VGG): 0.019
5h 20m 18s 26.20 GB
drums 25.90
Paper's PSNR: 25.84
0.948
Paper's SSIM: 0.944
0.054
Paper's LPIPS (VGG): 0.05
5h 20m 22s 26.20 GB
ficus 34.73
Paper's PSNR: 33.9
0.987
Paper's SSIM: 0.985
0.014
Paper's LPIPS (VGG): 0.015
5h 25m 57s 26.20 GB
hotdog 37.98
Paper's PSNR: 37.14
0.987
Paper's SSIM: 0.984
0.023
Paper's LPIPS (VGG): 0.02
5h 19m 2s 26.20 GB
materials 30.98
Paper's PSNR: 31.66
0.967
Paper's SSIM: 0.969
0.040
Paper's LPIPS (VGG): 0.032
5h 23m 48s 26.20 GB
mic 35.90
Paper's PSNR: 35.15
0.992
Paper's SSIM: 0.991
0.008
Paper's LPIPS (VGG): 0.007
5h 26m 19s 26.20 GB
ship 32.30
Paper's PSNR: 31.38
0.937
Paper's SSIM: 0.929
0.114
Paper's LPIPS (VGG): 0.091
5h 14m 50s 26.20 GB
chair 35.75
Paper's PSNR: 34.84
0.987
Paper's SSIM: 0.983
0.017
Paper's LPIPS (VGG): 0.017
5h 24m 56s 26.20 GB

PSNR

Peak Signal to Noise Ratio. The higher the better.

Method lego drums ficus hotdog materials mic ship chair
COLMAP 12.31 8.06 15.12 12.21 14.56 9.29 10.02 15.42
NeRF 32.86
Paper's PSNR: 32.54
25.12
Paper's PSNR: 25.01
30.31
Paper's PSNR: 30.13
36.39
Paper's PSNR: 36.18
29.84
Paper's PSNR: 29.62
13.03
Paper's PSNR: 32.91
28.80
Paper's PSNR: 28.65
33.43
Paper's PSNR: 33.0
NerfStudio 31.37 22.48 27.82 31.09 25.38 33.74 28.71 32.94
Mip-NeRF 360 33.20 24.36 26.66 36.44 27.91 31.50 28.66 34.01
gsplat 32.13 24.50 34.44 35.39 29.93 33.73 28.77 32.88
Tetra-NeRF 33.93
Paper's PSNR: 34.75
24.99
Paper's PSNR: 25.01
32.37
Paper's PSNR: 33.31
35.80
Paper's PSNR: 36.16
28.75
Paper's PSNR: 29.3
34.54
Paper's PSNR: 35.49
31.06
Paper's PSNR: 31.13
34.17
Paper's PSNR: 35.05
Instant NGP 35.65
Paper's PSNR: 36.39

Instant-NGP trained and evaluated on black background instead of white.

 24.57
Paper's PSNR: 26.02

Instant-NGP trained and evaluated on black background instead of white.

 30.29
Paper's PSNR: 33.51

Instant-NGP trained and evaluated on black background instead of white.

 37.02
Paper's PSNR: 37.4

Instant-NGP trained and evaluated on black background instead of white.

 28.96
Paper's PSNR: 29.78

Instant-NGP trained and evaluated on black background instead of white.

 35.41
Paper's PSNR: 36.22

Instant-NGP trained and evaluated on black background instead of white.

 30.61
Paper's PSNR: 31.1

Instant-NGP trained and evaluated on black background instead of white.

 35.07
Paper's PSNR: 35.0

Instant-NGP trained and evaluated on black background instead of white.
K-Planes 35.73 25.68 31.31 36.46 29.41 33.95 30.72 34.85
Scaffold-GS 34.96
Paper's PSNR: 35.69
26.28
Paper's PSNR: 26.44
34.38
Paper's PSNR: 35.21
37.62
Paper's PSNR: 37.73
30.29
Paper's PSNR: 30.65
35.99
Paper's PSNR: 37.25
29.97
Paper's PSNR: 31.17
35.16
Paper's PSNR: 35.28
TensoRF 36.49
Paper's PSNR: 36.46
26.01
Paper's PSNR: 26.01
34.06
Paper's PSNR: 33.99
37.49
Paper's PSNR: 37.41
30.08
Paper's PSNR: 30.12
34.85
Paper's PSNR: 34.61
30.69
Paper's PSNR: 30.77
35.72
Paper's PSNR: 35.76
Gaussian Splatting 35.70
Paper's PSNR: 35.78
26.15
Paper's PSNR: 26.15
34.79
Paper's PSNR: 34.87
37.64
Paper's PSNR: 37.72
30.01
Paper's PSNR: 30.0
35.49
Paper's PSNR: 35.36
30.85
Paper's PSNR: 30.8
35.84
Paper's PSNR: 35.83
Mip-Splatting 35.45 26.14 35.12 37.78 30.12 35.55 30.78 35.70
Gaussian Opacity Fields 35.56 26.17 35.19 37.46 30.20 36.06 30.68 36.28
Zip-NeRF 35.81
Paper's PSNR: 34.84
25.90
Paper's PSNR: 25.84
34.73
Paper's PSNR: 33.9
37.98
Paper's PSNR: 37.14
30.98
Paper's PSNR: 31.66
35.90
Paper's PSNR: 35.15
32.30
Paper's PSNR: 31.38
35.75
Paper's PSNR: 34.84

SSIM

Structural Similarity Index. The higher the better. The implementation matches JAX's SSIM and torchmetrics's SSIM (with default parameters).

Method lego drums ficus hotdog materials mic ship chair
COLMAP 0.776 0.657 0.838 0.831 0.802 0.765 0.616 0.847
NeRF 0.964
Paper's SSIM: 0.961
0.926
Paper's SSIM: 0.925
0.964
Paper's SSIM: 0.964
0.975
Paper's SSIM: 0.974
0.950
Paper's SSIM: 0.949
0.881
Paper's SSIM: 0.98
0.859
Paper's SSIM: 0.856
0.970
Paper's SSIM: 0.967
NerfStudio 0.967 0.897 0.957 0.963 0.903 0.984 0.881 0.977
Mip-NeRF 360 0.975 0.923 0.952 0.979 0.944 0.984 0.875 0.977
gsplat 0.977 0.948 0.986 0.983 0.959 0.991 0.899 0.986
Tetra-NeRF 0.972
Paper's SSIM: 0.987
0.927
Paper's SSIM: 0.947
0.977
Paper's SSIM: 0.989
0.978
Paper's SSIM: 0.989
0.941
Paper's SSIM: 0.968
0.987
Paper's SSIM: 0.993
0.896
Paper's SSIM: 0.994
0.977
Paper's SSIM: 0.99
Instant NGP 0.981 0.930 0.972 0.982 0.944 0.989 0.892 0.984
K-Planes 0.981 0.938 0.974 0.981 0.949 0.988 0.897 0.983
Scaffold-GS 0.980 0.949 0.986 0.983 0.962 0.991 0.895 0.984
TensoRF 0.983
Paper's SSIM: 0.983
0.936
Paper's SSIM: 0.937
0.982
Paper's SSIM: 0.982
0.982
Paper's SSIM: 0.982
0.952
Paper's SSIM: 0.952
0.988
Paper's SSIM: 0.988
0.894
Paper's SSIM: 0.895
0.984
Paper's SSIM: 0.985
Gaussian Splatting 0.982 0.953 0.987 0.985 0.959 0.991 0.904 0.987
Mip-Splatting 0.982 0.953 0.988 0.985 0.960 0.991 0.904 0.986
Gaussian Opacity Fields 0.982 0.955 0.988 0.985 0.961 0.992 0.901 0.988
Zip-NeRF 0.983
Paper's SSIM: 0.98
0.948
Paper's SSIM: 0.944
0.987
Paper's SSIM: 0.985
0.987
Paper's SSIM: 0.984
0.967
Paper's SSIM: 0.969
0.992
Paper's SSIM: 0.991
0.937
Paper's SSIM: 0.929
0.987
Paper's SSIM: 0.983

LPIPS (VGG)

Learned Perceptual Image Patch Similarity. The lower the better. The implementation uses VGG backbone and matches lpips pip package with checkpoint version 0.1

Method lego drums ficus hotdog materials mic ship chair
COLMAP 0.220 0.296 0.143 0.195 0.193 0.182 0.335 0.148
NeRF 0.049
Paper's LPIPS (VGG): 0.05
0.087
Paper's LPIPS (VGG): 0.091
0.042
Paper's LPIPS (VGG): 0.044
0.095
Paper's LPIPS (VGG): 0.121
0.062
Paper's LPIPS (VGG): 0.063
0.154
Paper's LPIPS (VGG): 0.028
0.210
Paper's LPIPS (VGG): 0.206
0.041
Paper's LPIPS (VGG): 0.046
NerfStudio 0.069 0.139 0.087 0.104 0.121 0.029 0.166 0.044
Mip-NeRF 360 0.028 0.083 0.048 0.039 0.067 0.021 0.164 0.032
gsplat 0.043 0.076 0.014 0.041 0.044 0.023 0.155 0.034
Tetra-NeRF 0.036 0.087 0.032 0.040 0.076 0.022 0.129 0.029
Instant NGP 0.020 0.109 0.031 0.037 0.069 0.016 0.136 0.023
K-Planes 0.047 0.096 0.052 0.033 0.070 0.019 0.141 0.036
Scaffold-GS 0.024 0.054 0.015 0.075 0.045 0.010 0.139 0.019
TensoRF 0.022
Paper's LPIPS (VGG): 0.018
0.076
Paper's LPIPS (VGG): 0.073
0.029
Paper's LPIPS (VGG): 0.022
0.033
Paper's LPIPS (VGG): 0.032
0.059
Paper's LPIPS (VGG): 0.058
0.021
Paper's LPIPS (VGG): 0.015
0.141
Paper's LPIPS (VGG): 0.138
0.027
Paper's LPIPS (VGG): 0.022
Gaussian Splatting 0.019 0.044 0.013 0.026 0.043 0.008 0.130 0.015
Mip-Splatting 0.021 0.046 0.013 0.028 0.044 0.008 0.132 0.018
Gaussian Opacity Fields 0.021 0.045 0.013 0.028 0.043 0.008 0.131 0.017
Zip-NeRF 0.019
Paper's LPIPS (VGG): 0.019
0.054
Paper's LPIPS (VGG): 0.05
0.014
Paper's LPIPS (VGG): 0.015
0.023
Paper's LPIPS (VGG): 0.02
0.040
Paper's LPIPS (VGG): 0.032
0.008
Paper's LPIPS (VGG): 0.007
0.114
Paper's LPIPS (VGG): 0.091
0.017
Paper's LPIPS (VGG): 0.017