LLFF

LLFF is a dataset of forward-facing scenes with a small variation in camera pose. NeRF methods usually use NDC-space parametrization for the scene representation.

Web: https://bmild.github.io/llff/
Paper: LLFF: A Large-Scale, Long-Form Video Dataset for 3D Scene Understanding
Authors: Ben Mildenhall, Pratul P. Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, Abhishek Kar
Method PSNR SSIM LPIPS (VGG) Time GPU mem.
Mip-NeRF 360 26.59 0.846 0.168 9h 55m 23s 104.98 GB
Fern 24.59 0.820 0.210 7h 29m 8s 129.06 GB
Flower 27.56 0.867 0.140 14h 2m 26s 64.85 GB
Fortress 31.34 0.900 0.117 14h 2m 6s 64.85 GB
Horns 28.51 0.909 0.124 7h 26m 41s 129.06 GB
Leaves 19.84 0.721 0.231 7h 26m 52s 129.06 GB
Orchids 19.51 0.660 0.246 7h 27m 12s 129.06 GB
Room 33.49 0.965 0.115 7h 26m 22s 129.06 GB
Trex 27.86 0.927 0.158 14h 2m 15s 64.85 GB
TensoRF 26.68
Paper's PSNR: 26.73
0.834
Paper's SSIM: 0.839
0.202
Paper's LPIPS (VGG): 0.204
31m 32s 20.38 GB
Fern 25.08
Paper's PSNR: 25.27
0.801
Paper's SSIM: 0.814
0.246
Paper's LPIPS (VGG): 0.237
36m 6s 18.28 GB
Flower 28.38
Paper's PSNR: 28.6
0.860
Paper's SSIM: 0.871
0.171
Paper's LPIPS (VGG): 0.169
33m 33s 23.74 GB
Fortress 31.48
Paper's PSNR: 31.36
0.898
Paper's SSIM: 0.897
0.141
Paper's LPIPS (VGG): 0.148
32m 41s 18.39 GB
Horns 28.33
Paper's PSNR: 28.14
0.882
Paper's SSIM: 0.877
0.180
Paper's LPIPS (VGG): 0.196
33m 26s 18.27 GB
Leaves 20.96
Paper's PSNR: 21.3
0.730
Paper's SSIM: 0.752
0.239
Paper's LPIPS (VGG): 0.217
28m 21s 20.11 GB
Orchids 19.85
Paper's PSNR: 19.87
0.644
Paper's SSIM: 0.649
0.279
Paper's LPIPS (VGG): 0.278
32m 3s 20.69 GB
Room 31.86
Paper's PSNR: 32.35
0.950
Paper's SSIM: 0.952
0.159
Paper's LPIPS (VGG): 0.167
27m 20s 16.38 GB
Trex 27.53
Paper's PSNR: 26.97
0.909
Paper's SSIM: 0.9
0.199
Paper's LPIPS (VGG): 0.221
28m 46s 27.20 GB

PSNR

Peak Signal to Noise Ratio. The higher the better.

Method Fern Flower Fortress Horns Leaves Orchids Room Trex
Mip-NeRF 360 24.59 27.56 31.34 28.51 19.84 19.51 33.49 27.86
TensoRF 25.08
Paper's PSNR: 25.27
28.38
Paper's PSNR: 28.6
31.48
Paper's PSNR: 31.36
28.33
Paper's PSNR: 28.14
20.96
Paper's PSNR: 21.3
19.85
Paper's PSNR: 19.87
31.86
Paper's PSNR: 32.35
27.53
Paper's PSNR: 26.97

SSIM

Structural Similarity Index. The higher the better. The implementation matches JAX's SSIM and torchmetrics's SSIM (with default parameters).

Method Fern Flower Fortress Horns Leaves Orchids Room Trex
Mip-NeRF 360 0.820 0.867 0.900 0.909 0.721 0.660 0.965 0.927
TensoRF 0.801
Paper's SSIM: 0.814
0.860
Paper's SSIM: 0.871
0.898
Paper's SSIM: 0.897
0.882
Paper's SSIM: 0.877
0.730
Paper's SSIM: 0.752
0.644
Paper's SSIM: 0.649
0.950
Paper's SSIM: 0.952
0.909
Paper's SSIM: 0.9

LPIPS (VGG)

Learned Perceptual Image Patch Similarity. The lower the better. The implementation uses VGG backbone and matches lpips pip package with checkpoint version 0.1

Method Fern Flower Fortress Horns Leaves Orchids Room Trex
Mip-NeRF 360 0.210 0.140 0.117 0.124 0.231 0.246 0.115 0.158
TensoRF 0.246
Paper's LPIPS (VGG): 0.237
0.171
Paper's LPIPS (VGG): 0.169
0.141
Paper's LPIPS (VGG): 0.148
0.180
Paper's LPIPS (VGG): 0.196
0.239
Paper's LPIPS (VGG): 0.217
0.279
Paper's LPIPS (VGG): 0.278
0.159
Paper's LPIPS (VGG): 0.167
0.199
Paper's LPIPS (VGG): 0.221