Mip-NeRF 360 Sparse

Modified Mip-NeRF 360 dataset with small train set (12 or 24) views. The dataset is used to evaluate sparse-view NVS methods.

Web: https://jonbarron.info/mipnerf360/
Paper: Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
Authors: Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, Peter Hedman
Method PSNR SSIM LPIPS (VGG) Time GPU mem.
Zip-NeRF 16.47 0.428 0.535 5h 27m 11s 29.82 GB
garden n12 19.69 0.521 0.372 5h 22m 55s 29.28 GB
bicycle n12 14.23 0.183 0.644 5h 18m 11s 29.08 GB
flowers n12 12.11 0.133 0.692 5h 54m 33s 29.00 GB
treehill n12 15.73 0.307 0.554 5h 31m 58s 29.19 GB
stump n12 17.11 0.239 0.623 5h 15m 30s 29.10 GB
kitchen n12 13.47 0.397 0.635 5h 56m 43s 30.69 GB
bonsai n12 14.37 0.474 0.623 5h 11m 15s 30.69 GB
counter n12 12.03 0.379 0.685 5h 23m 6s 30.69 GB
room n12 18.91 0.711 0.430 5h 56m 45s 30.69 GB
garden n24 24.47 0.754 0.209 5h 15m 40s 29.28 GB
bicycle n24 14.53 0.209 0.610 5h 14m 20s 29.08 GB
flowers n24 13.77 0.239 0.593 5h 14m 16s 29.00 GB
treehill n24 17.73 0.360 0.480 5h 41m 51s 29.19 GB
stump n24 17.29 0.239 0.607 5h 13m 15s 29.10 GB
kitchen n24 14.56 0.516 0.541 5h 22m 22s 30.69 GB
bonsai n24 17.48 0.718 0.425 5h 24m 17s 30.69 GB
counter n24 14.61 0.517 0.584 5h 27m 33s 30.69 GB
room n24 24.28 0.816 0.332 5h 24m 56s 30.69 GB
PGSR 17.20 0.491 0.468 26m 27s 6.95 GB
garden n12 18.01 0.499 0.370 23m 17s 5.90 GB
bicycle n12 15.58 0.242 0.562 22m 23s 5.35 GB
flowers n12 12.59 0.169 0.628 21m 43s 5.43 GB
treehill n12 13.78 0.277 0.578 23m 55s 5.70 GB
stump n12 17.22 0.266 0.597 18m 8s 5.54 GB
kitchen n12 16.64 0.607 0.429 28m 30s 7.36 GB
bonsai n12 13.65 0.536 0.577 24m 49s 7.36 GB
counter n12 15.14 0.544 0.511 26m 9s 7.05 GB
room n12 15.64 0.585 0.524 26m 7s 7.26 GB
garden n24 21.86 0.725 0.231 25m 21s 6.25 GB
bicycle n24 18.68 0.385 0.447 32m 11s 8.89 GB
flowers n24 15.23 0.281 0.497 29m 9s 8.07 GB
treehill n24 17.21 0.382 0.469 30m 26s 8.73 GB
stump n24 19.51 0.366 0.501 25m 3s 6.44 GB
kitchen n24 21.65 0.792 0.269 31m 58s 7.40 GB
bonsai n24 18.84 0.751 0.407 28m 8s 7.33 GB
counter n24 19.03 0.711 0.403 29m 47s 7.71 GB
room n24 19.31 0.718 0.416 28m 57s 7.23 GB
SparseGS 19.65 0.573 0.438 42m 5s 11.56 GB
garden n12 19.26 0.553 0.367 32m 50s 10.64 GB
bicycle n12 17.10 0.342 0.495 31m 41s 10.59 GB
flowers n12 14.08 0.241 0.582 32m 51s 10.45 GB
treehill n12 15.46 0.344 0.577 32m 57s 10.96 GB
stump n12 18.15 0.343 0.542 28m 11s 10.01 GB
kitchen n12 20.80 0.739 0.338 47m 38s 11.64 GB
bonsai n12 18.23 0.685 0.443 42m 58s 11.42 GB
counter n12 18.05 0.643 0.465 46m 29s 11.35 GB
room n12 20.40 0.733 0.443 45m 35s 11.41 GB
garden n24 23.74 0.744 0.243 41m 2s 12.36 GB
bicycle n24 19.80 0.479 0.424 39m 57s 12.54 GB
flowers n24 16.39 0.344 0.530 39m 35s 11.52 GB
treehill n24 18.81 0.444 0.529 38m 29s 11.79 GB
stump n24 20.03 0.443 0.481 36m 19s 12.22 GB
kitchen n24 24.20 0.839 0.258 57m 15s 12.31 GB
bonsai n24 22.76 0.818 0.381 52m 36s 12.52 GB
counter n24 22.64 0.778 0.382 55m 36s 12.13 GB
room n24 23.71 0.800 0.400 55m 29s 12.23 GB
Mip-Splatting 19.75 0.572 0.412 20m 19s 6.83 GB
garden n12 19.42 0.548 0.334 19m 10s 6.03 GB
bicycle n12 17.19 0.342 0.488 22m 26s 7.28 GB
flowers n12 14.02 0.237 0.559 18m 2s 6.21 GB
treehill n12 15.06 0.334 0.561 20m 8s 7.04 GB
stump n12 18.38 0.328 0.538 14m 28s 5.41 GB
kitchen n12 21.33 0.741 0.320 21m 49s 6.56 GB
bonsai n12 19.13 0.717 0.420 16m 4s 5.90 GB
counter n12 18.49 0.649 0.445 20m 45s 6.04 GB
room n12 20.42 0.722 0.425 21m 26s 6.41 GB
garden n24 23.80 0.748 0.199 22m 31s 7.53 GB
bicycle n24 19.82 0.475 0.395 26m 8s 9.93 GB
flowers n24 16.41 0.350 0.491 19m 38s 7.04 GB
treehill n24 17.99 0.427 0.482 19m 55s 7.65 GB
stump n24 19.76 0.413 0.479 17m 58s 7.17 GB
kitchen n24 24.22 0.840 0.231 23m 46s 6.84 GB
bonsai n24 23.00 0.836 0.338 17m 45s 6.46 GB
counter n24 23.06 0.786 0.346 20m 42s 6.30 GB
room n24 24.01 0.807 0.367 22m 57s 7.15 GB
Gaussian Splatting 19.76 0.568 0.412 19m 46s 7.16 GB
garden n12 19.51 0.541 0.335 19m 56s 6.72 GB
bicycle n12 17.21 0.334 0.484 20m 48s 7.21 GB
flowers n12 14.08 0.231 0.552 17m 53s 6.72 GB
treehill n12 15.08 0.327 0.557 20m 29s 7.75 GB
stump n12 18.36 0.324 0.535 13m 35s 5.28 GB
kitchen n12 21.38 0.738 0.324 21m 21s 6.37 GB
bonsai n12 18.97 0.710 0.423 15m 40s 6.11 GB
counter n12 18.43 0.647 0.445 19m 51s 5.97 GB
room n12 20.60 0.724 0.425 20m 38s 6.37 GB
garden n24 23.73 0.743 0.200 22m 21s 8.65 GB
bicycle n24 19.77 0.468 0.396 24m 49s 11.22 GB
flowers n24 16.49 0.345 0.489 19m 14s 7.90 GB
treehill n24 17.99 0.425 0.475 19m 54s 8.42 GB
stump n24 19.78 0.407 0.478 17m 1s 7.32 GB
kitchen n24 24.33 0.838 0.232 23m 11s 6.82 GB
bonsai n24 22.91 0.835 0.340 17m 1s 6.35 GB
counter n24 23.00 0.785 0.346 20m 2s 6.42 GB
room n24 24.02 0.803 0.370 22m 4s 7.35 GB
3DGS-MCMC 20.07 0.578 0.407 35m 60s 23.05 GB
garden n12 19.99 0.582 0.311 50m 18s 33.04 GB
bicycle n12 17.84 0.359 0.482 47m 16s 34.66 GB
flowers n12 14.49 0.236 0.542 34m 16s 38.85 GB
treehill n12 15.33 0.340 0.533 35m 24s 32.81 GB
stump n12 19.23 0.370 0.529 38m 47s 31.93 GB
kitchen n12 21.47 0.741 0.317 39m 17s 15.24 GB
bonsai n12 19.09 0.709 0.405 31m 33s 9.73 GB
counter n12 18.44 0.634 0.455 32m 53s 9.17 GB
room n12 20.28 0.720 0.411 32m 59s 9.67 GB
garden n24 23.81 0.709 0.361 10m 23s 4.38 GB
bicycle n24 19.80 0.480 0.393 44m 57s 39.34 GB
flowers n24 16.56 0.341 0.461 35m 2s 38.31 GB
treehill n24 18.47 0.434 0.444 34m 28s 32.98 GB
stump n24 20.59 0.450 0.463 37m 55s 33.02 GB
kitchen n24 24.49 0.852 0.219 39m 44s 16.78 GB
bonsai n24 23.83 0.852 0.311 32m 46s 9.96 GB
counter n24 23.10 0.784 0.338 34m 9s 11.81 GB
room n24 24.43 0.811 0.345 35m 57s 13.17 GB
Scaffold-GS 20.28 0.585 0.410 20m 41s 5.59 GB
garden n12 19.68 0.576 0.326 17m 21s 4.65 GB
bicycle n12 18.02 0.363 0.493 16m 60s 4.80 GB
flowers n12 14.52 0.240 0.553 15m 33s 4.73 GB
treehill n12 15.84 0.324 0.552 18m 55s 5.05 GB
stump n12 18.96 0.346 0.538 14m 9s 4.61 GB
kitchen n12 21.91 0.764 0.317 29m 31s 6.12 GB
bonsai n12 19.76 0.741 0.413 21m 55s 6.10 GB
counter n12 18.98 0.661 0.456 23m 7s 5.98 GB
room n12 20.91 0.740 0.415 20m 4s 6.13 GB
garden n24 23.11 0.719 0.236 19m 23s 5.13 GB
bicycle n24 20.29 0.495 0.396 19m 40s 5.66 GB
flowers n24 16.69 0.347 0.493 17m 14s 4.98 GB
treehill n24 18.92 0.453 0.468 19m 26s 5.67 GB
stump n24 20.32 0.426 0.483 17m 35s 5.43 GB
kitchen n24 24.98 0.861 0.217 31m 36s 6.79 GB
bonsai n24 24.37 0.850 0.320 23m 56s 6.29 GB
counter n24 23.04 0.794 0.348 24m 17s 6.12 GB
room n24 24.76 0.821 0.352 21m 38s 6.30 GB
DropGaussian 21.45 0.616 0.458 5m 21s 4.84 GB
garden n12 20.74 0.579 0.421 4m 36s 4.21 GB
bicycle n12 19.46 0.425 0.523 4m 21s 3.97 GB
flowers n12 16.39 0.296 0.615 4m 25s 3.96 GB
treehill n12 17.39 0.413 0.610 4m 23s 4.21 GB
stump n12 19.84 0.406 0.571 4m 5s 3.88 GB
kitchen n12 22.83 0.792 0.306 6m 60s 5.49 GB
bonsai n12 21.23 0.786 0.404 6m 14s 5.60 GB
counter n12 20.07 0.725 0.425 6m 25s 5.53 GB
room n12 22.78 0.797 0.422 6m 31s 5.49 GB
garden n24 24.26 0.694 0.369 4m 35s 4.46 GB
bicycle n24 21.27 0.496 0.496 4m 29s 4.42 GB
flowers n24 17.99 0.355 0.588 4m 35s 4.21 GB
treehill n24 20.31 0.487 0.576 4m 22s 4.33 GB
stump n24 21.09 0.466 0.556 4m 16s 4.21 GB
kitchen n24 25.85 0.863 0.248 6m 49s 5.74 GB
bonsai n24 25.12 0.863 0.351 6m 15s 5.78 GB
counter n24 24.27 0.817 0.360 6m 22s 5.82 GB
room n24 25.25 0.832 0.399 6m 37s 5.74 GB

PSNR

Peak Signal to Noise Ratio. The higher the better.

Method garden n12 bicycle n12 flowers n12 treehill n12 stump n12 kitchen n12 bonsai n12 counter n12 room n12 garden n24 bicycle n24 flowers n24 treehill n24 stump n24 kitchen n24 bonsai n24 counter n24 room n24
Zip-NeRF 19.69 14.23 12.11 15.73 17.11 13.47 14.37 12.03 18.91 24.47 14.53 13.77 17.73 17.29 14.56 17.48 14.61 24.28
PGSR 18.01 15.58 12.59 13.78 17.22 16.64 13.65 15.14 15.64 21.86 18.68 15.23 17.21 19.51 21.65 18.84 19.03 19.31
SparseGS 19.26 17.10 14.08 15.46 18.15 20.80 18.23 18.05 20.40 23.74 19.80 16.39 18.81 20.03 24.20 22.76 22.64 23.71
Mip-Splatting 19.42 17.19 14.02 15.06 18.38 21.33 19.13 18.49 20.42 23.80 19.82 16.41 17.99 19.76 24.22 23.00 23.06 24.01
Gaussian Splatting 19.51 17.21 14.08 15.08 18.36 21.38 18.97 18.43 20.60 23.73 19.77 16.49 17.99 19.78 24.33 22.91 23.00 24.02
3DGS-MCMC 19.99 17.84 14.49 15.33 19.23 21.47 19.09 18.44 20.28 23.81 19.80 16.56 18.47 20.59 24.49 23.83 23.10 24.43
Scaffold-GS 19.68 18.02 14.52 15.84 18.96 21.91 19.76 18.98 20.91 23.11 20.29 16.69 18.92 20.32 24.98 24.37 23.04 24.76
DropGaussian 20.74 19.46 16.39 17.39 19.84 22.83 21.23 20.07 22.78 24.26 21.27 17.99 20.31 21.09 25.85 25.12 24.27 25.25

SSIM

Structural Similarity Index. The higher the better. The implementation matches JAX's SSIM and torchmetrics's SSIM (with default parameters).

Method garden n12 bicycle n12 flowers n12 treehill n12 stump n12 kitchen n12 bonsai n12 counter n12 room n12 garden n24 bicycle n24 flowers n24 treehill n24 stump n24 kitchen n24 bonsai n24 counter n24 room n24
Zip-NeRF 0.521 0.183 0.133 0.307 0.239 0.397 0.474 0.379 0.711 0.754 0.209 0.239 0.360 0.239 0.516 0.718 0.517 0.816
PGSR 0.499 0.242 0.169 0.277 0.266 0.607 0.536 0.544 0.585 0.725 0.385 0.281 0.382 0.366 0.792 0.751 0.711 0.718
SparseGS 0.553 0.342 0.241 0.344 0.343 0.739 0.685 0.643 0.733 0.744 0.479 0.344 0.444 0.443 0.839 0.818 0.778 0.800
Mip-Splatting 0.548 0.342 0.237 0.334 0.328 0.741 0.717 0.649 0.722 0.748 0.475 0.350 0.427 0.413 0.840 0.836 0.786 0.807
Gaussian Splatting 0.541 0.334 0.231 0.327 0.324 0.738 0.710 0.647 0.724 0.743 0.468 0.345 0.425 0.407 0.838 0.835 0.785 0.803
3DGS-MCMC 0.582 0.359 0.236 0.340 0.370 0.741 0.709 0.634 0.720 0.709 0.480 0.341 0.434 0.450 0.852 0.852 0.784 0.811
Scaffold-GS 0.576 0.363 0.240 0.324 0.346 0.764 0.741 0.661 0.740 0.719 0.495 0.347 0.453 0.426 0.861 0.850 0.794 0.821
DropGaussian 0.579 0.425 0.296 0.413 0.406 0.792 0.786 0.725 0.797 0.694 0.496 0.355 0.487 0.466 0.863 0.863 0.817 0.832

LPIPS (VGG)

Learned Perceptual Image Patch Similarity. The lower the better. The implementation uses VGG backbone and matches lpips pip package with checkpoint version 0.1

Method garden n12 bicycle n12 flowers n12 treehill n12 stump n12 kitchen n12 bonsai n12 counter n12 room n12 garden n24 bicycle n24 flowers n24 treehill n24 stump n24 kitchen n24 bonsai n24 counter n24 room n24
Zip-NeRF 0.372 0.644 0.692 0.554 0.623 0.635 0.623 0.685 0.430 0.209 0.610 0.593 0.480 0.607 0.541 0.425 0.584 0.332
PGSR 0.370 0.562 0.628 0.578 0.597 0.429 0.577 0.511 0.524 0.231 0.447 0.497 0.469 0.501 0.269 0.407 0.403 0.416
SparseGS 0.367 0.495 0.582 0.577 0.542 0.338 0.443 0.465 0.443 0.243 0.424 0.530 0.529 0.481 0.258 0.381 0.382 0.400
Mip-Splatting 0.334 0.488 0.559 0.561 0.538 0.320 0.420 0.445 0.425 0.199 0.395 0.491 0.482 0.479 0.231 0.338 0.346 0.367
Gaussian Splatting 0.335 0.484 0.552 0.557 0.535 0.324 0.423 0.445 0.425 0.200 0.396 0.489 0.475 0.478 0.232 0.340 0.346 0.370
3DGS-MCMC 0.311 0.482 0.542 0.533 0.529 0.317 0.405 0.455 0.411 0.361 0.393 0.461 0.444 0.463 0.219 0.311 0.338 0.345
Scaffold-GS 0.326 0.493 0.553 0.552 0.538 0.317 0.413 0.456 0.415 0.236 0.396 0.493 0.468 0.483 0.217 0.320 0.348 0.352
DropGaussian 0.421 0.523 0.615 0.610 0.571 0.306 0.404 0.425 0.422 0.369 0.496 0.588 0.576 0.556 0.248 0.351 0.360 0.399