Mip-Splatting

A modification of Gaussian Splatting designed to better handle aliasing artifacts.

Web: https://niujinshuchong.github.io/mip-splatting/
Paper: Mip-Splatting: Alias-free 3D Gaussian Splatting
Authors: Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, Andreas Geiger

Mip-NeRF 360

Mip-NeRF 360 is a collection of four indoor and five outdoor object-centric scenes. The camera trajectory is an orbit around the object with fixed elevation and radius. The test set takes each n-th frame of the trajectory as test views.

Scene PSNR SSIM LPIPS (VGG) Time GPU mem.
garden 27.47
Paper's PSNR: 27.76

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.869
Paper's SSIM: 0.875

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.124
Paper's LPIPS (VGG): 0.103

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 30m 6s 13.45 GB
bicycle 25.25
Paper's PSNR: 25.72

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.765
Paper's SSIM: 0.78

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.243
Paper's LPIPS (VGG): 0.206

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 33m 19s 15.07 GB
flowers 21.60
Paper's PSNR: 21.93

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.605
Paper's SSIM: 0.623

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.371
Paper's LPIPS (VGG): 0.331

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 22m 47s 9.90 GB
treehill 22.65
Paper's PSNR: 22.98

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.633
Paper's SSIM: 0.655

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.381
Paper's LPIPS (VGG): 0.32

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 22m 49s 9.79 GB
stump 26.64
Paper's PSNR: 26.94

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.774
Paper's SSIM: 0.786

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.251
Paper's LPIPS (VGG): 0.209

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 25m 8s 12.49 GB
kitchen 31.25
Paper's PSNR: 31.55

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.926
Paper's SSIM: 0.933

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.155
Paper's LPIPS (VGG): 0.113

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 27m 47s 9.50 GB
bonsai 31.96
Paper's PSNR: 32.31

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.941
Paper's SSIM: 0.948

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.254
Paper's LPIPS (VGG): 0.173

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 20m 49s 8.98 GB
counter 29.04
Paper's PSNR: 29.16

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.907
Paper's SSIM: 0.916

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.258
Paper's LPIPS (VGG): 0.179

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 23m 21s 9.03 GB
room 31.54
Paper's PSNR: 31.74

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.918
Paper's SSIM: 0.928

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.286
Paper's LPIPS (VGG): 0.192

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 24m 25s 10.85 GB
Average 27.49
Paper's PSNR: 27.79

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.815
Paper's SSIM: 0.827

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 0.258
Paper's LPIPS (VGG): 0.203

Authors evaluated on larger images which were downscaled to the target size (avoiding JPEG compression artifacts) instead of using the official provided downscaled images. As mentioned in the 3DGS paper, this increases results slightly ~0.5 dB PSNR.

 25m 37s 11.01 GB

Blender

Blender (nerf-synthetic) is a synthetic dataset used to benchmark NeRF methods. It consists of 8 scenes of an object placed on a white background. Cameras are placed on a semi-sphere around the object. Scenes are licensed under various CC licenses.

Scene PSNR SSIM LPIPS (VGG) Time GPU mem.
lego 35.45 0.982 0.021 6m 30s 2.51 GB
drums 26.14 0.953 0.046 6m 36s 2.53 GB
ficus 35.12 0.988 0.013 5m 8s 2.43 GB
hotdog 37.78 0.985 0.028 6m 44s 2.58 GB
materials 30.12 0.960 0.044 5m 49s 2.46 GB
mic 35.55 0.991 0.008 8m 8s 2.75 GB
ship 30.78 0.904 0.132 9m 19s 3.54 GB
chair 35.70 0.986 0.018 6m 16s 2.40 GB
Average 33.33 0.969 0.039 6m 49s 2.65 GB

Tanks and Temples

Tanks and Temples is a benchmark for image-based 3D reconstruction. The benchmark sequences were acquired outside the lab, in realistic conditions. Ground-truth data was captured using an industrial laser scanner. The benchmark includes both outdoor scenes and indoor environments. The dataset is split into three subsets: training, intermediate, and advanced.

Scene PSNR SSIM LPIPS Time GPU mem.
auditorium 24.41 0.872 0.196 11m 47s 4.82 GB
ballroom 24.15 0.826 0.098 25m 4s 9.76 GB
courtroom 23.00 0.791 0.165 20m 46s 9.27 GB
museum 20.88 0.768 0.158 24m 55s 12.57 GB
palace 19.63 0.731 0.354 12m 53s 6.48 GB
temple 20.55 0.805 0.226 12m 3s 5.93 GB
family 24.55 0.872 0.095 15m 44s 6.99 GB
francis 27.61 0.899 0.172 11m 47s 4.40 GB
horse 23.94 0.879 0.104 12m 47s 4.58 GB
lighthouse 22.25 0.844 0.159 13m 47s 7.01 GB
m60 27.98 0.904 0.112 15m 13s 7.13 GB
panther 28.27 0.908 0.109 15m 43s 7.38 GB
playground 25.87 0.861 0.155 18m 26s 8.30 GB
train 21.82 0.795 0.172 13m 26s 5.65 GB
barn 27.75 0.855 0.161 12m 42s 6.12 GB
caterpillar 23.42 0.790 0.197 13m 16s 5.69 GB
church 22.76 0.812 0.176 19m 53s 8.37 GB
courthouse 22.15 0.779 0.265 13m 57s 10.28 GB
ignatius 21.73 0.780 0.159 18m 44s 8.46 GB
meetingroom 25.46 0.870 0.137 14m 13s 5.73 GB
truck 24.36 0.857 0.108 17m 32s 7.68 GB
Average 23.93 0.833 0.166 15m 56s 7.27 GB

SeaThru-NeRF

SeaThru-NeRF dataset contains four underwater forward-facing scenes.

Scene PSNR SSIM LPIPS Time GPU mem.
Curasao 24.41 0.739 0.316 23m 38s 9.54 GB
Panama 23.99 0.752 0.237 22m 42s 6.85 GB
IUI3 19.18 0.683 0.412 19m 32s 4.26 GB
Japanese Gradens 17.75 0.747 0.253 17m 43s 4.10 GB
Average 21.33 0.730 0.304 20m 53s 6.19 GB