Grid4D: 4D Decomposed Hash Encoding for High-fidelity Dynamic Gaussian Splatting

NeurIPS 2024

Jiawei Xu 1, Zexin Fan 1, Jian Yang 1 * and Jin Xie 2 3 *,
1 PCA Lab, VCIP, College of Computer Science, Nankai University 2 State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China 3 School of Intelligence Science and Technology, Nanjing University, Suzhou, China
* Corresponding Authors


Qualitative Results on the Synthetic D-NeRF Dataset



Qualitative Results on the Real World HyperNeRF Dataset



Qualitative Results on the Real World Neu3D Dataset

Abstract

Recently, Gaussian splatting has received more and more attention in the field of static scene rendering. Due to the low computational overhead and inherent flexibility of explicit representations, plane-based explicit methods are popular ways to predict deformations for Gaussian-based dynamic scene rendering models. However, plane-based methods rely on the inappropriate low-rank assumption and excessively decompose the space-time 4D encoding, resulting in overmuch feature overlap and unsatisfactory rendering quality. To tackle these problems, we propose Grid4D, a dynamic scene rendering model based on Gaussian splatting and employing a novel explicit encoding method for the 4D input through the hash encoding. Different from plane-based explicit representations, we decompose the 4D encoding into one spatial and three temporal 3D hash encodings without the low-rank assumption. Additionally, we design a novel attention module that generates the attention scores in a directional range to aggregate the spatial and temporal features. The directional attention enables Grid4D to more accurately fit the diverse deformations across distinct scene components based on the spatial encoded features. Moreover, to mitigate the inherent lack of smoothness in explicit representation methods, we introduce a smooth regularization term that keeps our model from the chaos of deformation prediction. Our experiments demonstrate that Grid4D significantly outperforms the state-of-the-art models in visual quality and rendering speed.

Pipeline of Grid4D


HyperNeRF architecture.


BibTeX

@article{xu2024grid4d,
    title={{Grid4D}: {4D} Decomposed Hash Encoding for High-fidelity Dynamic Scene Rendering},
    author={Jiawei, Xu and Zexin, Fan and Jian, Yang and Jin, Xie},
    journal={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
    year={2024},
}