【Hacker News搬运】CityGaussians：实时高质量的大规模场景渲染
-
Title: CityGaussian: Real-time high-quality large-scale scene rendering with Gaussians
CityGaussians：实时高质量的大规模场景渲染
Text:
Url: https://dekuliutesla.github.io/citygs/
标题：CityGaussian：使用高斯分布进行实时高质量大规模场景渲染 作者：杨柳1,2，何冠1,2，仇晨1，卢森1，彭俊然1，张兆翔1,2,3,4 单位：1中国科学院自动化研究所，2中国科学院大学（UCAS），3中国科学院人工智能与机器人中心（CAS），4 multimodal人工智能系统国家重点实验室 摘要：实时三维场景重建和新视角合成的发展得到了三维高斯喷射（3DGS）的显著推动。然而，有效地训练大规模3DGS并在各种规模上进行实时渲染仍然具有挑战性。本文介绍了CityGaussian（CityGS），它采用了一种新颖的划分和征服训练方法以及细节级别（LoD）策略，用于高效的大规模3DGS训练和渲染。特别是，全局场景先验和自适应训练数据选择使得训练高效且融合无缝。基于融合的高斯基元，我们通过压缩生成不同的细节级别，并通过提出的块状细节级别选择和聚合策略，实现了跨不同规模的快速渲染。在大型场景上的广泛实验结果表明，我们的方法取得了最先进的重渲染质量，实现了在截然不同规模上的一致实时渲染。 与现有技术的比较：CityGS：无LoD without our proposed LoD technique, the MatrixCity is depicted by 25 million Gaussians. The consequent speed of 18 FPS (tested on A100) leads to unpleasant roaming experience. CityGS With the support of LoD, our CityGS can be rendered in real-time under vastly different scales. The average speed is 36 FPS (tested on A100). BibTeX： @misc{liu2024citygaussian, title={CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussians}, author={Yang Liu and He Guan and Chuanchen Luo and Lue Fan and Junran Peng and Zhaoxiang Zhang}, year={2024}, eprint={2404.01133}, archivePrefix={arXiv}, primaryClass={cs.CV} } 参考文献： [Turki 2022] Turki, H., Ramanan, D., Satyanarayanan, M.: Mega-nerf: Scalable construction of large-scale nerfs for virtual fly-throughs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12922–12931 (2022) [Zhenxing 2022] Zhenxing, M., Xu, D.: Switch-nerf: Learning scene decomposition with mixture of experts for large-scale neural radiance fields. In: The Eleventh International Conference on Learning Representations (2022) [Yuqi 2023] Zhang, Y., Chen, G., Cui, S.: Efficient large-scale scene representation with a hybrid of high-resolution grid and plane features. arXiv preprint arXiv:2303.03003 (2023) [Bernhard 2023] Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics 42(4) (2023)Post by: smusamashah
Comments:
speps: Note that the dataset from the video is called Matrix city. It's highly likely extracted from the Unreal Engine 5 Matrix demo released a few years ago. The views look very similar, so it's photorealistic but not from photos.<p>EDIT: here it is, and I was right! <a href="https://city-super.github.io/matrixcity/" rel="nofollow">https://city-super.github.io/matrixcity/</a>
speps: 请注意，视频中的数据集称为矩阵城市。它；很可能是从几年前发布的虚幻引擎5矩阵演示中提取的。视图看起来非常相似，因此它；s照片级真实感，但不是来自照片<p> 编辑：给你，我是对的<a href=“https://；/；city-super.github.io/，matrixcity/”rel=“nofollow”>https:///；city super.github.io/；矩阵性</一
kfarr: Not quite the same thing, but over the weekend I hacked google maps 3d tiles (mesh) together with a gaussian splat and the effect is pretty similar and effective:<p>Example 1 with code linked: <a href="https://twitter.com/kfarr/status/1773934700878561396" rel="nofollow">https://twitter.com/kfarr/status/1773934700878561396</a><p>Example 2 <a href="https://twitter.com/3dstreetapp/status/1775203540442697782" rel="nofollow">https://twitter.com/3dstreetapp/status/1775203540442697782</a>
kfarr: 不完全相同，但在周末，我将谷歌地图的三维瓦片（网格）与高斯splat一起破解，效果非常相似且有效：<p>示例1，链接代码：<a href=“https://；/；twitter.com/；kfarr&x2F；status/：1773934700878561396”rel=“nofollow”>https:///；twitter；kfarr；status；1773934700878561396</a><p>示例2<a href=“https://；/；twitter.com/！3dstreetappȏ；status/：1775203540442697782”rel=“nofollow”>https:///；twitter；3dstreetapp/；status；1775203540442697782</a>
chpatrick: "The average speed is 36 FPS (tested on A100)."<p>Real-Time if you have $8k I guess.
chpatrick: "；平均速度为36 FPS（在A100上测试）"<p> 实时，如果你有8000美元，我想。
999900000999: Excited to see what license this is released under. Would love to see some open source games using this.
999900000999: 很高兴看到这是在什么许可证下发布的。很想看到一些开源游戏使用这个。
satvikpendem: Funny to see just how prolific Gauss was since so many things are named after him and continue to be newly named after him, such as this example of Gaussian splatting, which, while he obviously didn't directly invent it, contributed to the mathematics of it significantly.
satvikpendem: 有趣的是，看到高斯是多么多产，因为很多东西都是以他的名字命名的，并继续以他的名字新命名，比如这个高斯飞溅的例子，而他显然没有；没有直接发明它，对它的数学有重大贡献。