CompactSplat allocates 3D Gaussians by image content, not by pixel

A group of researchers has published CompactSplat, a feedforward framework for reconstructing 3D scenes from a small number of views. The method does not require per-scene optimization, and it changes the underlying logic of how 3D Gaussians are placed. Instead of the "one or several Gaussians per pixel" scheme, it distributes primitives according to image content. The method was tested on three benchmarks: RealEstate10K, DL3DV and ScanNet. According to the authors, it outperforms previous approaches both on standard metrics and on render consistency at high resolution, while requiring substantially fewer primitives.

Why one Gaussian per pixel is wasteful

Feedforward generalizable 3D Gaussian Splatting works like this: the network looks at the input frames and, in a single pass, outputs a set of Gaussians from which the scene is then rendered. There is no lengthy fitting to a specific room, and no minutes or hours of optimization. Many current methods, including pixelSplat and MVSplat, follow this design: every pixel of the input image gets its own primitive, and sometimes several.

This simplicity has a cost. A white wall, a clear sky or a flat floor takes up a huge area of the frame but carries almost no information. A dozen large Gaussians would be enough for them, yet the network spends thousands. The result is a bloated representation that eats memory and slows down rendering. The second problem is geometry. Planes appear constantly in real scenes, but uniform allocation of primitives does not exploit that knowledge. Structure is modeled worse than it could be.

Texture shows where to put primitives

CompactSplat addresses both problems with a single mechanism. First, the image is split into regions based on texture; then hierarchical sampling decides how many Gaussians each region needs. Complex zones with fine detail get a dense scattering of primitives. Flat areas are covered with a "tiling" of enlarged Gaussians whose scale adjusts to the size of the region.

This way, the method uses the geometric prior of flat surfaces directly instead of hoping the network will learn it on its own. The distribution becomes adaptive: the primitive budget goes where it affects image quality.

A slider between quality and memory

Another practical detail is dynamic sparsity control at inference time. Gaussian density can be changed without retraining the network. The same model can run in a "heavy" mode for maximum quality, or in a lightweight mode when memory footprint matters. Normally, this kind of trade-off requires separate models or fine-tuning.

What is known about the method:

• task: 3D scene reconstruction from sparse views without per-scene optimization;
• representation: 3D Gaussian Splatting, feedforward generation;
• key components: texture-based spatial partitioning, hierarchical sampling, scale-modulated primitives for flat regions;
• control: sparsity adjustment at inference without retraining;
• benchmarks: RealEstate10K, DL3DV, ScanNet;
• result: better metrics and stable high-resolution rendering with fewer primitives.

Where saving Gaussians will pay off first

Compactness matters where a scene has to be built quickly and displayed on limited hardware. That includes AR and VR on mobile devices, real estate previews (where RealEstate10K has already become a standard test), and robot navigation, where a map of the surroundings is needed without lengthy processing. In these scenarios, fewer primitives mean less traffic, less video memory and a higher frame rate.

The ability to change density at runtime is especially useful for products that target different classes of devices. One trained model can be deployed on a server and on a phone, changing only the sparsity parameter. For teams that currently maintain several versions of a reconstruction network, this reduces maintenance.

There are also open questions worth checking before deployment. These include behavior on scenes with many thin structures, on reflective and transparent surfaces, and on outdoor video, where texture changes more sharply than in interiors. But the direction is clear: the next step in feedforward reconstruction lies not in adding more Gaussians, but in placing them more intelligently.