The Covariance Matrix
What the parent explanation assumed you knew:
You understand that each Gaussian has a 'shape' that can be spherical, elongated, or flattened, and that this shape is controlled by something called the covariance matrix.
What this page explains:
What the covariance matrix actually is, how it's computed, and why it perfectly captures the shape of a 3D ellipsoid.
The Explanation
The covariance matrix is a 3x3 grid of numbers that completely describes the shape and orientation of a Gaussian ellipsoid.
Here's the key insight: instead of storing the covariance matrix directly (which would be complex to optimize), 3DGS stores two simpler things:
1. Scale (S): Three numbers saying how much to stretch in each direction (x, y, z) 2. Rotation (R): A quaternion (4 numbers) saying which way to point the stretch
The covariance matrix is then computed as: Σ = R · S · Sᵀ · Rᵀ
This is brilliant because:
- •Scale and rotation are intuitive to optimize
- •The result is always a valid (positive semi-definite) covariance matrix
- •We can easily clamp scales to prevent degenerate shapes
Geometrically, the eigenvectors of Σ point along the principal axes of the ellipsoid, and the eigenvalues tell you how stretched it is along each axis.
Visual Aid
Drag the scale sliders to stretch the Gaussian. Rotate to change orientation. Watch how the covariance matrix values change in real-time.
Open interactive demo →The "Aha" Moment
The covariance matrix turns the abstract concept of 'shape' into concrete numbers that GPUs can manipulate millions of times per second.
Go Even Deeper
This explanation assumes you understand these fundamentals. Click to learn more:
matrix multiplication
Level 1 fundamental
vectors 3d
Level 1 fundamental
eigenvalues
Level 1 fundamental