Back to services

Updating variance without storing the data

A geometric derivation of the online variance update, using [4, 7, 5] and exact squared-deviation areas.

An original mathematical explanation using [4, 7, 5]. The underlying recurrence is documented in the Iterflow manuscript. This is one of four portfolio perspectives on the same paper.

State before the update

After 4 and 7, n = 2, the mean is 5.5, and the sum of squared deviations from that mean is 4.5. The next observation is 5.

Why two residuals appear

The mean moves when the new point arrives. Welford’s update multiplies the residual from the old mean by the residual from the new mean: M₂′ = M₂ + (x − μ)(x − μ′). The new count is used when updating μ.

Check independently

For [4, 7, 5], the exact mean is 16/3. The squared deviations sum to 14/3. Dividing by three gives population variance 14/9 ≈ 1.5556, matching the streaming calculation.

What the denominator means

Population variance describes the observations themselves. Sample variance uses n − 1 when estimating a population variance under the relevant sampling assumptions. The two quantities should not be compared as if they were identical.

Source

Iterflow: Composable Streaming Statistics for JavaScript

Supporting source. Original visual explanation by Mathscapes. Research findings and illustrative calculations are identified above.