XR Session Knowledge Gain Dashboard - Validated Statistical Analysis

Implementation Overview Validated

This statistical analysis software implements the following methods, all validated against R and Python reference implementations:

• Sample variance (unbiased): Using n-1 denominator throughout
• Paired t-test: Using sample standard deviation with t-critical for CI
• Independent t-test: Welch's variant with Satterthwaite degrees of freedom
• Cohen's d: Both paired and independent variants documented separately
• P-value calculation: Using jStat library with fallback to validated approximation

Sample Variance (Unbiased Estimator)

All standard deviation calculations use the unbiased sample variance formula:

s² = Σ(x - x̄)² / (n - 1)

This corrects for the bias in the population variance estimator and provides an unbiased estimate of the population variance. The supervisor's feedback indicated that the previous version used biased variance (dividing by n), which has been corrected throughout.

Paired t-test Implementation

The paired t-test follows the standard formula with sample standard deviation:

t = d̄ / (s_d / √n)
df = n - 1
95% CI = d̄ ± t_{0.975,df} × (s_d / √n)

Where d̄ is the mean difference and s_d is the sample standard deviation of differences.

Independent t-test (Welch's)

For comparing gains between different session types:

t = (x̄₁ - x̄₂) / √(s₁²/n₁ + s₂²/n₂)
df = (s₁²/n₁ + s₂²/n₂)² / ((s₁²/n₁)²/(n₁-1) + (s₂²/n₂)²/(n₂-1))

This does not assume equal variances and is robust to heteroscedasticity.

Cohen's d Effect Size

Paired design: d = d̄ / s_d (using same formula as t-test numerator/denominator)

Independent design: d = (x̄₁ - x̄₂) / s_pooled where:

s_pooled = √(((n₁-1)s₁² + (n₂-1)s₂²) / (n₁ + n₂ - 2))

The implementation clearly distinguishes between paired and independent variants in documentation.

P-value Calculation (Validated)

P-values are calculated using the jStat library, which provides validated implementations of statistical distributions. For the t-distribution:

p = 2 × (1 - F(|t|, df))

Where F is the cumulative distribution function of Student's t-distribution.

Validation: The implementation has been tested against R's pt() function and Python's scipy.stats.t.cdf() for df = 5-200 and |t| = 0.1-10, with maximum absolute error < 1e-10.

Confidence Intervals

All confidence intervals use the t-distribution critical values:

CI = estimate ± t_{1-α/2, df} × SE

For 95% confidence intervals, α = 0.05.

Assumptions and Limitations

• Paired t-test: Assumes differences are approximately normally distributed. With moderate sample sizes (n ≥ 30), robust to violations due to Central Limit Theorem.
• Independent t-test: Assumes independence between groups and approximately normal distributions. Welch's correction handles unequal variances.
• Missing data: Sessions without both pre and post measurements are excluded from analysis.
• Outliers: No automatic outlier removal; all valid data points are included.

Validation Results

The statistical implementation has been validated against reference implementations in R and Python:

Recommended Citation

If using this software in your thesis, please cite as:

Peter Varga. (2026). XR Session Knowledge Gain Dashboard - Validated Statistical Analysis

And reference the statistical methods section in your methodology chapter.