The Σ-Model: A Dynamical Systems Framework for Compositional Representation Formation

Dynamical Systems Theory

The Two-Subspace Learning Law

Compositional failure in deep neural networks is not an optimizer anomaly or data scarcity artifact: standard Empirical Risk Minimization (ERM) drives continuous gradient flow into a low-schema-coherence shortcut attractor ($E_S$). Introducing structural pressure $\lambda$ induces an analytical transcritical bifurcation at $\lambda_{\text{crit}}$.

Theorem 1: Transcritical Bifurcation

Level 1 Theorem

Gradient flow decomposes into orthogonal subspaces: shortcut heuristic coordinate $u(t)$ and schema-aligned coordinate $v(t)$:

u̇ = u(θ_S - a_Su - b_Sv²)
v̇ = v(λa_C - b_C - κv²)

At the critical threshold λ_crit = b_C / a_C = 0.025 ($R_0 = \lambda a_C / b_C = 1$), the shortcut equilibrium $E_S$ destabilizes into an unstable saddle and stability exchanges to the coherent attractor $E_C$.

Theorem 2: Dimension Bounds

Level 1 Theorem

By the Cauchy-Schwarz inequality, the effective participation ratio of the internal representation space is strictly bounded from below:

D_eff = 1 / ∑ p_i² ≥ 2.29

Empirical PCA spectrum confirms $\text{PC}_1 = 58.2\%$, $\text{PC}_2 = 28.2\%$, and remaining dimensions carry $13.6\%$ of activation variance, ruling out 1D degenerate collapse.

Empirical Replication

Level 3 Observation

960 production neural network runs across 4 compositional benchmark suites (ℏ, SCAN, COGS, PCFG-SET) and 3 architecture classes confirm:

Sharp change-point separatrix ($k = 79.5 \gg 15.0$).
100% late-onset interventional rescue at step 1000.
Internal CKA alignment precedes behavioral jumps ($F(2,29)=3.72, p=0.037$).

Interactive Simulation

Two-Subspace Phase Portrait Explorer

Interact directly with the continuous dynamical system. Adjust compositional structural pressure $\lambda$ to observe vector field trajectories, stream particle convergence, and the exact stability exchange at $\lambda_{\text{crit}} = 0.025$.

• Horizontal: Shortcut coordinate u(t) • Vertical: Schema coordinate v(t) • Red circle: Unstable saddle • Green circle: Stable attractor

Compositional Pressure (λ) 0.0150

0.000 (ERM) λ_crit = 0.025 0.055 (Supercritical)

Reproduction Ratio R₀ 0.60

Critical Threshold 0.0250

Fixed Point Spectrum:

Empirical Evidence (960 Runs)

Sharp Phase Boundary & Multi-Benchmark Matrix

Empirical escape probabilities across 11 dense grid levels on ℏ ($N=330$ runs) decisively fit a sharp logistic transition ($k = 79.48, \text{AIC}_{\text{seed}} = 351.26$), rejecting smooth dose-response regularizers.

Figure: Empirical escape fraction across 11 dense grid levels on ℏ (N=330 runs) overlaid on fitted logistic change-point curve (k = 79.5).

Model Selection Parsimony

Seed-level Bernoulli log-likelihoods confirm continuous sigmoidal models are statistically indistinguishable (ΔAIC < 0.3):

• Logistic: AIC = 351.26 (Normal form)
• Probit: AIC = 351.52 (ΔAIC = +0.26)
• Gompertz: AIC = 351.41 (ΔAIC = +0.15)
• Piecewise-Linear: AIC = 763.81 (Rejected)

Negative Permutation Control

Randomly permuting structural substitution pairs under identical compute and loss magnitude collapses OOD accuracy:

• ℏ: 58.1% ± 3.4% (Welch t = 0.89, p = 0.38 vs ERM)
• COGS: 32.4% ± 4.1% (Welch t = 1.90, p = 0.062 vs ERM)

Benchmark Suite	Task Domain	Architecture	ERM Baseline (λ = 0.000)	Supercritical (λ = 0.050)	Gain (ΔOOD)	Permutation Control
ℏ Homomorphic Algebra	Algebraic Recursion & Composition	Transformer 2L (0.93M)	58.74% ± 13.97%	99.42% ± 0.38%	+40.68%	58.12% ± 3.41%
SCAN	Command-to-Action Sequences (`jump`)	Transformer 2L (0.93M)	12.45% ± 3.18%	98.85% ± 0.62%	+86.40%	14.20% ± 2.85%
COGS	Structural Semantic Parsing	Transformer 2L (0.93M)	34.50% ± 4.50%	98.15% ± 0.95%	+63.65%	32.48% ± 3.12%
PCFG-SET	Context-Free Syntactic Recursion	Transformer 2L (0.93M)	51.20% ± 6.25%	99.10% ± 0.45%	+47.90%	50.85% ± 4.10%

Publication Figures

Analytical & Empirical Diagnostic Suite

Figure 1: Analytical Phase Portraits. Vector fields and representative streamlines for subcritical (λ = 0.015) vs supercritical (λ = 0.035) regimes, illustrating the transcritical exchange of stability.

Figure 3: Representation Geometry Dynamics. Linear CKA and Whitened GCA trajectories through training, demonstrating internal alignment preceding behavioral OOD generalization jumps.

Figure 4: Cross-Benchmark Generalization Curves. Generalization performance across all four benchmark suites as a function of compositional pressure λ.

Figure 5: Hessian Curvature Dynamics. Top eigenvalue λ_max trajectories evaluated via matrix-free Lanczos iteration, confirming stability below the Edge of Stability ceiling (2/η = 2000.0).

Reproducibility & Open Science

Turnkey Single-Command Replication

All code, configurations, deterministic seed schedules, and raw logs are fully accessible for bit-for-bit independent verification.

# 1. Clone the authoritative repository
git clone https://github.com/basyirin-dev/sigma-model.git
cd sigma-model

# 2. Activate Python environment & install dependencies
python3 -m venv hbar_env
source hbar_env/bin/activate
pip install -e ".[dev]"

# 3. Run full automated test suite (184 unit & regression tests)
make test

# 4. Regenerate all 17 canonical evaluation tables from raw production logs
python -m paper.src.data.derive_processed_tables

# 5. Build master monograph and submission PDFs
make paper
make -C paper journal

Deterministic Seed Protocol

All production runs follow the prospective seed rule: seed = cell_seed_idx * 42 + 7, with CUDA deterministic backend enabled and cuDNN benchmarking disabled.

Zero Data Leakage

Syntactic support disjointness: supp(D_train) ∩ supp(D_test) = ∅. SHA-256 hash checks verify 0.0% sequence overlap across all benchmarks.

Citation

BibTeX Reference

@article{basri2026critical,
  title={Critical Compositional Pressure: A Phase-Boundary Framework for Compositional Representation Formation in Neural Networks},
  author={Basri, Basyirin Amsyar},
  journal={Advances in Artificial Intelligence and Machine Learning (AAIML)},
  year={2026},
  url={https://github.com/basyirin-dev/sigma-model},
  note={Reproducibility Portal: https://basyirin-dev.github.io/sigma-model/}
}