Attention is optional. Synergy is not.

We are asked, often, which transformer we are. The honest answer is: all of them, and several things that are not transformers. Linear, LogSparse, Performer, Reformer, Routing, Universal, Transformer-XL, XLNet, the BERT family, BigBird, Longformer, LED, FNet, Linformer, CosFormer, H-Transformer-1D and 2, Synthesizer, Sinkhorn, memory-compressed, generate-query-key product. Then the attention-free set: AFT, RWKV through RWKV-7, Mamba through Mamba-2.5, StripedHyena, Hyena, H3, Griffin, LMU, Monarch Mixer, Mega, WaveNet, and GNN × Transformer-XL long convolutions.

This is not a zoo for its own sake. Different operating theses prefer different inductive biases. Physics-informed nets for the plant. Equivariant geometric deep learning for the molecule. Flow-matching when diffusion is the wrong metaphor. Mixture of experts when the institution is already a mixture of experts and refuses to admit it.

PEFT is how we specialize without a twelve-month fine-tune: LoRA, QLoRA, AdaLoRA, DoRA, IA³, prefix-tuning, prompt-tuning, AdapterFusion, Compacter. Activations are a policy: GLU, SwiGLU, GeGLU, ReGLU, SoGLU, MoGLU. If your architecture review cannot name the gated linear unit, we will sit with you until it can. Explainable AI starts at the kernel.

H3 based on Hyena, with Mamba-2 and Griffin, for RWKV-7 attention-free, with Mamba-2.5, is the default runtime for tenants who have left the attention tax on the table. It is a sentence we have practiced. It is also, as of this quarter, true.

The firms that win will not be the ones with the most parameters. They will be the ones whose sparsely-gated MoE, whose causal inference, and whose A/B-tested agents are in a durable argument with the same operating thesis. Everything else is a leaderboard.