Researchers are surfacing work on a method called Dust that trains transformers without…

Understanding Learning Without Backprop Training deep neural networks has relied on one algorithm for decades. Backpropagation, developed in the 1980s, became the foundation of modern AI because it efficiently calculated how to adjust billions of parameters to reduce error. The method works backward through a network, calculating gradients layer by layer. But researchers have long asked whether backprop is the only way, or simply the way we've always done it. Recent work exploring alternative training algorithms reignites an old question: what if we could train transformers without it? What backpropagation actually does Backpropagation is a mathematical procedure, not a mysterious process. When a neural network makes a prediction, the error between that prediction and the correct answer gets measured. Backprop then traces backward through every layer, computing how much each parameter contributed to that error. These contribution values, called gradients, tell the optimizer which direction and how far to adjust each weight. The algorithm is elegant and computationally efficient enough to scale to billions of parameters. For decades, it remained nearly unopposed because alternatives were slower or required more memory. Why researchers keep searching for alternatives The question of whether brains use backprop has fascinated neuroscientists for years. Biological neural networks do not appear to reverse information flow the way artificial networks do. This gap between how AI learns and how brains seem to work has motivated theoretical curiosity. Beyond neuroscience, some researchers have wondered whether backprop is simply one solution among many, or if its dominance obscures other viable paths. Exploring alternatives also reveals which properties of backprop are actually essential and which are merely conventional. Computational efficiency matters too: any training method that reduced memory usage or training time while maintaining performance would be genuinely valuable at scale. Historical context of alternative learning rules Throughout the 1990s and 2000s, researchers published work on learning algorithms beyond backpropagation. Hebbian learning, spike-timing-dependent plasticity models, and local learning rules appeared in academic literature. Most never scaled effectively to large networks. The rise of deep learning and the practical success of backprop on vision and language tasks made exploration of alternatives feel less urgent. Transformers, which emerged around 2017 and achieved remarkable results in language understanding, were all trained with backprop. The assumption grew that this was the only practical way forward. What makes training without backprop interesting now A fresh wave of theoretical work has examined whether modern architectures, particularly transformers, might be trainable through different mechanisms. The core appeal is not mystical. If a network could learn effectively without backprop's backward pass, it