MAMBA: SSM model as alternative to the transformers

Abstract

Mamba is a recent State Space Model (SSM) architecture to improve the computational and scalability limitations of transformer-based sequence models. In this review, we synthesize and compare Mamba’s core design—interleaved SSM and feedforward layers with hardware-aware memory management—to standard Transformers, highlighting its linear complexity and ability to process extremely long contexts. We analyze published benchmarks showing that Mamba outperforms or matches open-source baselines (e.g. Pythia, RWKV) of similar and even twice the size on zero-shot tasks, scales more efficiently on genomic sequences (processing >1 M tokens with only 74 K parameters), and supports variants such as Jamba (MoE extension), Falcon Mamba 7B, Mamba-2 (Structured SSM), and Mamba-4 with further speed or capacity gains. We discuss adaptations to vision (VIM) and dependency parsing (DepMamba), and emerging hybrids (e.g. Bamba, IBM Granite) that fuse SSM efficiency with Transformer accuracy. Finally, we interpret these findings in the context of real-world constraints—compute cost, energy, and tooling maturity—outlining where Mamba excels, where hybrid models may be preferable, and which areas require further optimization. Our conclusions suggest that Mamba and its derivatives offer a viable path toward more sustainable, scalable sequence modeling.