SMART SLM: Structured Memory and Reasoning Transformer, A Small Language Model for Accurate Document Assistance

Small Language Models (SLMs) struggle with complex document understanding due to limited parameters. SMART SLM, a novel Structured Memory and Reasoning Transformer, enhances SLMs for accurate document assistance by integrating a dynamic, structured memory module that stores and retrieves contextual information. This enables multi-hop reasoning and precise answers, especially for structured data like tables and hierarchical headings. Outperforming existing SLMs and rivaling larger models in accuracy while maintaining a compact size, SMART SLM is suitable for real-world applications in legal, medical, and technical document processing. It achieves 21.3% higher accuracy than GPT-2 with 64% fewer parameters.