Multimodal Large Language Models: A Survey of Vision-Language Architectures

Abstract

Multimodal large language models, often called vision-language models, extend text-only language models so that a single system can read images together with words and produce grounded natural-language responses. This survey gives a structured and honest account of how these systems are built, trained, and measured. It first sets out the background that made them possible: the transformer sequence model, large pretrained language models, the vision transformer that treats an image as a sequence of patches, and contrastive image-text pretraining that aligns pictures and captions in a shared space. It then organises the design space into three parts, namely the vision encoder, the cross-modal connector that maps visual features into the language model, and the language decoder, and it contrasts query-based connectors, linear projectors, and gated cross-attention as well as early and late fusion. Representative systems including CLIP, Flamingo, BLIP-2, LLaVA, GPT-4V, Gemini, Qwen-VL, and PaliGemma are described from public reports, with proprietary details marked as undisclosed. The survey reviews training stages, benchmark families for visual question answering, document and chart reading, expert reasoning, and grounding, and it treats object hallucination and evaluation contamination as first-class problems. Reported scores are presented only as rounded representative illustrations drawn from heterogeneous sources. The article closes with open challenges in spatial reasoning, faithful evaluation, efficiency, and safety, and with directions for any-to-any and video-capable models.