Compact language model

Compact language model is a language model whose size has been significantly reduced compared to large models. It is cheap and fast, so it is in the spotlight for specific tasks.

A small language model (SLM) is a language model that maintains the structure of a large language model while greatly reducing the number of parameters, allowing it to run with fewer resources. Instead of a luxury hotel kitchen that does all the cooking, it can be likened to a specialty restaurant that makes one menu item quickly and inexpensively.

It attracted attention because it was recognized that using a large, high-end model for all tasks was a waste of money. For tasks with a defined scope, such as document classification, summarization, and customer service, it is cheaper and faster to use small models by fine-tuning them, and they are also the mainstay of on-device AI that is installed directly on devices such as smartphones.

Small models do not necessarily have lower quality, but there is a gap with large models in terms of extensive knowledge and complex reasoning. Therefore, a combination strategy where SLM takes on easy tasks and a large model takes on difficult tasks is commonly used.

✅ Why it matters

• Low operating costs and fast response speed
• When fine-tuned for specific tasks, it performs as well as larger models
• Can be mounted on devices, making it suitable for on-device AI and security needs

⚠️ Limits and debates

• When it comes to broad knowledge and complex inference, it lags behind large models
• There is a management burden of selecting and tuning models for each task
• Smaller models may have higher hallucinations and error frequencies