What Is an SLM (Small Language Model), and When Is Small Better?

How does an SLM work?#

An SLM works in the same way as a large language model (LLM): it reads text as tokens and predicts what comes next. The difference is scale. It has fewer parameters, the internal settings learned in training, so it needs less memory and computing power to run, and it holds less general knowledge.

If you have not read it yet, our guide to what an LLM is explains the basics this article builds on: pretraining, tokens and the context window.

Size is relative#

Anthropic's glossary describes an LLM as an AI language model with many parameters. This guide does not use a fixed cut-off. The 2020 research paper "Language Models are Few-Shot Learners", for example, trained eight sizes of model ranging from 125 million to 175 billion parameters. Each size is small next to the largest and large next to the smallest.

How models are made smaller#

A 2024 survey of small language models on arXiv describes SLMs as performing various language tasks with minimal computational resources, which makes them suited to on-device, mobile and edge settings. It groups the methods used to make models efficient, including:

• Model compression: reducing a model's size while trying to keep its useful behaviour.
• Pruning: removing parts of the model that contribute little.
• Quantization: storing the model's numbers at lower precision so they take less memory.

Where small models run#

Google describes its Gemma family as lightweight, open models that can run in your applications and on your hardware, mobile devices or hosted services, and can be customised with tuning techniques so they perform better on specific tasks. That pattern, a smaller model tuned for one job and run close to the data, is the typical SLM use.

How does an SLM compare with an LLM?#

An SLM trades breadth for efficiency. A large model usually knows more, handles open-ended reasoning better and copes with unfamiliar topics. A small model usually responds faster, costs less to run and can run on your own hardware. The right choice depends on the task, so compare them on the work you actually need done.

Anthropic's glossary lists model size among the factors that affect latency, the delay between sending a prompt and getting a response. The comparison below summarises the general trade-offs.

Two cautions apply. First, "generally" matters: a small model tuned for one narrow task may do that task well enough, and the only way to know is to test it. Second, both sizes can produce confident errors, so a small model still needs the same checks.

When is small better?#

Small is better when the task is narrow and repeated often, when quick responses matter, when budgets are tight, or when data should stay on your own hardware. Anthropic's guide to choosing a model recommends an efficiency-first approach for many uses: start with a faster, more cost-effective model, test thoroughly and upgrade only if needed.

That guide says the efficiency-first approach is best for:

• initial prototyping and development
• applications with tight latency requirements
• cost-sensitive implementations
• high-volume, straightforward tasks

It recommends starting capability-first, with a stronger model, for complex reasoning, scientific or mathematical work, tasks requiring nuanced understanding and applications where accuracy outweighs cost.

Questions to ask before choosing#

1. Is the task narrow? Sorting messages into fixed categories is narrow. Advising on any customer question is not.
2. How often does it run? A task done thousands of times a day magnifies differences in speed and running cost.
3. Where must the data stay? If data should not leave your systems, a model you can run locally may help.
4. Who will run it? A model on your own hardware needs someone to install, update and monitor it.
5. What does a wrong answer cost? The higher the cost of an error, the more testing and human review you need, whatever the model's size.

What might an SLM look like in a Singapore workplace?#

Imagine a small logistics firm in Tuas whose drivers type short notes when a delivery fails, such as "gate locked" or "customer asked to reschedule". Sorting hundreds of these notes into a few fixed categories each week is narrow, repetitive work that an SLM running on the firm's own server could plausibly help with.

This is an illustrative scenario, not a case study. A careful approach might be:

• Define the categories first. For example: access problem, customer not available, wrong address, damaged goods, other.
• Build a test set. Collect past notes with the correct category agreed by staff, removing customer names and phone numbers.
• Try a small model first. Run the test set through it and compare its labels with the agreed ones.
• Try a larger model for comparison. If the small model misses too many, check whether a larger one closes the gap enough to justify the cost.
• Keep a human in the loop. Send anything labelled "other", or anything the model is unsure about, to a staff member.

Keeping the notes on the firm's own server can reduce how many outside services see them. It does not remove the firm's obligations under Singapore's Personal Data Protection Act (PDPA), which provides a baseline standard of protection for personal data: data about an individual who can be identified from that data, or from that data and other information the organisation has or is likely to have access to.

What are the common mistakes with SLMs?#

The common mistakes are choosing a model by its size label instead of testing it, giving a small model open-ended work it cannot handle, assuming local means safe, and forgetting that someone has to maintain a self-hosted model. Each is avoidable with a clear task definition, a small test set and an agreed owner.

• Choosing by label. "Small" and "large" describe size, not quality on your task. Test with your own examples.
• Asking too much of it. Small models do best on narrow, well-defined jobs. Broad advice or long reasoning chains are a better fit for larger models.
• Assuming local means compliant. Data on your own hardware is still subject to your data protection duties and your organisation's rules.
• Forgetting maintenance. A model you run yourself needs updates, monitoring and someone accountable when it fails.
• Skipping checks. A smaller model can still state wrong things confidently. Our guide to tokens and context windows also explains why smaller context limits can catch people out.
• Never revisiting the choice. Re-run your test set when the task, the data or the available models change.

What to learn next#

After understanding small language models, the next steps are learning why every model, large or small, sometimes gets facts wrong, and how tokens and context windows shape what a model can handle. From there, building and testing a simple AI app shows these trade-offs in practice, with real tasks and real checks.

• Read more guides for everyday users in AI for users.
• See how these topics connect in the AI learning pathway, from first prompts to governance.
• To try model choices and testing hands-on, the GenAI Builder Sprint is a 2-day classroom course where adults plan, build, test and launch a simple AI app. No previous AI or programming experience is needed, although formal entry requirements apply.

Sources#

1. A Survey of Small Language Models, arXiv, accessed 8 November 2026
2. Glossary, Anthropic (Claude Platform Docs), accessed 8 November 2026
3. Language Models are Few-Shot Learners, arXiv, accessed 8 November 2026
4. Gemma models overview, Google AI for Developers, accessed 8 November 2026
5. Choosing the right model, Anthropic (Claude Platform Docs), accessed 8 November 2026
6. PDPA Overview, Personal Data Protection Commission (PDPC), accessed 8 November 2026

Frequently asked questions#

What is a small language model (SLM)?+

An SLM is a language model with far fewer parameters than a large language model. It works in the same way, predicting text one token at a time, but needs much less computing power, so it can suit on-device, mobile and edge settings.

How small is a small language model?+

Small and large are relative labels. One 2020 research paper trained eight sizes of model, from 125 million to 175 billion parameters. What matters for a business is whether a given model passes your tests on your task, not which label it carries.

How are small language models made smaller?+

A survey of small language models describes techniques including model compression, pruning and quantization. Compression and pruning reduce or remove parts of a model, and quantization stores its numbers at lower precision, so it needs less memory and computing power.

When is a small model better than a large one?+

When the task is narrow, repetitive and high-volume, when fast responses matter, when budgets are tight, or when the model needs to run on your own hardware. Anthropic's model guide suggests starting with a faster, more cost-effective model and upgrading only if tests show a gap.

When should I use a large model instead?+

Use a larger model for complex reasoning, tasks needing nuanced understanding, open-ended writing on unfamiliar topics, and work where accuracy matters more than cost. Larger models generally have broader knowledge.

Does running a model on our own computer make it PDPA compliant?+

No. Keeping data on your own hardware can reduce how many services see it, but it does not remove your obligations under the PDPA, which provides a baseline standard of protection for personal data in Singapore.