--- title: "What Is a Small Language Model (SLM)? | UCC" description: "What is a small language model (SLM)? How it differs from an LLM, how models are made smaller, and when a small, fast model is the better choice at work." url: https://workforcelearning.unitedceres.edu.sg/resources/what-is-an-slm/ --- AI FOR USERS # What Is an SLM (Small Language Model), and When Is Small Better? What a small language model is, how it differs from a large one, and how to decide when a smaller, faster model is the better fit for a workplace task. Prepared by UCC Workforce Learning · Updated 8 November 2026 · 6 min read Correct as at 8 November 2026 ![](https://workforcelearning.unitedceres.edu.sg/covers/what-is-an-slm.png) TL;DR A small language model (SLM) is a language model with far fewer parameters than a large one, built to run with modest computing power, sometimes on a laptop or phone. Small is often better for narrow, repetitive, high-volume tasks where speed, cost or keeping data on your own hardware matters more than broad knowledge. Key takeaways - An SLM works like a large language model but with fewer parameters, so it needs less computing power - Small and large are relative labels, so judge a model by testing it on your own task - Small models suit narrow, repetitive, high-volume tasks where speed and cost matter most - Running a model on your own hardware can keep data in-house, but it also makes you responsible for running it - Start with the smallest model that passes your tests and move up only when a real gap appears A [small language model (SLM)](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/#slm) is a language model with far fewer [parameters](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/#parameters) than a large one, designed to run on modest computing power, sometimes on a laptop, phone or office server. It drafts, sorts and summarises text like its larger cousins, but with less breadth. For narrow, repetitive tasks, smaller is often faster and cheaper. ## How does an SLM work? An SLM works in the same way as a [large language model (LLM)](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/#llm): it reads text as [tokens](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/#token) and predicts what comes next. The difference is scale. It has fewer parameters, the internal settings learned in training, so it needs less memory and computing power to run, and it holds less general knowledge. If you have not read it yet, our guide to [what an LLM is](https://workforcelearning.unitedceres.edu.sg/resources/what-is-an-llm/) explains the basics this article builds on: pretraining, tokens and the [context window](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/#context-window). ### Size is relative Anthropic's glossary describes an LLM as an AI language model with many parameters. This guide does not use a fixed cut-off. The 2020 research paper "Language Models are Few-Shot Learners", for example, trained eight sizes of model ranging from 125 million to 175 billion parameters. Each size is small next to the largest and large next to the smallest. ### How models are made smaller A 2024 survey of small language models on arXiv describes SLMs as performing various language tasks with minimal computational resources, which makes them suited to on-device, mobile and edge settings. It groups the methods used to make models efficient, including: - **Model compression:** reducing a model's size while trying to keep its useful behaviour. - **Pruning:** removing parts of the model that contribute little. - **Quantization:** storing the model's numbers at lower precision so they take less memory. ### Where small models run Google describes its Gemma family as lightweight, open models that can run in your applications and on your hardware, mobile devices or hosted services, and can be customised with tuning techniques so they perform better on specific tasks. That pattern, a smaller model tuned for one job and run close to the data, is the typical SLM use. ## How does an SLM compare with an LLM? An SLM trades breadth for efficiency. A large model usually knows more, handles open-ended reasoning better and copes with unfamiliar topics. A small model usually responds faster, costs less to run and can run on your own hardware. The right choice depends on the task, so compare them on the work you actually need done. Anthropic's glossary lists model size among the factors that affect latency, the delay between sending a [prompt](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/#prompt) and getting a response. The comparison below summarises the general trade-offs. Question Small language model Large language model Computing power needed Modest; may run on a laptop, phone or office server High; usually runs on a provider's cloud service Speed of response Generally faster Generally slower for the same output Running cost Generally lower Generally higher Breadth of knowledge Narrower Broader Complex or open-ended reasoning Weaker Stronger Where data goes Can stay on your own hardware Usually sent to the provider's service Best fit Narrow, repetitive, high-volume tasks Varied, complex or unfamiliar tasks Two cautions apply. First, "generally" matters: a small model tuned for one narrow task may do that task well enough, and the only way to know is to test it. Second, both sizes can produce confident errors, so a small model still needs the same checks. SkillsFuture Funded Course GenAI Builder Sprint: Build & Deploy Your First AI App A practical course for learning how to configure, evaluate and apply existing generative AI models, including vibe coding and AI-assisted app development. **S$1,600** excl. GST · Funding subject to eligibility [Apply now](https://workforcelearning.unitedceres.edu.sg/contact/?intent=apply&course=TGS-2026066909)[View course ](https://workforcelearning.unitedceres.edu.sg/courses/genai-builder-sprint/) [Next layer: AI for builders ](https://workforcelearning.unitedceres.edu.sg/resources/category/ai-for-builders/) ## When is small better? Small is better when the task is narrow and repeated often, when quick responses matter, when budgets are tight, or when data should stay on your own hardware. Anthropic's guide to choosing a model recommends an efficiency-first approach for many uses: start with a faster, more cost-effective model, test thoroughly and upgrade only if needed. That guide says the efficiency-first approach is best for: - initial prototyping and development - applications with tight latency requirements - cost-sensitive implementations - high-volume, straightforward tasks It recommends starting capability-first, with a stronger model, for complex reasoning, scientific or mathematical work, tasks requiring nuanced understanding and applications where accuracy outweighs cost. ### Questions to ask before choosing 1. **Is the task narrow?** Sorting messages into fixed categories is narrow. Advising on any customer question is not. 2. **How often does it run?** A task done thousands of times a day magnifies differences in speed and running cost. 3. **Where must the data stay?** If data should not leave your systems, a model you can run locally may help. 4. **Who will run it?** A model on your own hardware needs someone to install, update and monitor it. 5. **What does a wrong answer cost?** The higher the cost of an error, the more testing and human review you need, whatever the model's size. ## What might an SLM look like in a Singapore workplace? Imagine a small logistics firm in Tuas whose drivers type short notes when a delivery fails, such as "gate locked" or "customer asked to reschedule". Sorting hundreds of these notes into a few fixed categories each week is narrow, repetitive work that an SLM running on the firm's own server could plausibly help with. This is an illustrative scenario, not a case study. A careful approach might be: - **Define the categories first.** For example: access problem, customer not available, wrong address, damaged goods, other. - **Build a test set.** Collect past notes with the correct category agreed by staff, removing customer names and phone numbers. - **Try a small model first.** Run the test set through it and compare its labels with the agreed ones. - **Try a larger model for comparison.** If the small model misses too many, check whether a larger one closes the gap enough to justify the cost. - **Keep a [human in the loop](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/#human-in-the-loop).** Send anything labelled "other", or anything the model is unsure about, to a staff member. Keeping the notes on the firm's own server can reduce how many outside services see them. It does not remove the firm's obligations under Singapore's [Personal Data Protection Act (PDPA)](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/#pdpa), which provides a baseline standard of protection for [personal data](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/#personal-data): data about an individual who can be identified from that data, or from that data and other information the organisation has or is likely to have access to. ## What are the common mistakes with SLMs? The common mistakes are choosing a model by its size label instead of testing it, giving a small model open-ended work it cannot handle, assuming local means safe, and forgetting that someone has to maintain a self-hosted model. Each is avoidable with a clear task definition, a small test set and an agreed owner. - **Choosing by label.** "Small" and "large" describe size, not quality on your task. Test with your own examples. - **Asking too much of it.** Small models do best on narrow, well-defined jobs. Broad advice or long reasoning chains are a better fit for larger models. - **Assuming local means compliant.** Data on your own hardware is still subject to your data protection duties and your organisation's rules. - **Forgetting maintenance.** A model you run yourself needs updates, monitoring and someone accountable when it fails. - **Skipping checks.** A smaller model can still state wrong things confidently. Our guide to [tokens and context windows](https://workforcelearning.unitedceres.edu.sg/resources/tokens-and-context-windows/) also explains why smaller context limits can catch people out. - **Never revisiting the choice.** Re-run your test set when the task, the data or the available models change. ## What to learn next After understanding small language models, the next steps are learning why every model, large or small, sometimes gets facts wrong, and how tokens and context windows shape what a model can handle. From there, building and testing a simple AI app shows these trade-offs in practice, with real tasks and real checks. - Read more guides for everyday users in [AI for users](https://workforcelearning.unitedceres.edu.sg/resources/category/ai-for-users/). - See how these topics connect in the [AI learning pathway](https://workforcelearning.unitedceres.edu.sg/resources/ai-learning-pathway/), from first prompts to governance. - To try model choices and testing hands-on, the [GenAI Builder Sprint](https://workforcelearning.unitedceres.edu.sg/courses/genai-builder-sprint/) is a 2-day classroom course where adults plan, build, test and launch a simple AI app. No previous AI or programming experience is needed, although formal entry requirements apply. Tags: [SLM](https://workforcelearning.unitedceres.edu.sg/resources/tag/slm/), [LLM](https://workforcelearning.unitedceres.edu.sg/resources/tag/llm/) ## Sources 1. [A Survey of Small Language Models](https://arxiv.org/abs/2410.20011), arXiv, accessed 8 November 2026 2. [Glossary](https://docs.claude.com/en/docs/about-claude/glossary), Anthropic (Claude Platform Docs), accessed 8 November 2026 3. [Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165), arXiv, accessed 8 November 2026 4. [Gemma models overview](https://ai.google.dev/gemma/docs), Google AI for Developers, accessed 8 November 2026 5. [Choosing the right model](https://docs.claude.com/en/docs/about-claude/models/choosing-a-model), Anthropic (Claude Platform Docs), accessed 8 November 2026 6. [PDPA Overview](https://www.pdpc.gov.sg/about/the-legislation/pdpa-overview), Personal Data Protection Commission (PDPC), accessed 8 November 2026 ## Frequently asked questions What is a small language model (SLM)? An SLM is a language model with far fewer parameters than a large language model. It works in the same way, predicting text one token at a time, but needs much less computing power, so it can suit on-device, mobile and edge settings. How small is a small language model? Small and large are relative labels. One 2020 research paper trained eight sizes of model, from 125 million to 175 billion parameters. What matters for a business is whether a given model passes your tests on your task, not which label it carries. How are small language models made smaller? A survey of small language models describes techniques including model compression, pruning and quantization. Compression and pruning reduce or remove parts of a model, and quantization stores its numbers at lower precision, so it needs less memory and computing power. When is a small model better than a large one? When the task is narrow, repetitive and high-volume, when fast responses matter, when budgets are tight, or when the model needs to run on your own hardware. Anthropic's model guide suggests starting with a faster, more cost-effective model and upgrading only if tests show a gap. When should I use a large model instead? Use a larger model for complex reasoning, tasks needing nuanced understanding, open-ended writing on unfamiliar topics, and work where accuracy matters more than cost. Larger models generally have broader knowledge. Does running a model on our own computer make it PDPA compliant? No. Keeping data on your own hardware can reduce how many services see it, but it does not remove your obligations under the PDPA, which provides a baseline standard of protection for personal data in Singapore. ![Illustration of a woman with shoulder-length hair in a white shirt.](https://workforcelearning.unitedceres.edu.sg/illustrations/character-woman-shoulder-hair.webp) ## Explore the next step [View the GenAI Builder Sprint course ](https://workforcelearning.unitedceres.edu.sg/courses/genai-builder-sprint/)[Read the plain-English guide to LLMs ](https://workforcelearning.unitedceres.edu.sg/resources/what-is-an-llm/) Guides provide general information; course-specific requirements and funding should be checked against the approved records before enrolment. RELATED POSTS ## Keep reading. [ ![](https://workforcelearning.unitedceres.edu.sg/covers/ai-glossary.png) AI for users ### AI Glossary: Plain-English Definitions of Common AI Terms 3 min read](https://workforcelearning.unitedceres.edu.sg/resources/ai-glossary/)[ ![](https://workforcelearning.unitedceres.edu.sg/covers/ai-learning-pathway.png) AI for users ### From AI User to AI Governance: A Learning Pathway 8 min read](https://workforcelearning.unitedceres.edu.sg/resources/ai-learning-pathway/)[ ![](https://workforcelearning.unitedceres.edu.sg/covers/prompting-patterns-office-work.png) AI for users ### Prompting Basics: 10 Patterns for Everyday Office Work 8 min read](https://workforcelearning.unitedceres.edu.sg/resources/prompting-patterns-office-work/)[ ![](https://workforcelearning.unitedceres.edu.sg/covers/tokens-and-context-windows.png) AI for users ### Tokens and Context Windows Explained: Why AI Forgets and What It Costs 7 min read](https://workforcelearning.unitedceres.edu.sg/resources/tokens-and-context-windows/) START HERE ## Interested in practical AI training? GenAI Builder Sprint covers model configuration, evaluation and simple application development. Check whether the programme suits your goals. [Apply now](https://workforcelearning.unitedceres.edu.sg/contact/?intent=apply&course=TGS-2026066909)[See course details ](https://workforcelearning.unitedceres.edu.sg/courses/genai-builder-sprint/)[Have a question? Enquire ](https://workforcelearning.unitedceres.edu.sg/contact/?course=TGS-2026066909) ![Goldie, the UCC mascot](https://workforcelearning.unitedceres.edu.sg/assets/goldie.webp)