Open models: yours to run, tune and keep.

What is open source LLM?

An open-source LLM is a large language model whose trained weights are released publicly so anyone can download, run, inspect, fine-tune and redistribute them under the stated licence. Contrast a hosted model, which you can only call through an API and never see. Strictly, open source would also include the training code and data; most released models publish weights and inference code but not the data, so 'open weight' is the more accurate term, and the distinction matters when you care about reproducibility or licensing. In everyday use both terms mean a model you can run yourself.

What does open source LLM mean?

It means four practical freedoms. You can run the model where you like, including offline and on private data. You can fine-tune it on your own examples and keep the result. You can read the architecture and, where published, the training recipe. And you can build a product on it without a per-token bill or dependence on a provider's roadmap. The licence sets the limits: some are fully permissive (Apache 2.0), some add use restrictions or thresholds for very large deployments, and a few forbid certain uses. Read the licence before shipping a commercial product on a model.

What are some open source LLMs?

• Llama (Meta): the family that started the current wave; several generations and sizes, widely supported.
• Gemma (Google): compact models derived from Gemini research, strong for their size.
• Qwen (Alibaba): broad family with strong multilingual, coding and vision variants.
• Mistral and Mixtral (Mistral AI): efficient dense and mixture-of-experts models.
• DeepSeek: large reasoning and coding models with distilled small versions.
• gpt-oss (OpenAI): open-weight reasoning models released in 2025.
• Phi (Microsoft): small models trained on curated data, good on laptops.
• Nemotron (NVIDIA), Falcon, and many fine-tunes from the community on Hugging Face.

Which open source LLM is best?

There is no single best; there is a best for your task, your hardware and your licence needs, as of this month. Leaderboards such as the LMArena rankings and the Open LLM Leaderboard give a starting shortlist, but the models at the top change every few weeks and benchmarks do not predict your use case. My method: pick three candidates that fit your memory budget, write thirty real prompts with the outputs you want, run all three, and score them blind. The one that wins on your prompts is the best one, and the exercise takes an afternoon.

Which open source LLM is best for coding?

Code-specialised variants of the major families (Qwen's coder models, DeepSeek's coder and reasoning models, and the code-tuned Llama, Mistral and Gemma releases) lead the open rankings on coding benchmarks, and the picture changes with each release. For editor completion on a laptop, a small code model of a few billion parameters is fast enough to be useful. For chat-style help with larger problems, a mid-sized reasoning model is better. Test on your own repository: ask each candidate to write a function with tests in your stack and see which one you would merge. That beats any benchmark table.

What is Meta's open source LLM?

Llama is Meta's family of open-weight models, released in successive generations since 2023 across sizes from small on-device models to very large ones, with text and multimodal variants. Llama's release is widely credited with making serious open models mainstream and with seeding the tooling ecosystem (llama.cpp, Ollama and many fine-tuning libraries were built around it). The models ship under Meta's own community licence, which allows commercial use with conditions, including obligations for very large-scale services, so read it before building a business on it.

What is Google's open source LLM?

Gemma is Google's family of open-weight models, built with research and technology from the Gemini programme and released in several sizes including very small variants for phones and laptops, along with code, vision and specialised versions. Gemma models are notable for strong performance per parameter and for being easy to run on consumer hardware. They are published under Google's Gemma terms of use, which permit commercial use with some restrictions. Google also releases other open models for specific domains; Gemma is the general-purpose family most people mean.

How to host open source LLM

For one user or a prototype, Ollama or LM Studio on a laptop or a single GPU machine is enough. For an application serving many users, use an inference server built for throughput: vLLM is the common choice, with paged attention and continuous batching that serve many concurrent requests on one GPU; alternatives include text-generation-inference and SGLang. Put the server behind your own API that handles authentication, rate limits and logging. Choose a quantised version if memory is tight. Measure tokens per second and latency under load before launch; the numbers from a single-user test do not survive concurrency.

How to deploy open source LLM

• Pick the model and quantisation that fits the GPU you can afford; check the licence for your use.
• Containerise an inference server (vLLM for throughput, Ollama for simplicity) with the model weights baked in or mounted.
• Expose an OpenAI-compatible endpoint so application code is portable between local and hosted models.
• Add a gateway for keys, rate limits, logging and cost accounting per team or user.
• Set up monitoring for latency, tokens per second, GPU memory and error rate; alert on queue depth.
• Keep a fallback: a smaller model or a hosted provider for overflow, which I describe in the multi-provider fallback carousel.

How to train open source LLM

You do not train one from scratch; that takes trillions of tokens and thousands of GPUs. You fine-tune an existing open model. Collect a few hundred to a few thousand examples of input and desired output, hold out a test set, and run a parameter-efficient fine-tune (LoRA, or QLoRA to fit on a single consumer GPU) with Hugging Face's PEFT and TRL libraries or a tool built on them. Compare base and tuned outputs on the held-out prompts before you ship. FineTune Studio, the platform I built, wraps exactly this QLoRA workflow with the before-and-after comparison built in, because the comparison is where most teams discover their data needs work.