A fine-tuned adapter sitting in a folder is not a product. FineTune Studio offers three inference paths — local, a vLLM server, or a Hugging Face Space — because the right one depends on who will call the model and how often. Here is how I choose, and the checks I run before calling it deployed.

Keep the adapter or merge it?

Keeping the adapter separate lets you hot-swap behaviours on one base model and keeps files small. Merging bakes the adapter into the base weights, producing a single model that any serving stack can load without adapter support. Merge when you have one behaviour and want simplicity; keep adapters when you serve several tasks from one base.

Three serving paths

  • Local inference — the training machine loads the model and answers requests. Right for demos, evaluation and internal tools; wrong for anything with real traffic.
  • vLLM server — batched, fast, OpenAI-compatible API. Right for production on a GPU you control. Supports LoRA adapters directly, so merging is optional.
  • Hosted Space — a public or gated endpoint for sharing a demo. Right for portfolios and hackathon judges; be mindful of cold starts and free-tier limits.

Checks before you call it shipped

  1. The served model reproduces your evaluation outputs. Precision differences and a missing chat template produce a subtly different model.
  2. The chat template used at serving time is the one used in training.
  3. Stop tokens are configured; otherwise the model talks until the token limit.
  4. Throughput and memory are measured with realistic concurrency, not one request at a time.

Operating it

Put the endpoint behind the same reliability layer as any LLM API: timeouts, retries, structured logs with the model version in every record. A fine-tuned model is a dependency you now maintain; version the adapter, the dataset and the evaluation results together so you can answer "what changed?" six months later.

Training is the exciting half. Serving is the half that makes it real, and it is where students separate themselves from tutorial followers.

— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.