Ollama

What it's good for

• 1

  Running open models on-prem so no data leaves the client's network

• 2

  Offline AI for environments with no reliable internet or strict air-gap rules

• 3

  Local prototyping against real models without paying per token

• 4

  Serving a private model behind an internal API for a team or product

• 5

  Swapping in cheaper local inference for high-volume jobs where token cost adds up

Pricing

• Ollama

  Open source, MIT, run on your own hardware

  Free

How to use it

Install Ollama, then pull a model with a single command (for example, `ollama pull llama3`) and start chatting from the terminal right away. It exposes a local HTTP API on port 11434 that speaks a familiar chat-completions shape, so you can point most existing SDKs at it with a base-URL swap. For a team, run it on a shared GPU box and put your app in front of it. Match the model size to your hardware (quantized variants run on modest machines), and keep the same model pinned across environments so behavior stays consistent.

Pros & cons

Frequently asked questions

• Is Ollama free?

  Yes, completely. Ollama is free and open source under an MIT license. There is no subscription and no per-token fee. Your only cost is the hardware you run it on, since inference happens locally on your own machine or server.

• Ollama vs DeepSeek: which do I need?

  They solve different halves of the same problem. DeepSeek is a model (a free web chat plus a cheap hosted API, with open weights). Ollama is a runtime that runs open models, including DeepSeek, on your own hardware. If you want a hosted endpoint someone else operates, use DeepSeek's API. If you need the model to run on-prem or offline with no data leaving the box, run it through Ollama.

• Is Ollama open source?

  Yes, under an MIT license. The runtime itself is open, and it serves open-weight models like Llama, DeepSeek, and Mistral. Nothing is locked behind a vendor account, and no telemetry or tokens are required to use it.

• What is Ollama best for?

  Running AI privately on your own hardware: on-prem deployments, offline use, and any workload where data cannot leave the network. It is also handy for local prototyping and for high-volume jobs where paying per token would add up.

• What hardware do I need to run Ollama?

  It scales to what you have. Small quantized models run on a modern laptop with no dedicated GPU, while larger models want a GPU with enough VRAM to hold the weights. Pick a model size that fits your machine, and step up to a GPU server when you need bigger models or higher throughput.

More

• Ollama model library
• Ollama GitHub

Alternatives to Ollama

Other tools we'd consider for the same job.

• DeepSeek

  Free frontier-class chat plus the cheapest serious API on the market, with open weights you can self-host (MIT)

Other Data & ML Ops tools

• Qdrant

  Open-source vector database with a Rust engine. Self-host for data sovereignty, or run the managed cloud with a free tier

• Langfuse

  Open-source LLM engineering platform: tracing, evals, and prompt management. The own-your-traces answer to LangSmith

• Braintrust

  Commercial eval and observability platform for LLM apps: evals, logging, and a prompt playground with unlimited users

• Pinecone

  Fully-managed vector database for production RAG. Serverless tier, low operational overhead — the default for client deployments