Ollama

ollama.com

Ollama is a lightweight runtime for running large language models locally. It supports GGUF-quantized models like Llama, Mistral, Gemma, and Phi, with a simple CLI and REST API. Ideal for development, testing, and privacy-sensitive deployments where inference must stay on-premise or on local hardware. — extracted from the official website or Wikipedia.