Nitin Singh

LIFE-LONG STUDENT, TEACHER AND DEVELOPER

Deploying Ollama with Open WebUI Locally: A Step-by-Step Guide

by ·

Introduction

Large Language Models (LLMs) have become a cornerstone of modern AI applications, from chatbots that provide customer support to content generation tools for iamges and videos. They also power virtual assistants, automated translation systems, and personalized recommendation engines, showcasing their versatility across industries. However, running these models on a local machine has traditionally been a complex and resource-intensive task, requiring significant configuration and technical expertise. This complexity often deters beginners and even intermediate users who are eager to explore the capabilities of LLMs in a private, local environment.

Ollama revolutionizes this process with its clean and simplified approach. Unlike traditional methods that require manual dependency management and complex setups, Ollama leverages Docker to provide a ready-to-use environment. This not only saves time but also ensures that even beginners can start using advanced LLMs without extensive technical knowledge. By packaging open-source models such as LLaMA and Mistral into a user-friendly Docker-based system, Ollama eliminates the need for intricate setups and extensive dependencies. This streamlined solution offers significant advantages, especially for first-time users:

  • Flexibility: Supports various open-source models and customization options.
  • Ease of Setup: Ollama’s integration with Docker allows for rapid deployment without manual configuration of dependencies.
  • Beginner-Friendly: Its simplicity makes it an excellent choice for users who want to explore LLMs locally without diving into technical intricacies.
  • Data Privacy: By hosting models locally, you maintain complete control over your data.
  • Cost Efficiency: Avoid API usage costs by self-hosting.

In this guide, we’ll walk through two methods to deploy Ollama with Open WebUI locally. For beginners or those seeking a quick setup, the Docker Compose method is ideal due to its simplicity and minimal configuration. Advanced users who prefer greater control over the installation process may opt for the manual setup method.

  1. Using Docker Compose.
  2. Running the tools manually by downloading and configuring them.

By the end of this article, you’ll have a robust local environment for deploying and interacting with LLMs.

Understanding Ollama and Open WebUI

What is Ollama?

Ollama is a tool designed to simplify the deployment of LLMs on local machines. It supports various open-source models like LLaMA and Mistral and offers:

  • A CLI and API for seamless interaction.
  • Local deployment for enhanced data privacy.
  • Compatibility with machines having moderate hardware resources.

Official Repository: Ollama GitHub

What is Open WebUI?

Open WebUI is a web-based interface for interacting with LLMs hosted locally or on APIs. Its features include:

  • A clean, user-friendly UI.
  • Model management and API configuration.
  • Compatibility with Ollama and other backends.

Official Repository: Open WebUI GitHub

Why Deploy Locally?

  • Data Privacy: Keep all interactions and data secure within your environment.
  • Cost Savings: Avoid API usage costs by self-hosting.
  • Customization: Tailor the deployment to specific needs or projects.

Step-by-Step Implementation

1. Method 1: Using Docker Compose

Prerequisites

Ensure the following are installed:

Step 1: Create the Docker Compose File

  1. Create a directory for the project and navigate to it:bashCopy codemkdir ollama-webui && cd ollama-webui
  2. Create a docker-compose.yml file:yaml
version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434" # Ollama API port
    volumes:
      - ollama_data:/root/.ollama
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080" # Open Web UI port
    environment:
      - OLLAMA_API_BASE_URL=http://ollama:11434
    depends_on:
      - ollama
    restart: unless-stopped

volumes:
  ollama_data:

Step 2: Start the Services

Run the following command in the same directory:

docker-compose up -d

Step 3: Access Open WebUI

It will show a landing page, on start it will show the Signup page :

After Signup, you can search for a model or choose an existing model: (Please note in below screenshot, I had already downloaded llama3.3)

Once model pull starts, you will see the progress as below

Once Model downloaded, you can select and write your prompt and enter

2. Method 2: Manual Setup

Prerequisites

  • Install Node.js and npm (Download).
  • Install Python (3.7 or later) and pip.
  • Install Git.

Step 1: Install Ollama

Preview

  1. Download and install Ollama Download Ollama on Windows
  2. Pull a model, for example:
ollama pull llama2

Step 2: Clone and Run Open WebUI

  1. Clone the Open WebUI repository:
  2. Install dependencies
  3. Start the server
  4. Access Open WebUI at http://localhost:3000.
git clone https://github.com/open-webui/open-webui.git 
cd open-webui
pip install open-webui
open-webui serve

Using Open WebUI to Manage Models

Pulling Models

  • In the Open WebUI interface, go to the “Models” section.
  • Search for and download models like LLaMA2 or Mistral.

Alternatively, use the Ollama CLI:

ollama pull mistral

Interacting with Models

Once models are pulled, you can interact with them directly in the Open WebUI chat interface.

Custom Models

Define custom behavior with a Modelfile:

FROM llama2
SYSTEM "You are a helpful assistant."

Run:

ollama create my-model -f Modelfile
ollama run my-model

Common Pitfalls and Best Practices

  1. Docker Issues: Ensure Docker Desktop is running before using docker-compose.
  2. Port Conflicts: Verify that ports 11434 and 3000 are not in use.
  3. Model Compatibility: Confirm that your hardware supports the chosen models.
  4. Resource Optimization: Allocate sufficient memory to Docker containers for smooth performance.

Conclusion

By following this guide, you have gained insights into:

  • Deploying Ollama and Open WebUI using Docker Compose.
  • Manually setting up these tools for local deployment.
  • Effectively pulling and utilizing models.

This setup empowers you to work with LLMs locally, ensuring data privacy, cost efficiency, and enhanced flexibility.

In the next article, we’ll dive deeper into fine-tuning models and other advanced topics to maximize the potential of your local LLM deployment. Stay tuned!

Get Involved!

  • Join the Conversation: What challenges have you encountered when running LLMs locally? Share your thoughts and experiences in the comments!
  • Follow Us: Stay updated by following us on GitHub.
  • Subscribe: Sign up for our newsletter to receive expert Azure development tips.

Resources

Discover more from Nitin Singh

Subscribe to get the latest posts sent to your email.

Tags:

Leave a Reply