Skip to main content

Offline AI Coding in VSCode: Continue + Ollama Complete Setup Guide

Want GitHub Copilot-level code assistance — but fully offline, air-gapped, and private? This guide walks you through setting up Continue (a VSCode extension) with Ollama (a local model runner) so you get chat, autocomplete, and inline editing powered entirely by models running on your own machine. No internet required at runtime.

What Are Continue and Ollama?

Continue is an open-source VSCode (and JetBrains) extension that adds AI-powered coding features directly into your editor — think chat, inline edits, and tab autocomplete. By default it can connect to Claude, GPT-4, or any API. But it also supports local providers, which is exactly what we want here.

Ollama is a tool that lets you download and run large language models locally on your machine with a single command. It exposes those models via a simple REST API on localhost:11434, which Continue can talk to — no cloud involved.

Together, they replicate the core experience of GitHub Copilot or the Claude VSCode extension, but 100% offline and private.


Prerequisites & Hardware Requirements

RequirementMinimumRecommended
OSWindows 10 / Linux / macOS 12Any modern 64-bit OS
RAM8 GB16 GB+
Storage10 GB free20 GB+ free
GPUNot requiredNVIDIA (CUDA) or AMD (ROCm)
VSCode1.80+Latest stable
💡 Note on RAM
The model you run must fit in memory. A 7B parameter model needs around 8 GB of RAM; a 1.5B model needs only ~4 GB. If you're RAM-constrained, stick to the lightweight autocomplete model and use 7B for chat only.

Choosing the Right Models

We'll use three models, each optimised for a different role inside Continue. This covers everything from fast keystroke-level autocomplete to deep reasoning during chat.

Code Generation / Chat
qwen2.5-coder:7b
Alibaba's code-specialist model. Excellent at multi-language code generation, refactoring, and inline edits.
~8 GB RAM
Chat & Reasoning
llama3.1:8b
Meta's latest Llama with native tool/function-calling support. Best for agent mode and complex questions.
~8 GB RAM
Autocomplete
qwen2.5-coder:1.5b
Tiny but surprisingly capable. Low latency tab-completion that doesn't slow you down while you type.
~4 GB RAM

Step 1 — Installing Ollama & Pulling Models

⚠️ Do This While You Still Have Internet
Ollama and the models must be downloaded before you go offline. Once installed, everything runs locally with zero network calls.

Install Ollama

# Linux
curl -fsSL https://ollama.ai/install.sh | sh

# macOS
brew install ollama

# Windows → download installer from ollama.ai/download

Start the Service & Verify

# Start Ollama in the background
ollama serve

# In a second terminal — check it's running
curl http://localhost:11434
# Should respond: Ollama is running

Pull the Three Models

Always use ollama pull with the exact tag. Using just the model name without a tag pulls :latest, which might not be the version you expect.

# Code generation + chat
ollama pull qwen2.5-coder:7b

# Reasoning + agent mode (tool support)
ollama pull llama3.1:8b

# Lightweight autocomplete
ollama pull qwen2.5-coder:1.5b

# Verify all three are downloaded
ollama list

Enable as a System Service (Linux)

This ensures Ollama starts automatically on boot — important in offline environments where you can't just re-run a command.

sudo systemctl enable ollama
sudo systemctl start ollama
systemctl status ollama

Step 2 — Installing Continue in VSCode (Offline)

In an air-gapped environment, you cannot install extensions from the VSCode Marketplace. Instead, grab the .vsix bundle while you have internet access.

1
Go to github.com/continuedev/continue/releases and download the latest continue-*.vsix file.
2
Transfer the file to your offline machine (USB, internal network, etc.).
3
Open VSCode → click the Extensions icon → click ⋯ (three dots) → select "Install from VSIX…" → select your file.
4
Reload VSCode. The Continue icon (a small play button) should appear in your Activity Bar.

Step 3 — Configuring config.yaml

Open your Continue config file. Its location depends on your OS:

OSPath
Linux / macOS~/.continue/config.yaml
Windows%USERPROFILE%\.continue\config.yaml

Replace the entire file contents with the following configuration:

name: Local Offline Config
version: 0.0.1
schema: v1

models:

  # ── CHAT & REASONING ─────────────────────────────────────────
  - name: Llama 3.1 8B
    provider: ollama
    model: llama3.1:8b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - apply
    capabilities:
      - tool_use           # enables Agent mode
    defaultCompletionOptions:
      contextLength: 8192
      temperature: 0.7

  # ── CODE GENERATION ──────────────────────────────────────────
  - name: Qwen2.5 Coder 7B
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - apply
    defaultCompletionOptions:
      contextLength: 8192
      temperature: 0.3    # lower = more deterministic code

  # ── AUTOCOMPLETE ─────────────────────────────────────────────
  - name: Qwen2.5 Coder 1.5B (Autocomplete)
    provider: ollama
    model: qwen2.5-coder:1.5b
    apiBase: http://localhost:11434
    roles:
      - autocomplete
    defaultCompletionOptions:
      contextLength: 4096
      temperature: 0.1
      num_predict: 256    # keep suggestions short & fast

# Disable telemetry — critical for offline / air-gapped setups
analytics:
  enabled: false
💡 Pro Tip — Switching Models
In the Continue sidebar, you can switch between "Llama 3.1 8B" and "Qwen2.5 Coder 7B" for chat any time from the model selector dropdown. Use Llama for open-ended questions and reasoning; use Qwen for pure code tasks.

Step 4 — Disabling Telemetry

Continue, by default, sends anonymous usage data to PostHog. In an offline environment this will cause silent errors or slow timeouts as it tries to reach an external server. Disable it at two levels:

In config.yaml (already included above)

analytics:
  enabled: false

In VSCode Settings

Open settings.json (Ctrl+Shift+P → "Open User Settings JSON") and add:

{
  "continue.telemetryEnabled": false
}

Alternatively: File → Preferences → Settings → search "Continue telemetry" → uncheck the box.

After making both changes, restart VSCode to ensure they take effect.


Step 5 — Usage & Keyboard Shortcuts

ActionShortcutWhat it does
Open ChatCtrl+LOpens the Continue chat panel on the right
Inline EditCtrl+IOpens an inline prompt to edit selected code
Accept AutocompleteTabAccepts the grey ghost suggestion
Reject AutocompleteEscDismisses the suggestion
Select modelDropdown in panelSwitch between your configured models

Quick Workflow Examples

→
Write a function: type a comment describing what you want, press Tab to autocomplete with Qwen 1.5B, then hit Ctrl+L to ask Llama for a review.
→
Refactor code: select a block of code, press Ctrl+I, type "refactor this to use async/await" — Qwen 7B rewrites it inline.
→
Debug an error: paste the stack trace in the chat panel (Ctrl+L), ask Llama 3.1 what went wrong — it has tool support so it can analyze your codebase context.

Troubleshooting

"Model not found" / 404 error

# Check which models are actually installed
ollama list

# If missing, pull the exact tag
ollama pull qwen2.5-coder:7b

The model name in config.yaml must match exactly what ollama list shows.

"Model requires more system memory"

Reduce the context length in your config:

defaultCompletionOptions:
  contextLength: 2048   # down from 8192

Continue can't connect to Ollama

# Verify Ollama is running
curl http://localhost:11434

# Check Linux service status
systemctl status ollama

# Check port isn't blocked
ss -tlnp | grep 11434

"Agent mode is not supported"

Not all models support function calling. Make sure you're using llama3.1:8b and that your config includes:

capabilities:
  - tool_use

Slow autocomplete

If autocomplete feels laggy, reduce num_predict to 128 or lower in the autocomplete model config. You can also try qwen2.5-coder:1.5b with contextLength: 2048 for the fastest possible suggestions.

You're Set

With Ollama serving models locally and Continue wired up to use them, your VSCode now has a fully offline, private AI coding assistant. Chat with Llama 3.1 for reasoning, generate and refactor code with Qwen 7B, and let Qwen 1.5B handle fast tab completions — all without a single byte leaving your machine.

The same setup works in air-gapped corporate environments, secure networks, or simply when you'd rather keep your codebase off third-party servers. The only thing you need the internet for is the initial download — after that, everything runs entirely on your hardware.

🔗 Useful Links (for when you're online)
Continue documentation: docs.continue.dev
Ollama model library: ollama.com/library
Continue GitHub releases (for .vsix): github.com/continuedev/continue/releases
VSCodeOllamaContinueLocal AIOffline DevLLMPrivacy

Comments

Popular posts from this blog

Local LLM on Ubuntu (CPU-Only): Ollama + Qwen Coder + Open WebUI + RAG Setup

Local LLM on Ubuntu (CPU-Only): Ollama + Qwen Coder + Open WebUI + RAG Run a private, high-performance internal AI coding assistant on your own hardware—no GPU required. Feb 13, 2026 Qwen 2.5 Coder RAG Enabled Enterprise Ready Keeping your code and documents on your own infrastructure is the gold standard for enterprise security. By combining Ollama , Qwen Coder , and Open WebUI , we can build a production-grade AI assistant that respects your privacy. Environment: CPU-only Ubuntu server (e.g., i9/Threadripper) with 64GB+ RAM for smooth 14B model inference. 1. System Preparation First, update your local environment and install necessary dependencies: sudo apt update && sudo apt upgrade -y sudo apt install -y curl git ca-certificates lscpu 2. Ollama & Model Tuning Install Ollama and optimize the configuration for CPU-only inference by setting the thread count: ...

Defining tomcat and mysql server Amazon EC2 instance

-- Create new amazon ec2 instance http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html -- Give permisson to pem file chmod 400 mycertificate.pem -- Connect to EC2 instance ssh -i mycertificate.pem ubuntu@ec2-51-51-51-51.eu-west-1.compute.amazonaws.com -- Installation Oracle Java 1.8 $ wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u92-b14/jdk-8u92-linux-x64.tar.gz $ tar -xzvf jdk-8u92-linux-x64.tar.gz $ sudo mkdir -p /usr/java/ $ sudo mv jdk1.8.0_92/ /usr/java/ $ sudo vim  /etc/profile         JAVA_HOME=/usr/java/jdk1.8.0_92         PATH=$PATH:$HOME/bin:$JAVA_HOME/bin         JRE_HOME=/usr/java/jdk1.8.0_92/jre         PATH=$PATH:$HOME/bin:$JRE_HOME/bin         export JAVA_HOME...