Adding LLM models to GitHub Copilot CLI

There are various ways you can add models to Copilot CLI.

• Ollama: Copilot CLI automatically discovers models served by Ollama running on your machine, and you can add these models from the model picker.
• Manually configured model provider: Add an external model provider by specifying your own credentials in a configuration file. This method is often called bring your own key (BYOK).
• Installing MAI Code 1.1 Flash locally: Windows users with the required system specifications can install this model directly from the model picker and run it on their local machine.

Model requirements

Choose a model and endpoint that support tool calling (also called function calling) and streaming. These capabilities let Copilot CLI use tools and display responses as they arrive.

Configure token limits to match your model, especially for smaller local models. For an unknown model, Copilot CLI can default to a limit of 128,000 prompt tokens, which can exceed the model's actual capacity. Setting a higher limit does not increase the model's context window.

Adding models from Ollama

If Ollama is running on your local machine, and you have already downloaded a model, Copilot CLI can discover the model and allow you to add it from the model picker.

Prerequisite

You must have installed Ollama on your local machine and downloaded at least one model. For information, see Download Ollama on the Ollama website.

Adding a model

1. In an interactive session, enter /model.

2. Look for Ollama-provided models under "Local and self-hosted."

   Any models you have downloaded from Ollama but not yet added are shown with the status "Discovered."

3. Select a discovered model and press Enter.

4. Choose Add and use for this session to switch to the model, or Add without switching to keep using your current model.

5. Press Esc to exit the picker.

The CLI saves the model details to the providers.json file so it is available in later sessions.

Manually adding a model provider configuration

To use a model provider other than Ollama, you need to manually add the provider details to the providers.json file.

Prerequisite

You must have an API key to add a model provider that requires authentication.

Supported providers

For connections to a model provider's API, the following provider types are available:

Provider type	Compatible services
openai	OpenAI and services with an OpenAI-compatible API, such as Ollama, vLLM, and Foundry Local. This is the default provider type.
azure	Azure OpenAI Service.
anthropic	Anthropic (Claude models).

Adding a provider and a model

The providers.json file stores details of configured providers and models. It's located in your CLI configuration directory, normally ~/.copilot. You can use another file by setting COPILOT_PROVIDERS_CONFIG to its path before starting the CLI.

Models you configure in the providers.json file are available in the CLI's model picker, alongside the GitHub-provided models available to your account.

1. Open providers.json, or create it if it does not exist. Add your provider and model. For example, this configuration adds the OpenAI model provider and one model:

   {
      "providers": [
         {
            "name": "my-provider",
            "type": "openai",
            "baseUrl": "https://api.openai.com/v1",
            "apiKey": "YOUR-API-KEY"
         }
      ],
      "models": [
         {
            "id": "YOUR-MODEL-NAME",
            "provider": "my-provider"
         }
      ]
   }
   

   Replace YOUR-API-KEY with your provider's API key and YOUR-MODEL-NAME with a model name available to your account. For another OpenAI-compatible provider, change baseUrl to its API URL. Omit apiKey if the provider does not require authentication.

   If the file already contains providers and models, add to the existing data instead of replacing it.

   For providers or models that need additional settings:

   • Set the provider's type to azure for Azure OpenAI or anthropic for Anthropic, and use that provider's API URL and credentials.
   • For OpenAI or Azure models that use the Responses API, add "wireApi": "responses" to the provider. The default is completions.
   • For an Azure deployment, set the model's wireModel to the deployment name and modelId to the name of the model you deployed. If you need a specific API version, add "azure": { "apiVersion": "YOUR-API-VERSION" } to the provider. Replace YOUR-API-VERSION with the version required by your endpoint.
   • If the model's id is a name you chose rather than the name accepted by the provider, set wireModel to the provider's model name. Set modelId to a model name the CLI recognizes so it can use that model's capabilities and limits.
   • Set maxPromptTokens and maxOutputTokens on a model to match its supported limits. Use whole numbers greater than zero.
   • To authenticate with a bearer token instead of an API key, set bearerToken on the provider.

   Many other parameters can be configured in this file. For more information, see GitHub Copilot CLI configuration directory.

2. To add another model from the same provider, add another entry to models with a different id and the same provider.

   Note

   Each model's provider value must match a provider's name.

3. To add a different provider, add an entry to providers with a unique name, then add its models.

4. Save the file and restart the CLI.

5. To select the example model when starting a session, run:

   copilot --model PROVIDER-NAME/MODEL-NAME
   

6. To switch models during a session, use the /model slash command. For more information, see Choosing a model in GitHub Copilot CLI.

7. To stop loading a provider from your saved configuration, remove its entry from providers and its model entries from models, then restart the CLI. Choose a different default with /model --global before removing your current default model.

Installing a local model from the model picker

If you are using Windows and your computer meets the following prerequisites, you can use the CLI's model picker to install and run the following models locally on your device:

• MAI Code 1.1 Flash
• Flint NVFP4

Prerequisites

You must have:

• A Windows PC

• An NVIDIA GPU (AMD is not supported)

• Minimum available combined RAM per model:

  • MAI Code 1.1 Flash:  96 GB
  • Flint NVFP4:  48 GB

• Minimum available storage per model:

  • MAI Code 1.1 Flash:  60 GB
  • Flint NVFP4:  23 GB

• llama.cpp v0.7.0 or later installed and LLAMACPP_SERVER_PATH set to its path. Required build version:

  • MAI Code 1.1 Flash:  b11669 or later
  • Flint NVFP4:  b11803 or later

  See Installing llama.cpp for Windows.

Benefits of using MAI Code 1.1 Flash locally

MAI Code 1.1 Flash is a coding-optimized model that you can run locally on your Windows PC. By using a model that's running locally, you can use Copilot offline and avoid consuming GitHub AI Credits.

After installing MAI Code 1.1 Flash, you can select it from the model picker, just like any other model. The model also becomes available for Copilot Fusion's Fusion Standard effort level to use. When you set the model selection to Copilot Fusion with Fusion Standard, MAI Code 1.1 Flash will be automatically chosen to work on appropriate tasks, leaving cloud-based models for tasks that require more resources or higher performance.

Installing llama.cpp for Windows

To install llama.cpp on Windows with the CUDA build:

1. In your browser, go to the releases page of the llama.cpp repository: https://github.com/ggml-org/llama.cpp/releases.

2. Choose the release you want to install—for example, b11803.

3. Click the links to download both files for the Windows CUDA 13.4 ARM64 build:

   • Windows arm64 (CUDA 13)
   • CUDA 13.4 DLLs

4. Open Command Prompt and create the installation folder:

   Shell

   mkdir "%LOCALAPPDATA%\Programs\llama.cpp\BUILD-cuda-13.4-arm64"
   

   Replacing BUILD the the build number—for example, b11803.

5. Extract all files from both ZIP files directly into this folder. If you use Extract All in File Explorer, change the destination to this folder, because by default File Explorer creates a separate subfolder for each ZIP file. When you're done, the folder should contain a large number of files, including llama-server.exe, cudart64_13.dll, cublas64_13.dll, and cublasLt64_13.dll, with no subfolders.

6. Set the LLAMACPP_SERVER_PATH environment variable:

   Shell

   setx LLAMACPP_SERVER_PATH "%LOCALAPPDATA%\Programs\llama.cpp\BUILD-cuda-13.4-arm64\llama-server.exe"
   

   Replacing BUILD the the build number.

7. Close Command Prompt and open a new Command Prompt window. If the Copilot app or VS Code is already open, restart it. If Copilot CLI is running, exit it and start it again from a new terminal window.

8. To check the installation, run:

   Shell

   "%LLAMACPP_SERVER_PATH%" --list-devices
   

Using the model picker to install a local model

1. In a local interactive session, enter /model.

2. Look for the supported local models under "Local and self-hosted."

   The status is shown as "Download required".

   Note

   The local models are only available for download if your local machine is a Windows PC that meets the GPU and memory requirements.

3. Use the down arrow key to select a local model, then press Enter.

   The CLI checks that your computer meets the installation requirements. See the Prerequisites listed earlier in this article.

4. Select Download and set up to start downloading the model.

   You can continue using other models while the download is in progress. The CLI will notify you, in the conversation output, when the download is complete.

Using a local model

After the download is complete, the local model will be available for use in your sessions.

You can:

• Select it from the model picker, then choose Select this model. See Choosing a model in GitHub Copilot CLI.
• MAI Code 1.1 Flash only: Choose Copilot Fusion, then Fusion Standard in the model picker, and Copilot will automatically use the local model when appropriate.
• Assign it to a custom agent for focused tasks. You can then prompt Copilot to use that custom agent for part of a task, while keeping your main session on a different model in the cloud. See Creating and using custom agents for GitHub Copilot CLI.

Running in offline mode

To prevent Copilot CLI from contacting GitHub's servers—for example, to send telemetry data—set the COPILOT_OFFLINE environment variable to true before starting the CLI. For full network isolation, your model provider must also be local or within the same isolated environment. See Authenticating GitHub Copilot CLI.

Troubleshooting

• An Ollama model is missing: Check that Ollama is running and the model has been downloaded. Open /model and press Ctrl+R to scan again. For a server on another machine, configure its OpenAI-compatible API URL manually, including /v1.
• A saved model is missing: Check that its provider matches a provider's name, then restart the CLI. If adding a discovered model produced a save warning, follow the warning to fix the file or its permissions.
• Environment settings have no effect: Check whether providers.json, or the file selected by COPILOT_PROVIDERS_CONFIG, already contains providers or models. That file takes precedence.
• Requests fail: Check the API URL, credentials, model name, and API choice. For Azure OpenAI, check the deployment name. For errors about request size, set token limits that fit your model.