LiteLLM & Bwat Integration (using Codestral)

Learn how to configure LiteLLM with Bwat using the Codestral model. This guide covers Docker setup, environment configuration, and Bwat integration.

LiteLLM with Bwat Setup

This tutorial demonstrates running LiteLLM locally with the Codestral model for use with the Bwat VS Code extension.

Requirements

• Docker installed on your system
• Codestral API Key (distinct from Mistral API keys)

Configuration Steps

Create environment file

# Generate a secure master key:
# openssl rand -base64 32 | tr -dc 'A-Za-z0-9' | head -c 32
LITELLM_MASTER_KEY=your_secure_key_here
CODESTRAL_API_KEY=your_codestral_api_key

Set up configuration YAML

Create config.yaml with model specifications:

model_list:
    - model_name: codestral
      litellm_params:
          model: codestral/codestral-latest
          api_key: os.environ/CODESTRAL_API_KEY

Launching the Service

Start LiteLLM container

docker run \
    --env-file .env \
    -v $(pwd)/config.yaml:/app/config.yaml \
    -p 127.0.0.1:4000:4000 \
    ghcr.io/berriai/litellm:main-latest \
    --config /app/config.yaml --detailed_debug

Configure Bwat

In Bwat settings:

1. Set endpoint: http://localhost:4000/v1
2. Enter your LITELLM_MASTER_KEY as API Key
3. Specify model ID as codestral (matches config.yaml)

Additional Resources

• LiteLLM Documentation (opens in a new tab)
• Mistral AI Console (opens in a new tab)
• Bwat Community Support

Key security note: While this setup runs locally, always use strong credentials for the master key. The Docker container binds to localhost (127.0.0.1) by default for added security.