LLM API

Configuration

After completing the installation, follow these steps to configure the LLM API, using the OpenAI API as an example. This process is similar for other LLM APIs.

Steps

1. Initialize Configuration:

   • Execute strataai --init-config to generate ~/.strataai/config2.yaml. Edit this file with your configurations to avoid sharing your API key by accident.

2. Edit Configuration:

   • Update ~/.strataai/config2.yaml according to the example below:

llm:
  api_type: 'openai' # or azure / ollama / groq etc. Check LLMType for more options
  model: 'gpt-4-turbo' # or gpt-3.5-turbo
  base_url: 'https://api.openai.com/v1' # or forward url / other llm url
  api_key: 'YOUR_API_KEY'
  # proxy: 'YOUR_LLM_PROXY_IF_NEEDED' # Optional. If you want to use a proxy, set it here.
  # pricing_plan: 'YOUR_PRICING_PLAN' # Optional. If your pricing plan uses a different name than the `model`.

Note: Configuration priority is ~/.strataai/config2.yaml > config/config2.yaml.

1. Additional Configurations for OpenAI o1 Series:

llm:
  api_type: 'openai'
  api_key: 'sk-...'
  model: 'o1-mini'
  use_system_prompt: false
  stream: false

With these steps, your setup is complete. For starting with Strata AI, check out the Quickstart guide or our Tutorials.

Additional Model Configurations

Strata AI supports a range of LLM models. Configure your model API keys as needed:

• Anthropic / Claude API:

llm:
  api_type: 'claude'
  base_url: 'https://api.anthropic.com'
  api_key: 'YOUR_API_KEY'
  model: 'claude-3-opus-20240229'

• Azure OpenAI API:

llm:
  api_type: 'azure'
  base_url: 'YOUR_AZURE_BASE_URL'
  api_key: 'YOUR_API_KEY'
  api_version: 'YOUR_API_VERSION'
  model: 'YOUR_MODEL_NAME'

• Google Gemini:

llm:
  api_type: 'gemini'
  api_key: 'YOUR_API_KEY'
  model: 'gemini-pro'

• Baidu QianFan API:

llm:
  api_type: qianfan
  access_key: 'YOUR_ACCESS_KEY'
  secret_key: 'YOUR_SECRET_KEY'
  model: 'ERNIE-Bot-turbo'

• Amazon Bedrock API:

llm:
  api_type: 'bedrock'
  model: 'meta.llama3-70b-instruct-v1:0'
  region_name: 'REGION' # e.g., us-east-1
  access_key: 'YOUR_ACCESS_KEY'
  secret_key: 'YOUR_SECRET_KEY'