If we are personalizing routing through feedback or training a custom router with our evaluation data, we first need to create a preference ID.

Preference IDs are unique identifiers for the specific router we are calling. We can define preference IDs for different functions within our application, or even for different individual end-users.

Creating preference IDs

If we are training our own custom router, a preference ID will automatically be created when we run the fit method.

We can also create a new preference ID by navigating to the Router Preferences tab in our dashboard and selecting "Create preference ID":

Creating preference IDs programmatically

We can create and use a preference ID in our application in the following way:

from notdiamond import NotDiamond

client = NotDiamond()

preference_id = client.create_preference_id() # Create a new preference ID

result, session_id, provider = client.chat.completions.create(
    messages=[
        {"role": "system", "content": "You are a world class programmer."},
        {"role": "user", "content": "Concisely explain merge sort."},
    ],
    model=['openai/gpt-4o',  'anthropic/claude-3-5-sonnet-20240620'],
    preference_id=preference_id
)

print("ND session ID: ", session_id) 
print("LLM called: ", provider.model)
print("LLM output: ", result.content)

import { NotDiamond } from 'notdiamond';
require('dotenv').config()

const notDiamond = new NotDiamond({apiKey: process.env.NOTDIAMOND_API_KEY});

const result = await notDiamond.modelSelect({
  messages: [
    { role: 'system', content: 'You are a world class programmer.' },
    { role: 'user', content: 'Consiely explain merge sort.' },
  ],
  llmProviders: [
    { provider: 'openai', model: 'gpt-4o' },
    { provider: 'anthropic', model: 'claude-3-5-sonnet-20240620' }
	],
  preferenceId: 'YOUR_PREFERENCE_ID'
});

if ('detail' in result) {
  console.error('Error:', result.detail);
}

console.log('ND session ID:', result.session_id);
console.log('LLM called:', result.provider.model);

Updating and deleting preference IDs

We can also update or delete our preference IDs programmatically:

from typing import Optional
import requests

def update_preference_id(
    preference_id: str, 
    nd_api_key: str,
    name: Optional[str] = None
  ):
    url = "https://not-diamond-server.onrender.com/v2/preferences/userPreferenceUpdate"
    headers = {
        "Authorization": f"Bearer {nd_api_key}"
    }
    res = requests.put(
        url=url,
        headers=headers,
        json={
            "preference_id": preference_id,
            "name": name  # Update the preference name with new name
        }
    )
    if res.status_code != 200:
        raise Exception(f"Error updating preference ID: {res.text}")
        
def delete_preference_id(
    preference_id: str, 
    nd_api_key: str
  ):
    url = f"https://not-diamond-server.onrender.com/v2/preferences/userPreferenceDelete/{preference_id}"
    headers = {
        "Authorization": f"Bearer {nd_api_key}"
    }
    res = requests.delete(
        url=url,
        headers=headers
    )
    if res.status_code != 200:
        raise Exception(f"Error deleting preference ID: {res.text}")

import axios from 'axios';

async function updatePreferenceId(
  preferenceId: string,
  ndApiKey: string,
  name?: string
): Promise<void> {
  const url = "https://not-diamond-server.onrender.com/v2/preferences/userPreferenceUpdate";
  const headers = {
    "Authorization": `Bearer ${ndApiKey}`
  };

  try {
    const response = await axios.post(url, {
      preference_id: preferenceId,
      name: name  // Update the preference name with new name
    }, { headers });

    if (response.status !== 200) {
      throw new Error(`Error updating preference ID: ${response.statusText}`);
    }
  } catch (error) {
    throw new Error(`Error updating preference ID: ${error.message}`);
  }
}

async function deletePreferenceId(
  preferenceId: string,
  ndApiKey: string
): Promise<void> {
  const url = "https://not-diamond-server.onrender.com/v2/preferences/userPreferenceDelete";
  const headers = {
    "Authorization": `Bearer ${ndApiKey}`
  };

  try {
    const response = await axios.post(url, {
      preference_id: preferenceId,
    }, { headers });

    if (response.status !== 200) {
      throw new Error(`Error deleting preference ID: ${response.statusText}`);
    }
  } catch (error) {
    throw new Error(`Error deleting preference ID: ${error.message}`);
  }
}

While we can view all of our preference IDs in the Not Diamond dashboard, if we expect to be generating large numbers of preference IDs (for example, if we're generating a distinct preference ID for each end user) we can store preference IDs within our database for easy lookup.