AzureOpenAI

caution

You are currently on a page documenting the use of Azure OpenAI text completion models. The latest and most popular Azure OpenAI models are chat completion models.

Unless you are specifically using gpt-3.5-turbo-instruct, you are probably looking for this page instead.

info

Previously, LangChain.js supported integration with Azure OpenAI using the dedicated Azure OpenAI SDK. This SDK is now deprecated in favor of the new Azure integration in the OpenAI SDK, which allows to access the latest OpenAI models and features the same day they are released, and allows seemless transition between the OpenAI API and Azure OpenAI.

If you are using Azure OpenAI with the deprecated SDK, see the migration guide to update to the new API.

This will help you get started with AzureOpenAI completion models (LLMs) using LangChain. For detailed documentation on AzureOpenAI features and configuration options, please refer to the API reference.

Overview

Integration details

• TODO: Fill in table features.
• TODO: Remove JS support link if not relevant, otherwise ensure link is correct.
• TODO: Make sure API reference links are correct.

Class	Package	Local	Serializable	PY support	Package downloads	Package latest
AzureOpenAI	@langchain/openai	❌	✅	✅

Setup

To access AzureOpenAI models you’ll need to create an Azure account, get an API key, and install the @langchain/openai integration package.

Credentials

Head to azure.microsoft.com to sign up to AzureOpenAI and generate an API key.

You’ll also need to have an Azure OpenAI instance deployed. You can deploy a version on Azure Portal following this guide.

Once you have your instance running, make sure you have the name of your instance and key. You can find the key in the Azure Portal, under the “Keys and Endpoint” section of your instance.

If you’re using Node.js, you can define the following environment variables to use the service:

AZURE_OPENAI_API_INSTANCE_NAME=<YOUR_INSTANCE_NAME>
AZURE_OPENAI_API_DEPLOYMENT_NAME=<YOUR_DEPLOYMENT_NAME>
AZURE_OPENAI_API_KEY=<YOUR_KEY>
AZURE_OPENAI_API_VERSION="2024-02-01"

Alternatively, you can pass the values directly to the AzureOpenAI constructor.

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# export LANGCHAIN_TRACING_V2="true"
# export LANGCHAIN_API_KEY="your-api-key"

Installation

The LangChain AzureOpenAI integration lives in the @langchain/openai package:

tip

See this section for general instructions on installing integration packages.

npm

npm i @langchain/openai

yarn

yarn add @langchain/openai

pnpm

pnpm add @langchain/openai

Instantiation

Now we can instantiate our model object and generate chat completions:

import { AzureOpenAI } from "@langchain/openai";

const llm = new AzureOpenAI({
  model: "gpt-3.5-turbo-instruct",
  // azureOpenAIApiKey: "<your_key>", // In Node.js defaults to process.env.AZURE_OPENAI_API_KEY
  // azureOpenAIApiInstanceName: "<your_instance_name>", // In Node.js defaults to process.env.AZURE_OPENAI_API_INSTANCE_NAME
  // azureOpenAIApiDeploymentName: "<your_deployment_name>", // In Node.js defaults to process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME
  // azureOpenAIApiVersion: "<api_version>", // In Node.js defaults to process.env.AZURE_OPENAI_API_VERSION
  azureOpenAIApiKey: process.env.AZURE_OPENAI_API_KEY, // In Node.js defaults to
  // azureOpenAIApiInstanceName: process.env.AZURE_OPENAI_API_INSTANCE_NAME, // In Node.js defaults to
  azureOpenAIApiDeploymentName: process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME, // In Node.js defaults to
  azureOpenAIApiVersion: process.env.AZURE_OPENAI_API_VERSION, // In Node.js defaults to
  temperature: 0,
  maxTokens: undefined,
  timeout: undefined,
  maxRetries: 2,
  // other params...
});

Invocation

const inputText = "AzureOpenAI is an AI company that ";

const completion = await llm.invoke(inputText);
completion;

APIError: 410 The model associated with the deployment is deprecated and no longer available for use. Please refer to the Azure OpenAI service documentation for more information.
    at APIError.generate (/Users/bracesproul/code/lang-chain-ai/langchainjs/libs/langchain-openai/node_modules/openai/error.js:68:16)
    at AzureOpenAI.makeStatusError (/Users/bracesproul/code/lang-chain-ai/langchainjs/libs/langchain-openai/node_modules/openai/core.js:263:33)
    at AzureOpenAI.makeRequest (/Users/bracesproul/code/lang-chain-ai/langchainjs/libs/langchain-openai/node_modules/openai/core.js:306:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /Users/bracesproul/code/lang-chain-ai/langchainjs/libs/langchain-openai/dist/llms.cjs:517:29
    at async RetryOperation._fn (/Users/bracesproul/code/lang-chain-ai/langchainjs/node_modules/p-retry/index.js:50:12) {
  status: 410,
  headers: {
    'apim-request-id': '853eaf55-f6bf-4123-b9b2-523649c54544',
    'content-length': '213',
    'content-type': 'application/json',
    date: 'Thu, 01 Aug 2024 22:42:37 GMT',
    'strict-transport-security': 'max-age=31536000; includeSubDomains; preload',
    'x-content-type-options': 'nosniff',
    'x-ms-region': 'East US'
  },
  request_id: undefined,
  error: {
    code: 'ModelDeprecated',
    message: 'The model associated with the deployment is deprecated and no longer available for use. Please refer to the Azure OpenAI service documentation for more information.'
  },
  code: 'ModelDeprecated',
  param: undefined,
  type: undefined,
  attemptNumber: 3,
  retriesLeft: 0
}

Chaining

We can chain our completion model with a prompt template like so:

• TODO: Run cells so output can be seen.

import { PromptTemplate } from "@langchain/core/prompts";

const prompt = new PromptTemplate({
  template: "How to say {input} in {output_language}:\n",
});

const chain = prompt.pipe(llm);
await chain.invoke({
  output_language: "German",
  input: "I love programming.",
});

Using Azure Managed Identity

If you’re using Azure Managed Identity, you can configure the credentials like this:

import {
  DefaultAzureCredential,
  getBearerTokenProvider,
} from "@azure/identity";
import { AzureOpenAI } from "@langchain/openai";

const credentials = new DefaultAzureCredential();
const azureADTokenProvider = getBearerTokenProvider(
  credentials,
  "https://cognitiveservices.azure.com/.default"
);

const managedIdentityLLM = new AzureOpenAI({
  azureADTokenProvider,
  azureOpenAIApiInstanceName: "<your_instance_name>",
  azureOpenAIApiDeploymentName: "<your_deployment_name>",
  azureOpenAIApiVersion: "<api_version>",
});

Using a different domain

If your instance is hosted under a domain other than the default openai.azure.com, you’ll need to use the alternate AZURE_OPENAI_BASE_PATH environment variable. For example, here’s how you would connect to the domain https://westeurope.api.microsoft.com/openai/deployments/{DEPLOYMENT_NAME}:

import { AzureOpenAI } from "@langchain/openai";

const differentDomainLLM = new AzureOpenAI({
  azureOpenAIApiKey: "<your_key>", // In Node.js defaults to process.env.AZURE_OPENAI_API_KEY
  azureOpenAIApiDeploymentName: "<your_deployment_name>", // In Node.js defaults to process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME
  azureOpenAIApiVersion: "<api_version>", // In Node.js defaults to process.env.AZURE_OPENAI_API_VERSION
  azureOpenAIBasePath:
    "https://westeurope.api.microsoft.com/openai/deployments", // In Node.js defaults to process.env.AZURE_OPENAI_BASE_PATH
});

Migration from Azure OpenAI SDK

If you are using the deprecated Azure OpenAI SDK with the @langchain/azure-openai package, you can update your code to use the new Azure integration following these steps:

1. Install the new @langchain/openai package and remove the previous @langchain/azure-openai package:

   npm install @langchain/openai
   npm uninstall @langchain/azure-openai

2. Update your imports to use the new AzureOpenAI and AzureChatOpenAI classes from the @langchain/openai package:

   import { AzureOpenAI } from "@langchain/openai";

3. Update your code to use the new AzureOpenAI and AzureChatOpenAI classes and pass the required parameters:

   const model = new AzureOpenAI({
     azureOpenAIApiKey: "<your_key>",
     azureOpenAIApiInstanceName: "<your_instance_name>",
     azureOpenAIApiDeploymentName: "<your_deployment_name>",
     azureOpenAIApiVersion: "<api_version>",
   });

   Notice that the constructor now requires the azureOpenAIApiInstanceName parameter instead of the azureOpenAIEndpoint parameter, and adds the azureOpenAIApiVersion parameter to specify the API version.

   • If you were using Azure Managed Identity, you now need to use the azureADTokenProvider parameter to the constructor instead of credentials, see the Azure Managed Identity section for more details.

   • If you were using environment variables, you now have to set the AZURE_OPENAI_API_INSTANCE_NAME environment variable instead of AZURE_OPENAI_API_ENDPOINT, and add the AZURE_OPENAI_API_VERSION environment variable to specify the API version.

API reference

For detailed documentation of all AzureOpenAI features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_openai.AzureOpenAI.html