Quickstart - Using the ChatGPT API - Azure OpenAI Service (2023)

Prerequisites

• An Azure subscription - Create one for free.

• Access granted to Azure OpenAI in the desired Azure subscription.

  Currently, access to this service is granted only by application. You can apply for access to Azure OpenAI by completing the form at https://aka.ms/oai/access. Open an issue on this repo to contact us if you have an issue.

• An Azure OpenAI Service resource with the gpt-35-turbo model deployed. This model is currently available in East US and South Central US. For more information about model deployment, see the resource deployment guide.

Go to Azure OpenAI Studio

Navigate to Azure OpenAI Studio at https://oai.azure.com/ and sign-in with credentials that have access to your OpenAI resource. During or after the sign-in workflow, select the appropriate directory, Azure subscription, and Azure OpenAI resource.

From the Azure OpenAI Studio landing page, select ChatGPT playground (Preview)

Playground

Start exploring OpenAI capabilities with a no-code approach through the Azure OpenAI Studio ChatGPT playground. From this page, you can quickly iterate and experiment with the capabilities.

Assistant setup

You can use the Assistant setup dropdown to select a few pre-loaded System message examples to get started.

System messages give the model instructions about how it should behave and any context it should reference when generating a response. You can describe the assistant's personality, tell it what it should and shouldn't answer, and tell it how to format responses.

Add few-shot examples allows you to provide conversational examples that are used by the model for in-context learning.

At any time while using the ChatGPT playground you can select View code to see Python, curl, and json code samples pre-populated based on your current chat session and settings selections. You can then take this code and write an application to complete the same task you're currently performing with the playground.

Chat session

Selecting the Send button sends the entered text to the completions API and the results are returned back to the text box.

Select the Clear chat button to delete the current conversation history.

Settings

Show panels

By default there are three panels: assistant setup, chat session, and settings. Show panels allows you to add, remove, and rearrange the panels. If you ever close a panel and need to get it back, use Show panels to restore the lost panel.

Start a chat session

1. From the assistant setup drop-down, select Xbox customer support agent

2. You'll be prompted asking if you want to update the system message, select Continue.

3. In the chat session pane, enter the following question: "I'm interested in buying a new Xbox", and select Send.

4. You'll receive a response similar to:

   (Video) Using ChatGPT API in Microsoft Azure

5. Enter a follow-up question like: "What models are available?"

6. Now that you have a basic conversation select View code from under Assistant setup and you'll have a replay of the code behind the entire conversation so far:

#Note: The openai-python library support for Azure OpenAI is in preview.import osimport openaiopenai.api_type = "azure"openai.api_base = "https://docs-test-001.openai.azure.com/"openai.api_version = "2022-12-01"openai.api_key = os.getenv("OPENAI_API_KEY")response = openai.Completion.create( engine="gpt-35-turbo", prompt="<|im_start|>system\nYou are an Xbox customer support agent whose primary goal is to help users with issues they are experiencing with their Xbox devices. You are friendly and concise. You only provide factual answers to queries, and do not provide answers that are not related to Xbox.\n<|im_end|>\n<|im_start|>user\nI am interested in buying an Xbox?\n<|im_end|>\n<|im_start|>assistant\nGreat! Xbox is a popular gaming console that offers a wide variety of games and features. You can purchase an Xbox from various retailers such as Amazon, Best Buy, and the Microsoft Store. Additionally, you can also purchase an Xbox bundle that includes games and accessories. If you have any specific questions about the Xbox or the purchasing process, please let me know and I'll be happy to assist you.\n\n<|im_end|>\n<|im_start|>user\nWhat models are available?\n<|im_end|>\n<|im_start|>assistant\nCurrently, there are three Xbox models available for purchase: Xbox Series X, Xbox Series S, and Xbox One X. \n\nThe Xbox Series X is the most powerful Xbox console to date, with a custom-designed processor, 12 teraflops of processing power, and 16GB of memory. It also supports 4K gaming and has a built-in 4K Ultra HD Blu-ray player.\n\nThe Xbox Series S is a more affordable option, with a custom-\n<|im_end|>\n", temperature=0, max_tokens=100, top_p=0.95, frequency_penalty=0, presence_penalty=0, best_of=1, stop=["<|im_end|>"])

Understanding the prompt structure

If you examine the sample from View code you'll notice some unique tokens that weren't part of a typical GPT completion call. ChatGPT was trained to use special tokens to delineate different parts of the prompt. Content is provided to the model in between <|im_start|> and <|im_end|> tokens. The prompt begins with a system message that can be used to prime the model by including context or instructions for the model. After that, the prompt contains a series of messages between the user and the assistant.

The assistant's response to the prompt will then be returned below the <|im_start|>assistant token and will end with <|im_end|> denoting that the assistant has finished its response. You can also use the Show raw syntax toggle button to display these tokens within the chat session panel.

The ChatGPT how-to guide provides an in-depth introduction into the new prompt structure and how to use the gpt-35-turbo model effectively.

Clean up resources

Once you're done testing out the ChatGPT playground, if you want to clean up and remove an OpenAI resource, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it.

• Portal
• Azure CLI

Next steps

• Learn more about how to work with ChatGPT and the new gpt-35-turbo model with the ChatGPT how-to guide.
• For more examples check out the Azure OpenAI Samples GitHub repository

Library source code | Package (PyPi) |

Prerequisites

• An Azure subscription - Create one for free
• Access granted to the Azure OpenAI service in the desired Azure subscription.Currently, access to this service is granted only by application. You can apply for access to Azure OpenAI Service by completing the form at https://aka.ms/oai/access.
• Python 3.7.1 or later version.
• The following Python libraries: os.
• An Azure OpenAI Service resource with the gpt-35-turbo model deployed. This model is currently available in East US and South Central US. For more information about model deployment, see the resource deployment guide.

Set up

Install the OpenAI Python client library with:

pip install openai

Retrieve key and endpoint

To successfully make a call against Azure OpenAI, you'll need an endpoint and a key.

Go to your resource in the Azure portal. The Endpoint and Keys can be found in the Resource Management section. Copy your endpoint and access key as you'll need both for authenticating your API calls. You can use either KEY1 or KEY2. Always having two keys allows you to securely rotate and regenerate keys without causing a service disruption.

Create and assign persistent environment variables for your key and endpoint.

Environment variables

setx OPENAI_API_KEY "REPLACE_WITH_YOUR_KEY_VALUE_HERE" 

setx OPENAI_API_BASE "REPLACE_WITH_YOUR_ENDPOINT_HERE" 

Create a new Python application

1. Create a new Python file called quickstart.py. Then open it up in your preferred editor or IDE.

2. Replace the contents of quickstart.py with the following code:

   #Note: The openai-python library support for Azure OpenAI is in preview.import osimport openaiopenai.api_type = "azure"openai.api_base = os.getenv("OPENAI_API_BASE") openai.api_version = "2022-12-01"openai.api_key = os.getenv("OPENAI_API_KEY")response = openai.Completion.create( engine="gpt-35-turbo", prompt="<|im_start|>system\nThe system is an AI assistant that helps people find information.\n<|im_end|>\n<|im_start|>user\nDoes Azure OpenAI support customer managed keys?\n<|im_end|>\n<|im_start|>assistant", temperature=1, max_tokens=800, top_p=0.95, frequency_penalty=0, presence_penalty=0, stop=["<|im_end|>"])print(response['choices'][0]['text'])

3. Run the application with the python command on your quickstart file:

   python quickstart.py

Output

Yes, Azure OpenAI supports customer managed keys, which allow customers to control access to their data and models. Additionally, Azure OpenAI supports Azure Key Vault integration which allows customers to manage and control access to keys and other secrets in Azure.'

Understanding the prompt structure

ChatGPT was trained to use special tokens to delineate different parts of the prompt. Content is provided to the model in between <|im_start|> and <|im_end|> tokens. The prompt begins with a system message which can be used to prime the model by including context or instructions for the model. After that, the prompt contains a series of messages between the user and the assistant.

The assistant's response to the prompt will then be returned below the <|im_start|>assistant token and will end with <|im_end|> denoting that the assistant has finished its response.

The ChatGPT how-to guide provides an in-depth introduction into the new prompt structure and how to use the new model effectively.

Clean up resources

If you want to clean up and remove an OpenAI resource, you can delete the resource. Before deleting the resource you must first delete any deployed models.

• Portal
• Azure CLI

Next steps

• Learn more about how to work with ChatGPT and the new gpt-35-turbo model with the ChatGPT how-to guide.
• For more examples check out the Azure OpenAI Samples GitHub repository

REST API Spec |

Prerequisites

• An Azure subscription - Create one for free.
• Access granted to the Azure OpenAI service in the desired Azure subscription.Currently, access to this service is granted only by application. You can apply for access to Azure OpenAI Service by completing the form at https://aka.ms/oai/access.
• An Azure OpenAI Service resource with the gpt-35-turbo model deployed. This model is currently available in East US and South Central US. For more information about model deployment, see the resource deployment guide.

Set up

Retrieve key and endpoint

To successfully make a call against Azure OpenAI, you'll need an endpoint and a key.

Go to your resource in the Azure portal. The Endpoint and Keys can be found in the Resource Management section. Copy your endpoint and access key as you'll need both for authenticating your API calls. You can use either KEY1 or KEY2. Always having two keys allows you to securely rotate and regenerate keys without causing a service disruption.

Create and assign persistent environment variables for your key and endpoint.

Environment variables

setx OPENAI_API_KEY "REPLACE_WITH_YOUR_KEY_VALUE_HERE" 

setx OPENAI_API_BASE "REPLACE_WITH_YOUR_ENDPOINT_HERE" 

REST API

In a bash shell run the following:

curl https://$OPENAI_API_BASE/openai/deployments/gpt-35-turbo/completions?api-version=2022-12-01 \ -H "Content-Type: application/json" \ -H "api-key: $OPENAI_API_KEY" \ -d '{ "prompt": "<|im_start|>system\nThe system is an AI assistant that helps people find information.\n<|im_end|>\n<|im_start|>user\nDoes Azure OpenAI support customer managed keys?\n<|im_end|>\n<|im_start|>assistant", "max_tokens": 800, "temperature": 1, "frequency_penalty": 0, "presence_penalty": 0, "top_p": 0.95, "stop": ["<|im_end|>"]}'

Output

{"id":"cmpl-6mZPEDkBPasCTxueCy9iVRMY4ZGD4","object":"text_completion","created":1677033864,"model":"gpt-35-turbo","choices":[{"text":"\nYes, Azure OpenAI supports customer managed keys. These keys allow customers to manage their own encryption keys for the OpenAI services, rather than relying on Azure's managed keys. This provides an additional layer of security for customers' data and models.","index":0,"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":66,"completion_tokens":52,"total_tokens":118}}

Output formatting adjusted for readability, actual output is a single block of text without line breaks.

Understanding the prompt structure

ChatGPT was trained to use special tokens to delineate different parts of the prompt. Content is provided to the model in between <|im_start|> and <|im_end|> tokens. The prompt begins with a system message which can be used to prime the model by including context or instructions for the model. After that, the prompt contains a series of messages between the user and the assistant.

The assistant's response to the prompt will then be returned below the <|im_start|>assistant token and will end with <|im_end|> denoting that the assistant has finished its response.

The ChatGPT how-to guide provides an in-depth introduction into the new prompt structure and how to use the new model effectively.

Clean up resources

If you want to clean up and remove an OpenAI resource, you can delete the resource. Before deleting the resource you must first delete any deployed models.

• Portal
• Azure CLI

Next steps

• Learn more about how to work with ChatGPT and the new gpt-35-turbo model in the how-to guide on ChatGPT.
• For more examples check out the Azure OpenAI Samples GitHub repository