Request predictions with Claude models

This guide shows how to send requests to Anthropic's Claude models using Vertex AI, covering the following topics:

• Before you begin: Describes the initial setup steps required to enable and use the Claude models.
• Choose an interaction method: Provides a comparison of the different ways to send requests to the models.
• Use the Anthropic SDK: Explains how to make API requests programmatically using the Anthropic SDK.
• Use a curl command: Shows how to send requests from the command line using curl.
• Tool use (function calling): Demonstrates how to enhance model capabilities with function calling.
• Use Vertex AI Studio: Details how to prototype and test models using the Google Cloud console UI.
• Anthropic Claude quotas and region availability: Lists the quotas and available regions for each model.

The following diagram summarizes the overall workflow:

Before you begin

To use the Anthropic Claude models with Vertex AI, perform the following steps. You must enable the Vertex AI API (aiplatform.googleapis.com) to use Vertex AI. If you already have a project with the Vertex AI API enabled, you can use that project instead of creating a new one.

Make sure that you have the required permissions to enable and use partner models. For more information, see Grant the required permissions.

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Vertex AI API.

Enable the API

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Vertex AI API.

Enable the API

1. Go to one of the following Model Garden model cards, then click Enable:
   • Go to the Claude Opus 4.1 model card
   • Go to the Claude Opus 4 model card
   • Go to the Claude Sonnet 4 model card
   • Go to the Claude 3.7 Sonnet model card
   • Go to the Claude 3.5 Sonnet v2 model card
   • Go to the Claude 3.5 Haiku model card
   • Go to the Claude 3.5 Sonnet model card
   • Go to the Claude 3 Opus model card
   • Go to the Claude 3 Haiku model card

Anthropic recommends that you enable 30-day logging of your prompt and completion activity to record any model misuse. To enable logging, see Log requests and responses.

Choose an interaction method

You can send requests to Claude models in several ways. The following table provides an overview of the available options to help you decide which is best for your use case.

You can use Anthropic's SDK or curl commands to send requests to the Vertex AI endpoint using the following model names:

• For Claude Opus 4.1, use claude-opus-4-1@20250805
• For Claude Opus 4, use claude-opus-4@20250514
• For Claude Sonnet 4, use claude-sonnet-4@20250514
• For Claude 3.7 Sonnet, use claude-3-7-sonnet@20250219
• For Claude 3.5 Sonnet v2, use claude-3-5-sonnet-v2@20241022
• For Claude 3.5 Haiku, use claude-3-5-haiku@20241022
• For Claude 3.5 Sonnet, use claude-3-5-sonnet@20240620
• For Claude 3 Opus, use claude-3-opus@20240229
• For Claude 3 Haiku, use claude-3-haiku@20240307

Anthropic Claude model versions must be used with a suffix that starts with an @ symbol (such as claude-3-7-sonnet@20250219 or claude-3-5-haiku@20241022) to guarantee consistent behavior.

Use the Anthropic SDK

You can make API requests to the Anthropic Claude models using the Anthropic Claude SDK. To learn more, see the following:

• Claude messages API reference
• Anthropic Python API library
• Anthropic Vertex AI TypeScript API Library

Make a streaming call to a Claude model using the Anthropic Vertex SDK

The following code sample uses the Anthropic Vertex SDK to perform a streaming call to a Claude model.

# TODO(developer): Vertex AI SDK - uncomment below & run
# pip3 install --upgrade --user google-cloud-aiplatform
# gcloud auth application-default login
# pip3 install -U 'anthropic[vertex]'

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"

from anthropic import AnthropicVertex

client = AnthropicVertex(project_id=PROJECT_ID, region="us-east5")
result = []

with client.messages.stream(
    model="claude-3-5-sonnet-v2@20241022",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Send me a recipe for banana bread.",
        }
    ],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
        result.append(text)

# Example response:
# Here's a simple recipe for delicious banana bread:
# Ingredients:
# - 2-3 ripe bananas, mashed
# - 1/3 cup melted butter
# ...
# ...
# 8. Bake for 50-60 minutes, or until a toothpick inserted into the center comes out clean.
# 9. Let cool in the pan for a few minutes, then remove and cool completely on a wire rack.

Make a unary call to a Claude model using the Anthropic Vertex SDK

The following code sample uses the Anthropic Vertex SDK to perform a unary call to a Claude model.

# TODO(developer): Vertex AI SDK - uncomment below & run
# pip3 install --upgrade --user google-cloud-aiplatform
# gcloud auth application-default login
# pip3 install -U 'anthropic[vertex]'

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"

from anthropic import AnthropicVertex

client = AnthropicVertex(project_id=PROJECT_ID, region="us-east5")
message = client.messages.create(
    model="claude-3-5-sonnet-v2@20241022",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Send me a recipe for banana bread.",
        }
    ],
)
print(message.model_dump_json(indent=2))
# Example response:
# {
#   "id": "msg_vrtx_0162rhgehxa9rvJM5BSVLZ9j",
#   "content": [
#     {
#       "text": "Here's a simple recipe for delicious banana bread:\n\nIngredients:\n- 2-3 ripe bananas...
#   ...

Use a curl command

You can use a curl command to send a request to the Vertex AI endpoint for a specific Claude model.

Anthropic Claude model versions must be used with a suffix that starts with an @ symbol (such as claude-3-7-sonnet@20250219 or claude-3-5-haiku@20241022) to guarantee consistent behavior.

The following section shows you how to create a curl command and includes a sample curl command.

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:streamRawPredict

{
  "anthropic_version": "vertex-2023-10-16",
  "messages": [
   {
    "role": "ROLE",
    "content": "CONTENT"
   }],
  "max_tokens": MAX_TOKENS,
  "stream": STREAM,
  "thinking": {
    "type": "TYPE",
    "budget_tokens": BUDGET_TOKENS
  }
}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:streamRawPredict"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:streamRawPredict" | Select-Object -Expand Content

{
  "id":"msg_012NDLxqh6LsztWCU7zTb14C",
  "type":"message",
  "role":"assistant",
  "content":[{
    "type":"text",
    "text":"Hello! Nice to meet you."
  }],
  "model":"claude-2.1",
  "stop_reason":"end_turn",
  "stop_sequence":null,
  "usage":{
    "input_tokens":11,
    "output_tokens":11
  }
}

MODEL_ID="MODEL"
LOCATION="us-central1"
PROJECT_ID="PROJECT_ID"

curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/anthropic/models/${MODEL_ID}:streamRawPredict -d \
'{
  "anthropic_version": "vertex-2023-10-16",
  "messages": [{
    "role": "user",
    "content": "Hello!"
  }],
  "max_tokens": 50,
  "stream": true}'

Tool use (function calling)

The Anthropic Claude models support tool use and function calling to enhance a model's capabilities. For more information, see the Tool use overview in the Anthropic documentation.

The following samples demonstrate how to use tools with an SDK or a curl command. The samples search for nearby restaurants in San Francisco that are open.

# TODO(developer): Vertex AI SDK - uncomment below & run
# pip3 install --upgrade --user google-cloud-aiplatform
# gcloud auth application-default login
# pip3 install -U 'anthropic[vertex]'
from anthropic import AnthropicVertex

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"

client = AnthropicVertex(project_id=PROJECT_ID, region="us-east5")
message = client.messages.create(
    model="claude-3-5-sonnet-v2@20241022",
    max_tokens=1024,
    tools=[
        {
            "name": "text_search_places_api",
            "description": "returns information about a set of places based on a string",
            "input_schema": {
                "type": "object",
                "properties": {
                    "textQuery": {
                        "type": "string",
                        "description": "The text string on which to search",
                    },
                    "priceLevels": {
                        "type": "array",
                        "description": "Price levels to query places, value can be one of [PRICE_LEVEL_INEXPENSIVE, PRICE_LEVEL_MODERATE, PRICE_LEVEL_EXPENSIVE, PRICE_LEVEL_VERY_EXPENSIVE]",
                    },
                    "openNow": {
                        "type": "boolean",
                        "description": "whether those places are open for business.",
                    },
                },
                "required": ["textQuery"],
            },
        }
    ],
    messages=[
        {
            "role": "user",
            "content": "What are some affordable and good Italian restaurants open now in San Francisco??",
        }
    ],
)
print(message.model_dump_json(indent=2))
# Example response:
# {
#   "id": "msg_vrtx_018pk1ykbbxAYhyWUdP1bJoQ",
#   "content": [
#     {
#       "text": "To answer your question about affordable and good Italian restaurants
#       that are currently open in San Francisco....
# ...

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:rawPredict

{
  "anthropic_version": "vertex-2023-10-16",
  "max_tokens": MAX_TOKENS,
  "stream": STREAM,
  "tools": [
    {
      "name": "text_search_places_api",
      "description": "Returns information about a set of places based on a string",
      "input_schema": {
        "type": "object",
        "properties": {
          "textQuery": {
            "type": "string",
            "description": "The text string on which to search"
          },
          "priceLevels": {
            "type": "array",
            "description": "Price levels to query places, value can be one of [PRICE_LEVEL_INEXPENSIVE, PRICE_LEVEL_MODERATE, PRICE_LEVEL_EXPENSIVE, PRICE_LEVEL_VERY_EXPENSIVE]",
          },
          "openNow": {
            "type": "boolean",
            "description": "Describes whether a place is open for business at
            the time of the query."
          },
        },
        "required": ["textQuery"]
      }
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": "What are some affordable and good Italian restaurants that are open now in San Francisco??"
    }
  ]
}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:rawPredict"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:rawPredict" | Select-Object -Expand Content

{
  "id": "msg_vrtx_01ErR7VMNQdnvDt3n7Nmc4ER",
  "type": "message",
  "role": "assistant",
  "model": "claude-3-opus-20240229",
  "content": [
    {
      "type": "text",
      "text": "\nTo find affordable and good Italian restaurants that are currently open in San Francisco, the text_search_places_api tool seems most relevant. \n\nThe required textQuery parameter can be inferred as \"Italian restaurants in San Francisco\", since the user specified Italian restaurants and the ___location of San Francisco.\n\nTwo optional parameters are also relevant:\nopenNow - this should be set to true, since the user specified they want restaurants open now\npriceLevels - to find affordable restaurants, this can be set to [PRICE_LEVEL_INEXPENSIVE, PRICE_LEVEL_MODERATE]\n\nWith the textQuery provided and the two optional parameters that can help narrow the results to match the user's criteria, we have enough information to make a good call to the text_search_places_api tool to try to answer the user's request.\n"
    },
    {
      "type": "tool_use",
      "id": "toolu_vrtx_01TAJCTkxe8HhRoaQ69N4ouP",
      "name": "text_search_places_api",
      "input": {
        "textQuery": "Italian restaurants in San Francisco",
        "openNow": true,
        "priceLevels": [
          "PRICE_LEVEL_INEXPENSIVE",
          "PRICE_LEVEL_MODERATE"
        ]
      }
    }
  ],
  "stop_reason": "tool_use",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 727,
    "output_tokens": 308
  }
}

Use Vertex AI Studio

For some of the Anthropic Claude models, you can use Vertex AI Studio to prototype and test generative AI models in the Google Cloud console. As an example, you can use Vertex AI Studio to compare Claude model responses with other supported models such as Google Gemini.

For more information, see Quickstart: Send text prompts to Gemini using Vertex AI Studio.

Anthropic Claude quotas and region availability

Claude models have regional quotas and, for models that support a global endpoint, a global quota. The quota is specified in queries per minute (QPM) and tokens per minute (TPM). TPM includes both input and output tokens.

Maximum quotas can vary by account to maintain service performance and ensure acceptable use. In some cases, access might be restricted. To view your project's quotas, see the Quotas & Systems Limits page in the Google Cloud console. You also need the following quotas:

• online_prediction_requests_per_base_model and global_online_prediction_requests_per_base_model defines your QPM quota.

• For TPM, there are three quota values that apply to certain models:

  • For models that count input and output tokens together, online_prediction_tokens_per_minute_per_base_model and global_online_prediction_tokens_per_minute_per_base_model defines the model TPM quota.
  • For models that count input and output tokens separately, online_prediction_input_tokens_per_minute_per_base_model and global_online_prediction_input_tokens_per_minute_per_base_model defines the input TPM quota and online_prediction_output_tokens_per_minute_per_base_model and global_online_prediction_output_tokens_per_minute_per_base_model defines the output TPM quota.

  To see which models count input and output tokens separately, see Quotas by model and region.

Input

The following list defines the input tokens that can count towards your input TPM quota. The input tokens that each model counts can vary. To see which input tokens a model counts, see Quotas by model and region.

• Input tokens includes all input tokens, including cache read and cache write tokens.
• Uncached input tokens includes only the input tokens that weren't read from a cache (cache read tokens).
• Cache write tokens includes tokens that were used to create or update a cache.

Quotas by model and region

The following table shows the default quotas and supported context length for each model in each region.

If you want to increase any of your quotas for Generative AI on Vertex AI, you can use the Google Cloud console to request a quota increase. To learn more about quotas, see Work with quotas.