AI Endpoints

Accédez facilement à des modèles d'IA pré-entraînés de renommée mondiale.
Innovez avec des API simples et sécurisées sur l'infrastructure robuste et confidentielle d'OVHcloud. Optimisez vos applications avec des capacités d'IA évolutives, éliminant le besoin d'une expertise approfondie. Gagnez en efficacité avec des AI Endpoints puissants, conçus pour la simplicité et la fiabilité.

Essayez un autre modèle

Meta-Llama-3_3-70B-Instruct

Large Language Models (LLM)

Llama 3.3 est un modèle de langage génératif ajusté pour les instructions, optimisé pour les cas d'usage de dialogue multilingue. Il a été publié par Meta AI le 6 décembre 2024 et utilise une architecture transformer avancée conçue pour s'aligner sur les préférences humaines en matière d'utilité et de sécurité grâce à l'ajustement fin supervisé (SFT) et à l'apprentissage par renforcement avec retour humain (RLHF). La limite de connaissance pour ce modèle de 70B est le 31 décembre 2023.

Fonctionnalités prises en chargeFunction callingStreaming

Formats de sortieraw_textjson_objectjson_schema

Tarification

0.67€/ Mtoken en entrée

0.67€/ Mtoken en sortie

Publié sur

huggingface

Date de sortie

6 déc. 2024

Ajouté au catalogue le

20 févr. 2025

Taille du contexte

131k

Paramètres

70B

Licence

Llama 3.3 Community

https://oai.endpoints.kepler.ai.cloud.ovh.net/v1

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

First install the requests library:

pip install requests

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

If you do not have an access token key yet, follow the instructions in the AI Endpoints – Getting Started.

Finally, run the following Python code:

import os
import requests
 
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {os.getenv('OVH_AI_ENDPOINTS_ACCESS_TOKEN')}",
}
 
# With /chat/completions
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions"
payload = {
    "max_tokens": 512,
    "messages": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    # Parse JSON response
    choices = response_data["choices"]
    for choice in choices:
        text = choice["message"]["content"]
        # Process text and finish_reason
        print(text)
else:
    print("Error:", response.status_code, response.text)

# With /responses
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/responses"
payload = {
    "max_output_tokens": 512,
    "input": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "store": False,
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    text = response_data["output"][0]["content"][0]["text"]
    print(text)
else:
    print("Error:", response.status_code, response.text)

With the Python OpenAI library

The Meta-Llama-3_3-70B-Instruct API is compatible with the OpenAI specification.

First install the openai library:

pip install openai

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

Finally, run the following Python code:

import os

from openai import OpenAI

url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1"

client = OpenAI(
    base_url=url,
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN")
)

def chat_completion(new_message: str) -> str:
    history_openai_format = [{"role": "user", "content": new_message}]

    return client.chat.completions.create(
        model="Meta-Llama-3_3-70B-Instruct",
        messages=history_openai_format,
        temperature=0,
        max_tokens=1024
    ).choices.pop().message.content

def responses(new_message: str) -> str:
    response = client.responses.create(
        model="Meta-Llama-3_3-70B-Instruct",
        input=new_message,
        temperature=0,
        max_output_tokens=1024,
        store=False
    )
    return response.output[0].content[0].text

if __name__ == '__main__':
    # With chat completion endpoint
    print(chat_completion("Explain gravity for a 6 years old"))
    # With responses endpoint
    print(responses("Explain gravity for a 12 years old"))

Batch Mode

The Batch API (/v1/batches) lets you submit a large number of inference requests in a single asynchronous job, instead of sending them one by one through our synchronous endpoints. Batch mode is ideal when you do not need an immediate answer but want to process a high volume of prompts (evaluations, offline labelling, content generation at scale, dataset preparation, etc...).

Batch mode requests are billed at 50% off compared to directly calling the generation endpoints (/v1/chat/completions or /v1/responses).

Results are delivered within a maximum of 24 hours after the batch is submitted.

The workflow follows the OpenAI Batch API specification:

Prepare a JSONL file where each line is one request, identified by a unique custom_id.
Upload that file to /v1/files with purpose=batch. The API returns a file ID.
Create the batch on /v1/batches, referencing the file ID and the target endpoint.
Poll the batch until its status is completed.
Download the output file: one JSON result per line, each having back its custom_id.

With the Python OpenAI library

Install the OpenAI library and export your access token:

pip install openai
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

The example below builds a JSONL file, runs the full batch lifecycle, and prints each response next to its custom_id:

import json
import os
import time

from openai import OpenAI

client = OpenAI(
    base_url="https://oai.endpoints.kepler.ai.cloud.ovh.net/v1",
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"),
)

# 1. Build the JSONL input file
prompts = [
    "What's the capital of France?",
    "What's the capital of Québec?",
]
with open("batch_input.jsonl", "w") as f:
    for i, prompt in enumerate(prompts):
        f.write(json.dumps({
            "custom_id": f"request-{i}",
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": {
                "model": "Meta-Llama-3_3-70B-Instruct",
                "messages": [{"role": "user", "content": prompt}],
                "max_tokens": 512,
            },
        }) + "\n")

# 2. Upload it
input_file = client.files.create(
    file=open("batch_input.jsonl", "rb"),
    purpose="batch",
)

# 3. Create the batch
batch = client.batches.create(
    input_file_id=input_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(f"Batch created: {batch.id}")

# 4. Poll until the batch reaches a terminal state
terminal = {"completed", "failed", "expired", "cancelled"}
while batch.status not in terminal:
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
    print(f"Status: {batch.status}")

# 5. Download and parse the results
if batch.status == "completed":
    output = client.files.content(batch.output_file_id).text
    for line in output.strip().splitlines():
        result = json.loads(line)
        custom_id = result["custom_id"]
        content = result["response"]["body"]["choices"][0]["message"]["content"]
        print(f"{custom_id}: {content}")
else:
    print(f"Batch finished with status: {batch.status}")

You can have more details on the dedicated Batch Mode documentation.

Model rate limit

When using AI Endpoints, the following rate limits apply:

Anonymous: 2 requests per minute, per IP and per model.
Authenticated with an API access key: 400 requests per minute, per Public Cloud project and per model.

If you exceed this limit, a 429 error code will be returned.

If you require higher usage, please get in touch with us to discuss increasing your rate limits.

Going Further

Want to explore the full capabilities of the LLM API? Dive into our dedicated Structured Output and Function Calling guides.

For a broader overview of AI Endpoints, explore the full AI Endpoints Documentation.

Reach out to our support team or join the OVHcloud Discord #ai-endpoints channel to share your questions, feedback, and suggestions for improving the service, to the team and the community.

AI Endpoints

Meta-Llama-3_3-70B-Instruct

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

With the Python OpenAI library

Batch Mode

With the Python OpenAI library

Model rate limit

Going Further

Commencez avec Meta-Llama-3_3-70B-Instruct

AI Endpoints - Getting Started

AI Endpoints - Features, Capabilities and Limitations

AI Endpoints - Troubleshooting

AI Endpoints - Structured Output

AI Endpoints - Function Calling

AI Endpoints - Using Virtual Models

Create your own Audio Summarizer Assistant

Build a Powerful Voice Assistant

Build a Python Chatbot with LangChain

Build a JavaScript Chatbot with LangChain

Create your own AI chatbot using LangChain4j

Create a Streaming Chatbot with LangChain4j

Enable Memory in your Chatbot using LangChain

Create a Memory Chatbot with LangChain4j

Build a RAG Chatbot with LangChain

Build a RAG Chatbot with LangChain4j

Using Structured Output with LangChain4j

Using Function Calling with LangChain4j

Model Context Protocol (MCP) with LangChain4j