AI Endpoints

AI Endpoints: catalogo e playground

Confronta e prova modelli open weight per testo, codice, voce e visione, senza registrazione. Poi passa in produzione con la stessa API compatibile con OpenAI.

Prova un altro modello

Meta-Llama-3_3-70B-Instruct

Large Language Models (LLM)

Llama 3.3 è un modello di linguaggio generativo ottimizzato con instruction tuning per casi d'uso di dialogo multilingue. È stato rilasciato da Meta AI il 6 dicembre 2024, utilizza un'architettura trasformer avanzata ed è progettato per allinearsi alle preferenze umane in termini di utilità e sicurezza tramite addestramento supervisionato (Supervised Fine-Tuning, SFT) e apprendimento per rinforzo da feedback umano (Reinforcement Learning from Human Feedback, RLHF). La data di aggiornamento delle conoscenze di questo modello 70B è il 31 dicembre 2023.

Funzionalità supportateChiamata di funzioneStreaming

Formati di outputraw_textjson_objectjson_schema

Prezzi

0.67€/ Mtoken in ingresso

0.67€/ Mtoken in uscita

Pubblicato il

huggingface

Data di rilascio

6 dic 2024

Aggiunto al catalogo il

20 feb 2025

Dimensioni del contesto

131k

Parametri

70B

Licenza

Llama 3.3 Community

https://oai.endpoints.kepler.ai.cloud.ovh.net/v1

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

First install the requests library:

pip install requests

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

If you do not have an access token key yet, follow the instructions in the AI Endpoints – Getting Started.

Finally, run the following Python code:

import os
import requests
 
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {os.getenv('OVH_AI_ENDPOINTS_ACCESS_TOKEN')}",
}
 
# With /chat/completions
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions"
payload = {
    "max_tokens": 512,
    "messages": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    # Parse JSON response
    choices = response_data["choices"]
    for choice in choices:
        text = choice["message"]["content"]
        # Process text and finish_reason
        print(text)
else:
    print("Error:", response.status_code, response.text)

# With /responses
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/responses"
payload = {
    "max_output_tokens": 512,
    "input": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "store": False,
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    text = response_data["output"][0]["content"][0]["text"]
    print(text)
else:
    print("Error:", response.status_code, response.text)

With the Python OpenAI library

The Meta-Llama-3_3-70B-Instruct API is compatible with the OpenAI specification.

First install the openai library:

pip install openai

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

Finally, run the following Python code:

import os

from openai import OpenAI

url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1"

client = OpenAI(
    base_url=url,
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN")
)

def chat_completion(new_message: str) -> str:
    history_openai_format = [{"role": "user", "content": new_message}]

    return client.chat.completions.create(
        model="Meta-Llama-3_3-70B-Instruct",
        messages=history_openai_format,
        temperature=0,
        max_tokens=1024
    ).choices.pop().message.content

def responses(new_message: str) -> str:
    response = client.responses.create(
        model="Meta-Llama-3_3-70B-Instruct",
        input=new_message,
        temperature=0,
        max_output_tokens=1024,
        store=False
    )
    return response.output[0].content[0].text

if __name__ == '__main__':
    # With chat completion endpoint
    print(chat_completion("Explain gravity for a 6 years old"))
    # With responses endpoint
    print(responses("Explain gravity for a 12 years old"))

Batch Mode

The Batch API (/v1/batches) lets you submit a large number of inference requests in a single asynchronous job, instead of sending them one by one through our synchronous endpoints. Batch mode is ideal when you do not need an immediate answer but want to process a high volume of prompts (evaluations, offline labelling, content generation at scale, dataset preparation, etc...).

Batch mode requests are billed at 50% off compared to directly calling the generation endpoints (/v1/chat/completions or /v1/responses).

Results are delivered within a maximum of 24 hours after the batch is submitted.

The workflow follows the OpenAI Batch API specification:

Prepare a JSONL file where each line is one request, identified by a unique custom_id.
Upload that file to /v1/files with purpose=batch. The API returns a file ID.
Create the batch on /v1/batches, referencing the file ID and the target endpoint.
Poll the batch until its status is completed.
Download the output file: one JSON result per line, each having back its custom_id.

With the Python OpenAI library

Install the OpenAI library and export your access token:

pip install openai
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

The example below builds a JSONL file, runs the full batch lifecycle, and prints each response next to its custom_id:

import json
import os
import time

from openai import OpenAI

client = OpenAI(
    base_url="https://oai.endpoints.kepler.ai.cloud.ovh.net/v1",
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"),
)

# 1. Build the JSONL input file
prompts = [
    "What's the capital of France?",
    "What's the capital of Québec?",
]
with open("batch_input.jsonl", "w") as f:
    for i, prompt in enumerate(prompts):
        f.write(json.dumps({
            "custom_id": f"request-{i}",
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": {
                "model": "Meta-Llama-3_3-70B-Instruct",
                "messages": [{"role": "user", "content": prompt}],
                "max_tokens": 512,
            },
        }) + "\n")

# 2. Upload it
input_file = client.files.create(
    file=open("batch_input.jsonl", "rb"),
    purpose="batch",
)

# 3. Create the batch
batch = client.batches.create(
    input_file_id=input_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(f"Batch created: {batch.id}")

# 4. Poll until the batch reaches a terminal state
terminal = {"completed", "failed", "expired", "cancelled"}
while batch.status not in terminal:
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
    print(f"Status: {batch.status}")

# 5. Download and parse the results
if batch.status == "completed":
    output = client.files.content(batch.output_file_id).text
    for line in output.strip().splitlines():
        result = json.loads(line)
        custom_id = result["custom_id"]
        content = result["response"]["body"]["choices"][0]["message"]["content"]
        print(f"{custom_id}: {content}")
else:
    print(f"Batch finished with status: {batch.status}")

You can have more details on the dedicated Batch Mode documentation.

Model rate limit

When using AI Endpoints, the following rate limits apply:

Anonymous: 2 requests per minute, per IP and per model.
Authenticated with an API access key: 400 requests per minute, per Public Cloud project and per model.

If you exceed this limit, a 429 error code will be returned.

If you require higher usage, please get in touch with us to discuss increasing your rate limits.

Going Further

Want to explore the full capabilities of the LLM API? Dive into our dedicated Structured Output and Function Calling guides.

For a broader overview of AI Endpoints, explore the full AI Endpoints Documentation.

Reach out to our support team or join the OVHcloud Discord #ai-endpoints channel to share your questions, feedback, and suggestions for improving the service, to the team and the community.

AI Endpoints: catalogo e playground

Meta-Llama-3_3-70B-Instruct

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

With the Python OpenAI library

Batch Mode

With the Python OpenAI library

Model rate limit

Going Further

Inizia con Meta-Llama-3_3-70B-Instruct

AI Endpoints - Getting Started

AI Endpoints - Features, Capabilities and Limitations

AI Endpoints - Troubleshooting

AI Endpoints - Structured Output

AI Endpoints - Function Calling

AI Endpoints - Using Virtual Models

Create your own Audio Summarizer Assistant

Build a Powerful Voice Assistant

Build a Python Chatbot with LangChain

Build a JavaScript Chatbot with LangChain

Create your own AI chatbot using LangChain4j

Create a Streaming Chatbot with LangChain4j

Enable Memory in your Chatbot using LangChain

Create a Memory Chatbot with LangChain4j

Build a RAG Chatbot with LangChain

Build a RAG Chatbot with LangChain4j

Using Structured Output with LangChain4j

Using Function Calling with LangChain4j

Model Context Protocol (MCP) with LangChain4j