Puntos finales de IA

AI Endpoints: catálogo y playground

Compare y pruebe modelos open weight para texto, código, voz y visión, sin registrarse. Después, pase a producción con la misma API compatible con OpenAI.

Prueba otro modelo

Meta-Llama-3_3-70B-Instruct

Large Language Models (LLM)

Llama 3.3 es un modelo de lenguaje generativo ajustado por instrucciones optimizado para casos de uso de diálogo multilingüe. Fue lanzado por Meta AI el 6 de diciembre de 2024, y utiliza una arquitectura avanzada de transformador, además de estar diseñado para alinearse con las preferencias humanas en cuanto a utilidad y seguridad mediante ajuste supervisado (SFT) y aprendizaje por refuerzo con retroalimentación humana (RLHF). La fecha de corte de conocimiento para este modelo 70B es el 31 de diciembre de 2023.

Características soportadasLlamada a funciónStreaming

Formatos de salidaraw_textjson_objectjson_schema

Precios

0.67€/ Mtoken de entrada

0.67€/ Mtoken de salida

Publicado el

huggingface

Fecha de lanzamiento

6 dic 2024

Añadido al catálogo el

20 feb 2025

Tamaños de contexto

131k

Parámetros

70B

Licencia

Llama 3.3 Community

https://oai.endpoints.kepler.ai.cloud.ovh.net/v1

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

First install the requests library:

pip install requests

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

If you do not have an access token key yet, follow the instructions in the AI Endpoints – Getting Started.

Finally, run the following Python code:

import os
import requests
 
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {os.getenv('OVH_AI_ENDPOINTS_ACCESS_TOKEN')}",
}
 
# With /chat/completions
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions"
payload = {
    "max_tokens": 512,
    "messages": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    # Parse JSON response
    choices = response_data["choices"]
    for choice in choices:
        text = choice["message"]["content"]
        # Process text and finish_reason
        print(text)
else:
    print("Error:", response.status_code, response.text)

# With /responses
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/responses"
payload = {
    "max_output_tokens": 512,
    "input": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "store": False,
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    text = response_data["output"][0]["content"][0]["text"]
    print(text)
else:
    print("Error:", response.status_code, response.text)

With the Python OpenAI library

The Meta-Llama-3_3-70B-Instruct API is compatible with the OpenAI specification.

First install the openai library:

pip install openai

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

Finally, run the following Python code:

import os

from openai import OpenAI

url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1"

client = OpenAI(
    base_url=url,
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN")
)

def chat_completion(new_message: str) -> str:
    history_openai_format = [{"role": "user", "content": new_message}]

    return client.chat.completions.create(
        model="Meta-Llama-3_3-70B-Instruct",
        messages=history_openai_format,
        temperature=0,
        max_tokens=1024
    ).choices.pop().message.content

def responses(new_message: str) -> str:
    response = client.responses.create(
        model="Meta-Llama-3_3-70B-Instruct",
        input=new_message,
        temperature=0,
        max_output_tokens=1024,
        store=False
    )
    return response.output[0].content[0].text

if __name__ == '__main__':
    # With chat completion endpoint
    print(chat_completion("Explain gravity for a 6 years old"))
    # With responses endpoint
    print(responses("Explain gravity for a 12 years old"))

Batch Mode

The Batch API (/v1/batches) lets you submit a large number of inference requests in a single asynchronous job, instead of sending them one by one through our synchronous endpoints. Batch mode is ideal when you do not need an immediate answer but want to process a high volume of prompts (evaluations, offline labelling, content generation at scale, dataset preparation, etc...).

Batch mode requests are billed at 50% off compared to directly calling the generation endpoints (/v1/chat/completions or /v1/responses).

Results are delivered within a maximum of 24 hours after the batch is submitted.

The workflow follows the OpenAI Batch API specification:

Prepare a JSONL file where each line is one request, identified by a unique custom_id.
Upload that file to /v1/files with purpose=batch. The API returns a file ID.
Create the batch on /v1/batches, referencing the file ID and the target endpoint.
Poll the batch until its status is completed.
Download the output file: one JSON result per line, each having back its custom_id.

With the Python OpenAI library

Install the OpenAI library and export your access token:

pip install openai
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

The example below builds a JSONL file, runs the full batch lifecycle, and prints each response next to its custom_id:

import json
import os
import time

from openai import OpenAI

client = OpenAI(
    base_url="https://oai.endpoints.kepler.ai.cloud.ovh.net/v1",
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"),
)

# 1. Build the JSONL input file
prompts = [
    "What's the capital of France?",
    "What's the capital of Québec?",
]
with open("batch_input.jsonl", "w") as f:
    for i, prompt in enumerate(prompts):
        f.write(json.dumps({
            "custom_id": f"request-{i}",
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": {
                "model": "Meta-Llama-3_3-70B-Instruct",
                "messages": [{"role": "user", "content": prompt}],
                "max_tokens": 512,
            },
        }) + "\n")

# 2. Upload it
input_file = client.files.create(
    file=open("batch_input.jsonl", "rb"),
    purpose="batch",
)

# 3. Create the batch
batch = client.batches.create(
    input_file_id=input_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(f"Batch created: {batch.id}")

# 4. Poll until the batch reaches a terminal state
terminal = {"completed", "failed", "expired", "cancelled"}
while batch.status not in terminal:
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
    print(f"Status: {batch.status}")

# 5. Download and parse the results
if batch.status == "completed":
    output = client.files.content(batch.output_file_id).text
    for line in output.strip().splitlines():
        result = json.loads(line)
        custom_id = result["custom_id"]
        content = result["response"]["body"]["choices"][0]["message"]["content"]
        print(f"{custom_id}: {content}")
else:
    print(f"Batch finished with status: {batch.status}")

You can have more details on the dedicated Batch Mode documentation.

Model rate limit

When using AI Endpoints, the following rate limits apply:

Anonymous: 2 requests per minute, per IP and per model.
Authenticated with an API access key: 400 requests per minute, per Public Cloud project and per model.

If you exceed this limit, a 429 error code will be returned.

Going Further

Want to explore the full capabilities of the LLM API? Dive into our dedicated Structured Output and Function Calling guides.

For a broader overview of AI Endpoints, explore the full AI Endpoints Documentation.

Reach out to our support team or join the OVHcloud Discord #ai-endpoints channel to share your questions, feedback, and suggestions for improving the service, to the team and the community.

AI Endpoints: catálogo y playground

Meta-Llama-3_3-70B-Instruct

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

With the Python OpenAI library

Batch Mode

With the Python OpenAI library

Model rate limit

Going Further

Comienza con Meta-Llama-3_3-70B-Instruct

AI Endpoints - Getting Started

AI Endpoints - Features, Capabilities and Limitations

AI Endpoints - Troubleshooting

AI Endpoints - Structured Output

AI Endpoints - Function Calling

AI Endpoints - Using Virtual Models

Create your own Audio Summarizer Assistant

Build a Powerful Voice Assistant

Build a Python Chatbot with LangChain

Build a JavaScript Chatbot with LangChain

Create your own AI chatbot using LangChain4j

Create a Streaming Chatbot with LangChain4j

Enable Memory in your Chatbot using LangChain

Create a Memory Chatbot with LangChain4j

Build a RAG Chatbot with LangChain

Build a RAG Chatbot with LangChain4j

Using Structured Output with LangChain4j

Using Function Calling with LangChain4j

Model Context Protocol (MCP) with LangChain4j