AI-eindpunten

AI Endpoints: catalogus en playground

Vergelijk en test open-weight-modellen voor tekst, code, spraak en beeld, zonder registratie. Ga daarna in productie met dezelfde OpenAI-compatibele API.

Probeer een ander model

Meta-Llama-3_3-70B-Instruct

Large Language Models (LLM)

Llama 3.3 is een instructie-geoptimaliseerd generatief taalmodel dat is geoptimaliseerd voor meertalige use cases met dialoog. Het werd uitgebracht door Meta AI op 6 december 2024 en maakt gebruik van een geavanceerde transformerarchitectuur en is ontworpen om zich aan te passen aan menselijke voorkeuren io gebied van behulpzaamheid en veiligheid door middel van bewaakte fine-tuning (SFT) en versterkend leren met menselijke feedback (RLHF). De kennis-aflsuitdatum voor dit 70B-model is 31 december 2023.

Ondersteunde functiesFunctieaanroepStreaming

Uitvoerformatenraw_textjson_objectjson_schema

Prijzen

0.67€/ Mtoken invoer

0.67€/ Mtoken uitvoer

Gepubliceerd op

huggingface

Releasedatum

6 dec 2024

Toegevoegd aan de catalogus op

20 feb 2025

Contextgroottes

131k

Parameters

70B

Licentie

Llama 3.3 Community

https://oai.endpoints.kepler.ai.cloud.ovh.net/v1

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

First install the requests library:

pip install requests

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

If you do not have an access token key yet, follow the instructions in the AI Endpoints – Getting Started.

Finally, run the following Python code:

import os
import requests
 
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {os.getenv('OVH_AI_ENDPOINTS_ACCESS_TOKEN')}",
}
 
# With /chat/completions
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions"
payload = {
    "max_tokens": 512,
    "messages": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    # Parse JSON response
    choices = response_data["choices"]
    for choice in choices:
        text = choice["message"]["content"]
        # Process text and finish_reason
        print(text)
else:
    print("Error:", response.status_code, response.text)

# With /responses
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/responses"
payload = {
    "max_output_tokens": 512,
    "input": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "store": False,
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    text = response_data["output"][0]["content"][0]["text"]
    print(text)
else:
    print("Error:", response.status_code, response.text)

With the Python OpenAI library

The Meta-Llama-3_3-70B-Instruct API is compatible with the OpenAI specification.

First install the openai library:

pip install openai

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

Finally, run the following Python code:

import os

from openai import OpenAI

url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1"

client = OpenAI(
    base_url=url,
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN")
)

def chat_completion(new_message: str) -> str:
    history_openai_format = [{"role": "user", "content": new_message}]

    return client.chat.completions.create(
        model="Meta-Llama-3_3-70B-Instruct",
        messages=history_openai_format,
        temperature=0,
        max_tokens=1024
    ).choices.pop().message.content

def responses(new_message: str) -> str:
    response = client.responses.create(
        model="Meta-Llama-3_3-70B-Instruct",
        input=new_message,
        temperature=0,
        max_output_tokens=1024,
        store=False
    )
    return response.output[0].content[0].text

if __name__ == '__main__':
    # With chat completion endpoint
    print(chat_completion("Explain gravity for a 6 years old"))
    # With responses endpoint
    print(responses("Explain gravity for a 12 years old"))

Batch Mode

The Batch API (/v1/batches) lets you submit a large number of inference requests in a single asynchronous job, instead of sending them one by one through our synchronous endpoints. Batch mode is ideal when you do not need an immediate answer but want to process a high volume of prompts (evaluations, offline labelling, content generation at scale, dataset preparation, etc...).

Batch mode requests are billed at 50% off compared to directly calling the generation endpoints (/v1/chat/completions or /v1/responses).

Results are delivered within a maximum of 24 hours after the batch is submitted.

The workflow follows the OpenAI Batch API specification:

Prepare a JSONL file where each line is one request, identified by a unique custom_id.
Upload that file to /v1/files with purpose=batch. The API returns a file ID.
Create the batch on /v1/batches, referencing the file ID and the target endpoint.
Poll the batch until its status is completed.
Download the output file: one JSON result per line, each having back its custom_id.

With the Python OpenAI library

Install the OpenAI library and export your access token:

pip install openai
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

The example below builds a JSONL file, runs the full batch lifecycle, and prints each response next to its custom_id:

import json
import os
import time

from openai import OpenAI

client = OpenAI(
    base_url="https://oai.endpoints.kepler.ai.cloud.ovh.net/v1",
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"),
)

# 1. Build the JSONL input file
prompts = [
    "What's the capital of France?",
    "What's the capital of Québec?",
]
with open("batch_input.jsonl", "w") as f:
    for i, prompt in enumerate(prompts):
        f.write(json.dumps({
            "custom_id": f"request-{i}",
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": {
                "model": "Meta-Llama-3_3-70B-Instruct",
                "messages": [{"role": "user", "content": prompt}],
                "max_tokens": 512,
            },
        }) + "\n")

# 2. Upload it
input_file = client.files.create(
    file=open("batch_input.jsonl", "rb"),
    purpose="batch",
)

# 3. Create the batch
batch = client.batches.create(
    input_file_id=input_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(f"Batch created: {batch.id}")

# 4. Poll until the batch reaches a terminal state
terminal = {"completed", "failed", "expired", "cancelled"}
while batch.status not in terminal:
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
    print(f"Status: {batch.status}")

# 5. Download and parse the results
if batch.status == "completed":
    output = client.files.content(batch.output_file_id).text
    for line in output.strip().splitlines():
        result = json.loads(line)
        custom_id = result["custom_id"]
        content = result["response"]["body"]["choices"][0]["message"]["content"]
        print(f"{custom_id}: {content}")
else:
    print(f"Batch finished with status: {batch.status}")

You can have more details on the dedicated Batch Mode documentation.

Model rate limit

When using AI Endpoints, the following rate limits apply:

Anonymous: 2 requests per minute, per IP and per model.
Authenticated with an API access key: 400 requests per minute, per Public Cloud project and per model.

If you exceed this limit, a 429 error code will be returned.

If you require higher usage, please get in touch with us to discuss increasing your rate limits.

Going Further

Want to explore the full capabilities of the LLM API? Dive into our dedicated Structured Output and Function Calling guides.

For a broader overview of AI Endpoints, explore the full AI Endpoints Documentation.

Reach out to our support team or join the OVHcloud Discord #ai-endpoints channel to share your questions, feedback, and suggestions for improving the service, to the team and the community.

AI Endpoints: catalogus en playground

Meta-Llama-3_3-70B-Instruct

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

With the Python OpenAI library

Batch Mode

With the Python OpenAI library

Model rate limit

Going Further

Begin met Meta-Llama-3_3-70B-Instruct

AI Endpoints - Getting Started

AI Endpoints - Features, Capabilities and Limitations

AI Endpoints - Troubleshooting

AI Endpoints - Structured Output

AI Endpoints - Function Calling

AI Endpoints - Using Virtual Models

Create your own Audio Summarizer Assistant

Build a Powerful Voice Assistant

Build a Python Chatbot with LangChain

Build a JavaScript Chatbot with LangChain

Create your own AI chatbot using LangChain4j

Create a Streaming Chatbot with LangChain4j

Enable Memory in your Chatbot using LangChain

Create a Memory Chatbot with LangChain4j

Build a RAG Chatbot with LangChain

Build a RAG Chatbot with LangChain4j

Using Structured Output with LangChain4j

Using Function Calling with LangChain4j

Model Context Protocol (MCP) with LangChain4j