Meta-Llama-3_3-70B-Instruct

Large Language Models (LLM)

Llama 3.3 es un modelo de lenguaje generativo ajustado por instrucciones optimizado para casos de uso de diálogo multilingüe. Fue lanzado por Meta AI el 6 de diciembre de 2024, y utiliza una arquitectura avanzada de transformador, además de estar diseñado para alinearse con las preferencias humanas en cuanto a utilidad y seguridad mediante ajuste supervisado (SFT) y aprendizaje por refuerzo con retroalimentación humana (RLHF). La fecha de corte de conocimiento para este modelo 70B es el 31 de diciembre de 2023.

Prueba otro modelo

Acerca del modelo Meta-Llama-3_3-70B-Instruct

Publicado el huggingface

06/12/2024

Licencia: Llama 3.3 Community

Precio de entrada

0.67 € /Mtoken(entrada)

Precio de salida

0.67 € /Mtoken(salida)

URL de la API

https://llama-3-3-70b-instruct.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/

Características soportadas

Llamada a funciónStreaming

Formatos de salida

raw_textjson_objectjson_schema

Tamaños de contexto

131k

Parámetros

70B

Prueba el modelo jugando con él.

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

First install the requests library:

pip install requests

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

If you do not have an access token key yet, follow the instructions in the AI Endpoints – Getting Started.

Finally, run the following Python code:

import os
import requests
 
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {os.getenv('OVH_AI_ENDPOINTS_ACCESS_TOKEN')}",
}
 
# With /chat/completions
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions"
payload = {
    "max_tokens": 512,
    "messages": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    # Parse JSON response
    choices = response_data["choices"]
    for choice in choices:
        text = choice["message"]["content"]
        # Process text and finish_reason
        print(text)
else:
    print("Error:", response.status_code, response.text)

# With /responses
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/responses"
payload = {
    "max_output_tokens": 512,
    "input": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "store": False,
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    text = response_data["output"][0]["content"][0]["text"]
    print(text)
else:
    print("Error:", response.status_code, response.text)

With the Python OpenAI library

The Meta-Llama-3_3-70B-Instruct API is compatible with the OpenAI specification.

First install the openai library:

pip install openai

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

Finally, run the following Python code:

import os

from openai import OpenAI

url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1"

client = OpenAI(
    base_url=url,
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN")
)

def chat_completion(new_message: str) -> str:
    history_openai_format = [{"role": "user", "content": new_message}]

    return client.chat.completions.create(
        model="Meta-Llama-3_3-70B-Instruct",
        messages=history_openai_format,
        temperature=0,
        max_tokens=1024
    ).choices.pop().message.content

def responses(new_message: str) -> str:
    response = client.responses.create(
        model="Meta-Llama-3_3-70B-Instruct",
        input=new_message,
        temperature=0,
        max_output_tokens=1024,
        store=False
    )
    return response.output[0].content[0].text

if __name__ == '__main__':
    # With chat completion endpoint
    print(chat_completion("Explain gravity for a 6 years old"))
    # With responses endpoint
    print(responses("Explain gravity for a 12 years old"))

Batch Mode

The Batch API (/v1/batches) lets you submit a large number of inference requests in a single asynchronous job, instead of sending them one by one through our synchronous endpoints. Batch mode is ideal when you do not need an immediate answer but want to process a high volume of prompts (evaluations, offline labelling, content generation at scale, dataset preparation, etc...).

Batch mode requests are billed at 50% off compared to directly calling the generation endpoints (/v1/chat/completions or /v1/responses).

Results are delivered within a maximum of 24 hours after the batch is submitted.

The workflow follows the OpenAI Batch API specification:

Prepare a JSONL file where each line is one request, identified by a unique custom_id.
Upload that file to /v1/files with purpose=batch. The API returns a file ID.
Create the batch on /v1/batches, referencing the file ID and the target endpoint.
Poll the batch until its status is completed.
Download the output file: one JSON result per line, each having back its custom_id.

With the Python OpenAI library

Install the OpenAI library and export your access token:

pip install openai
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

The example below builds a JSONL file, runs the full batch lifecycle, and prints each response next to its custom_id:

import json
import os
import time

from openai import OpenAI

client = OpenAI(
    base_url="https://oai.endpoints.kepler.ai.cloud.ovh.net/v1",
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"),
)

# 1. Build the JSONL input file
prompts = [
    "What's the capital of France?",
    "What's the capital of Québec?",
]
with open("batch_input.jsonl", "w") as f:
    for i, prompt in enumerate(prompts):
        f.write(json.dumps({
            "custom_id": f"request-{i}",
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": {
                "model": "Meta-Llama-3_3-70B-Instruct",
                "messages": [{"role": "user", "content": prompt}],
                "max_tokens": 512,
            },
        }) + "\n")

# 2. Upload it
input_file = client.files.create(
    file=open("batch_input.jsonl", "rb"),
    purpose="batch",
)

# 3. Create the batch
batch = client.batches.create(
    input_file_id=input_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(f"Batch created: {batch.id}")

# 4. Poll until the batch reaches a terminal state
terminal = {"completed", "failed", "expired", "cancelled"}
while batch.status not in terminal:
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
    print(f"Status: {batch.status}")

# 5. Download and parse the results
if batch.status == "completed":
    output = client.files.content(batch.output_file_id).text
    for line in output.strip().splitlines():
        result = json.loads(line)
        custom_id = result["custom_id"]
        content = result["response"]["body"]["choices"][0]["message"]["content"]
        print(f"{custom_id}: {content}")
else:
    print(f"Batch finished with status: {batch.status}")

You can have more details on the dedicated Batch Mode documentation.

Model rate limit

When using AI Endpoints, the following rate limits apply:

Anonymous: 2 requests per minute, per IP and per model.
Authenticated with an API access key: 400 requests per minute, per Public Cloud project and per model.

If you exceed this limit, a 429 error code will be returned.

If you require higher usage, please get in touch with us to discuss increasing your rate limits.

Going Further

Want to explore the full capabilities of the LLM API? Dive into our dedicated Structured Output and Function Calling guides.

For a broader overview of AI Endpoints, explore the full AI Endpoints Documentation.

Reach out to our support team or join the OVHcloud Discord #ai-endpoints channel to share your questions, feedback, and suggestions for improving the service, to the team and the community.

Meta-Llama-3_3-70B-Instruct

Acerca del modelo Meta-Llama-3_3-70B-Instruct

Precio de entrada

Precio de salida

URL de la API

Características soportadas

Formatos de salida

Tamaños de contexto

Parámetros

Prueba el modelo jugando con él.

Meta-Llama-3_3-70B-Instruct API

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

With the Python OpenAI library

Batch Mode

With the Python OpenAI library

Model rate limit

Going Further

Comienza con Meta-Llama-3_3-70B-Instruct

AI Endpoints - Getting Started

AI Endpoints - Features, Capabilities and Limitations

AI Endpoints - Troubleshooting

AI Endpoints - Structured Output

AI Endpoints - Function Calling

AI Endpoints - Using Virtual Models

Create your own Audio Summarizer Assistant

Build a Powerful Voice Assistant

Build a Python Chatbot with LangChain

Build a JavaScript Chatbot with LangChain

Create your own AI chatbot using LangChain4j

Create a Streaming Chatbot with LangChain4j

Enable Memory in your Chatbot using LangChain

Create a Memory Chatbot with LangChain4j

Build a RAG Chatbot with LangChain

Build a RAG Chatbot with LangChain4j

Using Structured Output with LangChain4j

Using Function Calling with LangChain4j

Model Context Protocol (MCP) with LangChain4j