Meta-Llama-3_3-70B-Instruct

Large Language Models (LLM)

Llama 3.3 ist ein anweisungsoptimiertes generatives Sprachmodell, das für Use Cases mit mehrsprachigen Dialogen optimiert ist. Es wurde von Meta AI am 06. Dezember 2024 veröffentlicht, stützt sich auf eine fortschrittliche Transformer-Architektur und ist darauf ausgelegt, sich durch beaufsichtigte Feinabstimmung (Supervised Fine-Tuning, SFT) und bestärkendes Lernen mit menschlichem Feedback (Reinforcement Learning with Human Feedback, RLHF) an menschliche Präferenzen für Nützlichkeit und Sicherheit anzupassen. Der Wissensstichtag für dieses Modell mit 70 Milliarden Parametern ist der 31. Dezember 2023.

Versuchen Sie ein anderes Modell

Über das Meta-Llama-3_3-70B-Instruct Modell

Veröffentlicht am huggingface

06/12/2024

Lizenz: Llama 3.3 Community

Eingabepreis

0.67 € /Mtoken(Eingabe)

Ausgabepreis

0.67 € /Mtoken(Ausgabe)

Unterstützte Funktionen

FunktionsaufrufStreaming

Ausgabeformate

raw_textjson_objectjson_schema

Kontextgrößen

131k

Parameter

70B

Testen Sie das Modell, indem Sie damit spielen.

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

First install the requests library:

pip install requests

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

If you do not have an access token key yet, follow the instructions in the AI Endpoints – Getting Started.

Finally, run the following Python code:

import os
import requests
 
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions"
payload = {
    "max_tokens": 512,
    "messages": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
 
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {os.getenv('OVH_AI_ENDPOINTS_ACCESS_TOKEN')}",
}
 
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    # Parse JSON response
    choices = response_data["choices"]
    for choice in choices:
        text = choice["message"]["content"]
        # Process text and finish_reason
        print(text)
else:
    print("Error:", response.status_code, response.text)

With the Python OpenAI library

The Meta-Llama-3_3-70B-Instruct API is compatible with the OpenAI specification.

First install the openai library:

pip install openai

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

Finally, run the following Python code:

import os

from openai import OpenAI

url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1"

client = OpenAI(
    base_url=url,
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN")
)

def chat_completion(new_message: str) -> str:
    history_openai_format = [{"role": "user", "content": new_message}]

    return client.chat.completions.create(
        model="Meta-Llama-3_3-70B-Instruct",
        messages=history_openai_format,
        temperature=0,
        max_tokens=1024
    ).choices.pop().message.content


if __name__ == '__main__':
    print(chat_completion("Explain gravity for a 6 years old"))  

Model rate limit

When using AI Endpoints, the following rate limits apply:

Anonymous: 2 requests per minute, per IP and per model.
Authenticated with an API access key: 400 requests per minute, per Public Cloud project and per model.

If you exceed this limit, a 429 error code will be returned.

If you require higher usage, please get in touch with us to discuss increasing your rate limits.

Going Further

Want to explore the full capabilities of the LLM API? Dive into our dedicated Structured Output and Function Calling guides.

For a broader overview of AI Endpoints, explore the full AI Endpoints Documentation.

Reach out to our support team or join the OVHcloud Discord #ai-endpoints channel to share your questions, feedback, and suggestions for improving the service, to the team and the community.

Meta-Llama-3_3-70B-Instruct