Meta-Llama-3_3-70B-Instruct

Large Language Models (LLM)

Llama 3.3 is an instruction-tuned generative language model optimized for multilingual dialogue use cases. It was released by Meta AI on December 6, 2024, and utilizes an advanced transformer architecture and is designed to align with human preferences for helpfulness and safety through supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF). The knowledge cutoff for this 70B model is December 31, 2023.

Prova un altro modello

Informazioni sul modello Meta-Llama-3_3-70B-Instruct

Pubblicato il huggingface

06/12/2024

Licenza: Llama 3.3 Community

Prezzo di input

0.67 € /Mtoken(input)

Prezzo di output

0.67 € /Mtoken(output)

Funzionalità supportate

Chiamata di funzioneStreaming

Formati di output

raw_textjson_objectjson_schema

Dimensioni del contesto

131k

Parametri

70B

Prova il modello giocando con esso.

Example of how to use Meta-Llama-3_3-70B-Instruct API in python

With a simple HTTP client (requests)

First install the requests library:

pip install requests

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

If you do not have an access token key yet, follow the instructions in the AI Endpoints – Getting Started.

Finally, run the following python code:

import os
import requests
 
# You can use the model dedicated URL
url = "https://llama-3-3-70b-instruct.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1/chat/completions"
# Or our unified endpoint for easy model switching with optimal OpenAI compatibility
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions"
payload = {
    "max_tokens": 512,
    "messages": [
        {
            "content": "Explain gravity to a 6 years old",
            "role": "user"
        }
    ],
    "model": "Meta-Llama-3_3-70B-Instruct",
    "temperature": 0,
}
 
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {os.getenv('OVH_AI_ENDPOINTS_ACCESS_TOKEN')}",
}
 
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
    # Handle response
    response_data = response.json()
    # Parse JSON response
    choices = response_data["choices"]
    for choice in choices:
        text = choice["message"]["content"]
        # Process text and finish_reason
        print(text)
else:
    print("Error:", response.status_code, response.text)

With the Python OpenAI library

The Meta-Llama-3_3-70B-Instruct API is compatible with the openai specification.

First install the openai library:

pip install openai

Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:

export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>

Finally, run the following python code:

import os

from openai import OpenAI

# You can use the model dedicated URL
url = "https://llama-3-3-70b-instruct.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1"
# Or our unified endpoint for easy model switching with optimal OpenAI compatibility
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1

client = OpenAI(
    base_url=url,
    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN")
)

def chat_completion(new_message: str) -> str:
    history_openai_format = [{"role": "user", "content": new_message}]

    return client.chat.completions.create(
        model="Meta-Llama-3_3-70B-Instruct",
        messages=history_openai_format,
        temperature=0,
        max_tokens=1024
    ).choices.pop().message.content


if __name__ == '__main__':
    print(chat_completion("Explain gravity for a 6 years old"))

Model rate limit

When using AI Endpoints, the following rate limits apply:

Anonymous: 2 requests per minute, per IP and per model.
Authenticated with an API access key: 400 requests per minute, per Public Cloud project and per model.

If you exceed this limit, a 429 error code will be returned.

If you require higher usage, please get in touch with us to discuss increasing your rate limits.

Meta-Llama-3_3-70B-Instruct