Llama 3.3 is an instruction-tuned generative language model optimized for multilingual dialogue use cases. It was released by Meta AI on December 6, 2024, and utilizes an advanced transformer architecture and is designed to align with human preferences for helpfulness and safety through supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF). The knowledge cutoff for this 70B model is December 31, 2023.
First install the requests library:
pip install requests
Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>
If you do not have an access token key yet, follow the instructions in the AI Endpoints – Getting Started.
Finally, run the following python code:
import os
import requests
# You can use the model dedicated URL
url = "https://llama-3-3-70b-instruct.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1/chat/completions"
# Or our unified endpoint for easy model switching with optimal OpenAI compatibility
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions"
payload = {
"max_tokens": 512,
"messages": [
{
"content": "Explain gravity to a 6 years old",
"role": "user"
}
],
"model": "Meta-Llama-3_3-70B-Instruct",
"temperature": 0,
}
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {os.getenv('OVH_AI_ENDPOINTS_ACCESS_TOKEN')}",
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
# Handle response
response_data = response.json()
# Parse JSON response
choices = response_data["choices"]
for choice in choices:
text = choice["message"]["content"]
# Process text and finish_reason
print(text)
else:
print("Error:", response.status_code, response.text)
The Meta-Llama-3_3-70B-Instruct API is compatible with the openai specification.
First install the openai library:
pip install openai
Next, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>
Finally, run the following python code:
import os
from openai import OpenAI
# You can use the model dedicated URL
url = "https://llama-3-3-70b-instruct.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1"
# Or our unified endpoint for easy model switching with optimal OpenAI compatibility
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1
client = OpenAI(
base_url=url,
api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN")
)
def chat_completion(new_message: str) -> str:
history_openai_format = [{"role": "user", "content": new_message}]
return client.chat.completions.create(
model="Meta-Llama-3_3-70B-Instruct",
messages=history_openai_format,
temperature=0,
max_tokens=1024
).choices.pop().message.content
if __name__ == '__main__':
print(chat_completion("Explain gravity for a 6 years old"))
When using AI Endpoints, the following rate limits apply:
If you exceed this limit, a 429 error code will be returned.
If you require higher usage, please get in touch with us to discuss increasing your rate limits.
New to AI Endpoints? This guide walks you through everything you need to get an access token, call AI models, and integrate AI APIs into your apps with ease.
Start tutorialExplore what AI Endpoints can do. This guide breaks down current features, future roadmap items, and the platform's core capabilities so you know exactly what to expect.
Start tutorialRunning into issues? This guide helps you solve common problems on AI Endpoints, from error codes to unexpected responses. Get quick answers, clear fixes, and helpful tips to keep your projects running smoothly.
Start tutorialLearn how to use Structured Output with OVHcloud AI Endpoints.
Start tutorialLearn how to use Function Calling with OVHcloud AI Endpoints.
Start tutorialLearn how to use OVHcloud AI Endpoints Virtual Models.
Start tutorialTurn hours of audio into sharp, readable summaries! This guide walks you through building an AI-powered audio assistant using ASR and LLMs, perfect for meetings, podcasts, or any voice recordings.
Start tutorialBuild a voice-enabled assistant in under 100 lines of code! Learn how to combine ASR, LLM, and TTS endpoints to create an AI that listens, understands, and responds.
Start tutorialImplement a Python chatbot from scratch with LangChain and AI Endpoints! Learn to set up streaming, prompt handling, and command-line interactivity for a better user experience.
Start tutorialBring LangChain to the JavaScript world! Learn how to build a chatbot in Node.js using AI Endpoints, complete with real-time response streaming.
Start tutorialCreate a Java-based AI chatbot using LangChain4j and Quarkus. Connect to AI Endpoints, customize prompts, and expose your assistant as a full-featured API.
Start tutorialLevel up your chatbot! Learn how to stream LLM responses in real time using LangChain4j, Quarkus, and OVHcloud AI Endpoints for seamless, conversational experiences.
Start tutorialMake your chatbot feel human! This guide teaches you how to add memory with LangChain, enabling your assistant to remember past interactions and respond contextually.
Start tutorialBuild a Java chatbot that remembers! Learn how to use LangChain4j to create a memory-enabled assistant that holds rich, ongoing conversations.
Start tutorialCombine AI with your own data! This tutorial shows how to build a Python-based RAG chatbot that retrieves custom documents to enhance its answers.
Start tutorialCreate a Java-based RAG chatbot that blends LLMs with your own data. Learn how to use LangChain4j and AI Endpoints to build smart, document-aware assistants.
Start tutorialLearn how to use Structured Output with Java, LangChain4j and OVHcloud AI Endpoints.
Start tutorialLearn how to use Function Calling with Java, LangChain4j and OVHcloud AI Endpoints.
Start tutorialLearn how to use Model Context Protocol (MCP) with Java, LangChain4j and OVHcloud AI Endpoints.
Start tutorial