Porównuj i testuj modele open weight do tekstu, kodu, mowy i obrazu, bez rejestracji. Następnie przejdź do produkcji z tym samym API zgodnym z OpenAI.
Llama 3.3 to generatywny model językowy dostrojony do wykonywania instrukcji i prowadzenia rozmów w wielu językach. Został udostępniony przez Meta AI 6 grudnia 2024 roku. Wykorzystuje zaawansowaną architekturę transformera, nadzorowane dostrajanie (SFT) oraz uczenie przez wzmocnienie połączone z informacją zwrotną od człowieka (RLHF), dzięki czemu lepiej odpowiada ludzkim oczekiwaniom w zakresie użyteczności i bezpieczeństwa. Wiedza modelu 70B obejmuje informacje do 31 grudnia 2023.
First install the requests library:
pip install requestsNext, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>If you do not have an access token key yet, follow the instructions in the AI Endpoints – Getting Started.
Finally, run the following Python code:
import os
import requests
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {os.getenv('OVH_AI_ENDPOINTS_ACCESS_TOKEN')}",
}
# With /chat/completions
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions"
payload = {
"max_tokens": 512,
"messages": [
{
"content": "Explain gravity to a 6 years old",
"role": "user"
}
],
"model": "Meta-Llama-3_3-70B-Instruct",
"temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
# Handle response
response_data = response.json()
# Parse JSON response
choices = response_data["choices"]
for choice in choices:
text = choice["message"]["content"]
# Process text and finish_reason
print(text)
else:
print("Error:", response.status_code, response.text)
# With /responses
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/responses"
payload = {
"max_output_tokens": 512,
"input": [
{
"content": "Explain gravity to a 6 years old",
"role": "user"
}
],
"store": False,
"model": "Meta-Llama-3_3-70B-Instruct",
"temperature": 0,
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 200:
# Handle response
response_data = response.json()
text = response_data["output"][0]["content"][0]["text"]
print(text)
else:
print("Error:", response.status_code, response.text)The Meta-Llama-3_3-70B-Instruct API is compatible with the OpenAI specification.
First install the openai library:
pip install openaiNext, export your access token to the OVH_AI_ENDPOINTS_ACCESS_TOKEN environment variable:
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>Finally, run the following Python code:
import os
from openai import OpenAI
url = "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1"
client = OpenAI(
base_url=url,
api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN")
)
def chat_completion(new_message: str) -> str:
history_openai_format = [{"role": "user", "content": new_message}]
return client.chat.completions.create(
model="Meta-Llama-3_3-70B-Instruct",
messages=history_openai_format,
temperature=0,
max_tokens=1024
).choices.pop().message.content
def responses(new_message: str) -> str:
response = client.responses.create(
model="Meta-Llama-3_3-70B-Instruct",
input=new_message,
temperature=0,
max_output_tokens=1024,
store=False
)
return response.output[0].content[0].text
if __name__ == '__main__':
# With chat completion endpoint
print(chat_completion("Explain gravity for a 6 years old"))
# With responses endpoint
print(responses("Explain gravity for a 12 years old"))The Batch API (/v1/batches) lets you submit a large number of inference requests in a single asynchronous job, instead of sending them one by one through our synchronous endpoints. Batch mode is ideal when you do not need an immediate answer but want to process a high volume of prompts (evaluations, offline labelling, content generation at scale, dataset preparation, etc...).
Batch mode requests are billed at 50% off compared to directly calling the generation endpoints (/v1/chat/completions or /v1/responses).
Results are delivered within a maximum of 24 hours after the batch is submitted.
The workflow follows the OpenAI Batch API specification:
Install the OpenAI library and export your access token:
pip install openai
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-access-token>The example below builds a JSONL file, runs the full batch lifecycle, and prints each response next to its custom_id:
import json
import os
import time
from openai import OpenAI
client = OpenAI(
base_url="https://oai.endpoints.kepler.ai.cloud.ovh.net/v1",
api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"),
)
# 1. Build the JSONL input file
prompts = [
"What's the capital of France?",
"What's the capital of Québec?",
]
with open("batch_input.jsonl", "w") as f:
for i, prompt in enumerate(prompts):
f.write(json.dumps({
"custom_id": f"request-{i}",
"method": "POST",
"url": "/v1/chat/completions",
"body": {
"model": "Meta-Llama-3_3-70B-Instruct",
"messages": [{"role": "user", "content": prompt}],
"max_tokens": 512,
},
}) + "\n")
# 2. Upload it
input_file = client.files.create(
file=open("batch_input.jsonl", "rb"),
purpose="batch",
)
# 3. Create the batch
batch = client.batches.create(
input_file_id=input_file.id,
endpoint="/v1/chat/completions",
completion_window="24h",
)
print(f"Batch created: {batch.id}")
# 4. Poll until the batch reaches a terminal state
terminal = {"completed", "failed", "expired", "cancelled"}
while batch.status not in terminal:
time.sleep(30)
batch = client.batches.retrieve(batch.id)
print(f"Status: {batch.status}")
# 5. Download and parse the results
if batch.status == "completed":
output = client.files.content(batch.output_file_id).text
for line in output.strip().splitlines():
result = json.loads(line)
custom_id = result["custom_id"]
content = result["response"]["body"]["choices"][0]["message"]["content"]
print(f"{custom_id}: {content}")
else:
print(f"Batch finished with status: {batch.status}")You can have more details on the dedicated Batch Mode documentation.
When using AI Endpoints, the following rate limits apply:
If you exceed this limit, a 429 error code will be returned.
If you require higher usage, please get in touch with us to discuss increasing your rate limits.
Want to explore the full capabilities of the LLM API? Dive into our dedicated Structured Output and Function Calling guides.
For a broader overview of AI Endpoints, explore the full AI Endpoints Documentation.
Reach out to our support team or join the OVHcloud Discord #ai-endpoints channel to share your questions, feedback, and suggestions for improving the service, to the team and the community.
New to AI Endpoints? This guide walks you through everything you need to get an access token, call AI models, and integrate AI APIs into your apps with ease.
Rozpocznij samouczekExplore what AI Endpoints can do. This guide breaks down current features, future roadmap items, and the platform's core capabilities so you know exactly what to expect.
Rozpocznij samouczekRunning into issues? This guide helps you solve common problems on AI Endpoints, from error codes to unexpected responses. Get quick answers, clear fixes, and helpful tips to keep your…
Rozpocznij samouczekLearn how to use Structured Output with OVHcloud AI Endpoints.
Rozpocznij samouczekLearn how to use Function Calling with OVHcloud AI Endpoints.
Rozpocznij samouczekLearn how to use OVHcloud AI Endpoints Virtual Models.
Rozpocznij samouczekTurn hours of audio into sharp, readable summaries! This guide walks you through building an AI-powered audio assistant using ASR and LLMs, perfect for meetings, podcasts, or any voice recordings.
Rozpocznij samouczekBuild a voice-enabled assistant in under 100 lines of code! Learn how to combine ASR, LLM, and TTS endpoints to create an AI that listens, understands, and responds.
Rozpocznij samouczekImplement a Python chatbot from scratch with LangChain and AI Endpoints! Learn to set up streaming, prompt handling, and command-line interactivity for a better user experience.
Rozpocznij samouczekBring LangChain to the JavaScript world! Learn how to build a chatbot in Node.js using AI Endpoints, complete with real-time response streaming.
Rozpocznij samouczekCreate a Java-based AI chatbot using LangChain4j and Quarkus. Connect to AI Endpoints, customize prompts, and expose your assistant as a full-featured API.
Rozpocznij samouczekLevel up your chatbot! Learn how to stream LLM responses in real time using LangChain4j, Quarkus, and OVHcloud AI Endpoints for seamless, conversational experiences.
Rozpocznij samouczekMake your chatbot feel human! This guide teaches you how to add memory with LangChain, enabling your assistant to remember past interactions and respond contextually.
Rozpocznij samouczekBuild a Java chatbot that remembers! Learn how to use LangChain4j to create a memory-enabled assistant that holds rich, ongoing conversations.
Rozpocznij samouczekCombine AI with your own data! This tutorial shows how to build a Python-based RAG chatbot that retrieves custom documents to enhance its answers.
Rozpocznij samouczekCreate a Java-based RAG chatbot that blends LLMs with your own data. Learn how to use LangChain4j and AI Endpoints to build smart, document-aware assistants.
Rozpocznij samouczekLearn how to use Structured Output with Java, LangChain4j and OVHcloud AI Endpoints.
Rozpocznij samouczekLearn how to use Function Calling with Java, LangChain4j and OVHcloud AI Endpoints.
Rozpocznij samouczekLearn how to use Model Context Protocol (MCP) with Java, LangChain4j and OVHcloud AI Endpoints.
Rozpocznij samouczek