Run Meta's Llama 3, in 8B or 70B, behind an OpenAI-compatible API on a GPU instance in your own AWS account. Point your OpenAI tools at your own endpoint and keep your data private.
Llama 3 is Meta's open large language model family, with refined post-training and better scalability than Llama 2. It handles language understanding, translation, dialogue, reasoning and code generation.
8B for cost and speed, 70B for harder tasks.
Assistants, Q&A and multi-step answers.
Write and explain code.
Chat, completions, embeddings and model listing endpoints.
Pick a model id from /v1/models and pass it in your request.
Prompts and outputs stay in your own account.
Each AMI ships the model and an OpenAI-compatible API server on Ubuntu 22.04, ready on a g4dn GPU instance.
Choose 8B or 70B and check your account has vCPU quota for the matching g4dn instance.
Subscribe on AWS Marketplace and launch the CloudFormation stack with your domain and admin email.
After 5-10 minutes, create a DNS record with the PublicIp from the stack outputs.
Open the DashboardUrl for the API docs, then point your OpenAI client at your server.
Two AMIs on AWS Marketplace, each launched on a g4dn GPU instance in your own AWS account with CloudFormation.
Browse Meetrix on AWS | Model | Recommended instance | AWS |
|---|---|---|
| Llama 3 8B | g4dn.xlarge | Launch on AWS |
| Llama 3 70B | g4dn.metal | Launch on AWS |
The 8B listing supports g4dn instances from xlarge up to metal.
| Endpoint | Purpose |
|---|---|
| /v1/chat/completions | Chat completions from a list of messages |
| /v1/completions | Completions from a prompt |
| /v1/embeddings | Embeddings for input text |
| /v1/models | List available models |
Switching models takes a little longer on the first response. Read the Llama 3 developer guide →
Meta's large language model family with improved post-training, for understanding, translation, dialogue, reasoning and code generation.
8B on g4dn.xlarge for lower cost; 70B on g4dn.metal for more capable output.
Call /v1/models, copy the id you want and use it as the model value in your request. The first response after a switch is slower.
SSH in and run sudo systemctl restart llama.service, wait a few minutes and reload the dashboard.
Yes. OpenAI SDKs work with your server as the base URL.
SSH in and run sudo /root/certificate_generate_standalone.sh with your admin email.
Back up your server data, remove the old stack and launch the new version from AWS Marketplace.
Setup steps, comparisons and cost management.
How Llama compares with other open models.
Swap the OpenAI base URL without rewriting app code.
We build and run self-hosted AI infrastructure every day, from model selection to GPU sizing. Tell us what you need.
Contact us