Run seven Mistral 7B-based models behind an OpenAI-compatible API on a GPU instance in your own AWS account. Point existing OpenAI tooling at your own endpoint and keep your prompts private.
Mistral 7B is an open-source large language model with 7 billion parameters, built for efficiency without giving up quality. Its compact size makes it practical to self-host for coding help, content generation and custom AI features.
Use existing OpenAI SDKs and tools with a new base URL.
/v1/chat/completions and /v1/completions endpoints.
/v1/embeddings for search and retrieval.
/v1/models shows every model in the bundle.
API documentation at /docs on your server.
Customise and run the models on your own terms.
The AMI ships seven Mistral-based models behind one OpenAI-compatible API on Ubuntu 22.04.
Make sure your account has enough vCPU quota for a g4dn instance in your region.
Subscribe on AWS Marketplace and launch the CloudFormation stack with your domain and admin email.
Update the Route 53 record with the PublicIp from the stack outputs.
Open /docs on your domain, then point your OpenAI client at your server's /v1 endpoint.
An x86 AMI on AWS Marketplace, launched on a g4dn GPU instance in your own AWS account with a CloudFormation stack.
Launch on AWS The server implements the OpenAI request and response shapes, so clients only need a new base URL.
| Endpoint | Purpose |
|---|---|
| /v1/chat/completions | Chat-style requests with a messages array |
| /v1/completions | Single-prompt text completion |
| /v1/embeddings | Vector embeddings for search and retrieval |
| /v1/models | List the models in the bundle |
| /docs | Interactive API documentation |
Want to compare serving stacks? Read how to self-host an OpenAI-compatible API →
An efficient, open-source large language model with 7 billion parameters, suited to coding help, content generation and custom AI features.
mistral-7b-v0.1, mistral-7b-instruct-v0.1, mistral-7b-instruct-v0.2, dolphin-2.6-mistral-7b, openhermes-2.5-mistral-7b, yarn-mistral-7b-128k and mistral-7b-openorca.
Yes. Point your OpenAI SDK or tools at your server's /v1 endpoint and they work with minimal changes.
The x86 listing runs on g4dn GPU instances, from g4dn.xlarge upward.
Only the x86 bundle is publicly listed on AWS Marketplace at the moment.
Check that the domain is on Route 53, then SSH in and run sudo /root/certificate_generate_standalone.sh.
Back up your server data, remove the old stack and launch the new version from AWS Marketplace.
Ollama, vLLM or a pre-built AMI, without rewriting app code.
How Mistral compares with other open models.
Capabilities, deployment steps and API integration.
We build and run self-hosted AI infrastructure every day, from model selection to GPU sizing. Tell us what you need.
Contact us