Model on AWS
Deploy Phi-3 Mini 4K Instruct on AWS
Punches far above its size on reasoning and instruction following. ForgeStack runs microsoft/Phi-3-mini-4k-instruct as a private LLM API in your own AWS account: an g4dn.xlarge node serving it with llama.cpp, a knowledge base in your S3 bucket, and a chat endpoint with citations.
- Parameters
- 3.8B
- Context
- 4K
- License
- MIT
- Speed
- Fast
- Instance
- g4dn.xlarge
- Hardware
- GPU
- AWS / month
- ≈ $399
- Compute / hour
- $0.5260
What you get when you deploy Phi-3 Mini 4K Instruct
- Model node: a dedicated g4dn.xlarge (GPU) in the region you pick, serving Phi-3-mini-4k-instruct-Q4_K_M.gguf from bartowski/Phi-3-mini-4k-instruct-GGUF with llama.cpp.
- Chat API: POST /api/{projectId}/chat with a scoped x-api-key, returning chat.completion-shaped responses with source citations.
- Knowledge base (RAG): upload documents or sync GitHub, Confluence and Jira; they are stored and indexed inside your stack.
- LoRA fine-tuning on your own question/answer pairs, on the same node.
- Human handoff to Slack, Teams, Zendesk, Freshdesk, Intercom, email or a webhook when the model can’t answer. See chatbot integrations.
How to deploy it
- Create a ForgeStack account and pick Phi-3 Mini 4K Instruct from the catalog.
- Connect AWS with IAM Identity Center or a cross-account role. ForgeStack only gets a short-lived STS session.
- Set a system prompt and turn on retrieval.
- Review and download the generated CloudFormation template.
- Apply it. The stack goes live once the model node passes its health check.
What it costs
About $399 a month in us-east-1 for an always-on node, paid to AWS directly: the g4dn.xlarge instance, its root volume and one VPC endpoint. ForgeStack itself is free to prototype and adds no markup. Delete the stack and the charges stop.
Other GPU models
- Zephyr 7B β on g5.xlarge, ≈ $750/month
- Mistral 7B Instruct v0.2 on g5.xlarge, ≈ $750/month
- Qwen 2.5 7B Instruct on g5.xlarge, ≈ $750/month
- Phi-3.5 Mini Instruct on g4dn.xlarge, ≈ $399/month
Deploy it into your own AWS account
Free to prototype. You pay AWS directly, and the CloudFormation template is yours to keep.