Deploy Qwen2.5 0.5B Instruct on AWS

• Model node: a dedicated t3.medium (CPU) in the region you pick, serving Qwen2.5-0.5B-Instruct-Q4_K_M.gguf from bartowski/Qwen2.5-0.5B-Instruct-GGUF with llama.cpp.
• Chat API: POST /api/{projectId}/chat with a scoped x-api-key, returning chat.completion-shaped responses with source citations.
• Knowledge base (RAG): upload documents or sync GitHub, Confluence and Jira; they are stored and indexed inside your stack.
• LoRA fine-tuning on your own question/answer pairs, on the same node.
• Human handoff to Slack, Teams, Zendesk, Freshdesk, Intercom, email or a webhook when the model can’t answer. See chatbot integrations.

1. Create a ForgeStack account and pick Qwen2.5 0.5B Instruct from the catalog.
2. Connect AWS with IAM Identity Center or a cross-account role. ForgeStack only gets a short-lived STS session.
3. Set a system prompt and turn on retrieval.
4. Review and download the generated CloudFormation template.
5. Apply it. The stack goes live once the model node passes its health check.

About $42.47 a month in us-east-1 for an always-on node, paid to AWS directly: the t3.medium instance, its root volume and one VPC endpoint. ForgeStack itself is free to prototype and adds no markup. Delete the stack and the charges stop.

• SmolLM2 360M Instruct on t3.medium, ≈ $42.47/month
• SmolLM2 135M Instruct on t3.medium, ≈ $42.47/month
• TinyLlama 1.1B Chat on t3.large, ≈ $72.84/month
• Phi-2 on t3.xlarge, ≈ $134/month

See the full catalog.