Model on AWS
Deploy TinyLlama 1.1B Chat on AWS
Featherweight. Runs on CPU-only nodes, sub-second replies. ForgeStack runs TinyLlama/TinyLlama-1.1B-Chat-v1.0 as a private LLM API in your own AWS account: an t3.large node serving it with llama.cpp, a knowledge base in your S3 bucket, and a chat endpoint with citations.
- Parameters
- 1.1B
- Context
- 2K
- License
- Apache-2.0
- Speed
- Slow
- Instance
- t3.large
- Hardware
- CPU
- AWS / month
- ≈ $72.84
- Compute / hour
- $0.0832
What you get when you deploy TinyLlama 1.1B Chat
- Model node: a dedicated t3.large (CPU) in the region you pick, serving tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf from TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF with llama.cpp.
- Chat API: POST /api/{projectId}/chat with a scoped x-api-key, returning chat.completion-shaped responses with source citations.
- Knowledge base (RAG): upload documents or sync GitHub, Confluence and Jira; they are stored and indexed inside your stack.
- LoRA fine-tuning on your own question/answer pairs, on the same node.
- Human handoff to Slack, Teams, Zendesk, Freshdesk, Intercom, email or a webhook when the model can’t answer. See chatbot integrations.
How to deploy it
- Create a ForgeStack account and pick TinyLlama 1.1B Chat from the catalog.
- Connect AWS with IAM Identity Center or a cross-account role. ForgeStack only gets a short-lived STS session.
- Set a system prompt and turn on retrieval.
- Review and download the generated CloudFormation template.
- Apply it. The stack goes live once the model node passes its health check.
What it costs
About $72.84 a month in us-east-1 for an always-on node, paid to AWS directly: the t3.large instance, its root volume and one VPC endpoint. ForgeStack itself is free to prototype and adds no markup. Delete the stack and the charges stop.
Other CPU models
- Qwen2.5 0.5B Instruct on t3.medium, ≈ $42.47/month
- SmolLM2 360M Instruct on t3.medium, ≈ $42.47/month
- SmolLM2 135M Instruct on t3.medium, ≈ $42.47/month
- Phi-2 on t3.xlarge, ≈ $134/month
Deploy it into your own AWS account
Free to prototype. You pay AWS directly, and the CloudFormation template is yours to keep.