Model on AWS
Deploy SmolLM2 1.7B Instruct on AWS
Tiny but instruction-tuned properly. Good CPU-node default for short replies. ForgeStack runs HuggingFaceTB/SmolLM2-1.7B-Instruct as a private LLM API in your own AWS account: an t3.large node serving it with llama.cpp, a knowledge base in your S3 bucket, and a chat endpoint with citations.
- Parameters
- 1.7B
- Context
- 8K
- License
- Apache-2.0
- Speed
- Slow
- Instance
- t3.large
- Hardware
- CPU
- AWS / month
- ≈ $72.84
- Compute / hour
- $0.0832
What you get when you deploy SmolLM2 1.7B Instruct
- Model node: a dedicated t3.large (CPU) in the region you pick, serving SmolLM2-1.7B-Instruct-Q4_K_M.gguf from bartowski/SmolLM2-1.7B-Instruct-GGUF with llama.cpp.
- Chat API: POST /api/{projectId}/chat with a scoped x-api-key, returning chat.completion-shaped responses with source citations.
- Knowledge base (RAG): upload documents or sync GitHub, Confluence and Jira; they are stored and indexed inside your stack.
- LoRA fine-tuning on your own question/answer pairs, on the same node.
- Human handoff to Slack, Teams, Zendesk, Freshdesk, Intercom, email or a webhook when the model can’t answer. See chatbot integrations.
How to deploy it
- Create a ForgeStack account and pick SmolLM2 1.7B Instruct from the catalog.
- Connect AWS with IAM Identity Center or a cross-account role. ForgeStack only gets a short-lived STS session.
- Set a system prompt and turn on retrieval.
- Review and download the generated CloudFormation template.
- Apply it. The stack goes live once the model node passes its health check.
What it costs
About $72.84 a month in us-east-1 for an always-on node, paid to AWS directly: the t3.large instance, its root volume and one VPC endpoint. ForgeStack itself is free to prototype and adds no markup. Delete the stack and the charges stop.
Other CPU models
- Qwen2.5 0.5B Instruct on t3.medium, ≈ $42.47/month
- SmolLM2 360M Instruct on t3.medium, ≈ $42.47/month
- SmolLM2 135M Instruct on t3.medium, ≈ $42.47/month
- TinyLlama 1.1B Chat on t3.large, ≈ $72.84/month
Deploy it into your own AWS account
Free to prototype. You pay AWS directly, and the CloudFormation template is yours to keep.