Model catalog
Open-source LLMs you can deploy on AWS
Every model below runs on llama.cpp from a Q4_K_M GGUF, the same engine and format local tools like Ollama use, on an EC2 instance in your own AWS account. Instances are sized to fit the model, and the monthly figure is what AWS bills for an always-on node in us-east-1. You can also deploy any other GGUF model from the Hugging Face Hub.
| Model | Size | Runs on | AWS / month |
|---|---|---|---|
| Qwen2.5 0.5B Instruct Best answers that still fit a free t3.micro. Chat + RAG, no GPU. | 494M | t3.medium · CPU | ≈ $42.47 |
| SmolLM2 360M Instruct Half the weights of Qwen 0.5B. Fastest replies on a 2 vCPU box. | 362M | t3.medium · CPU | ≈ $42.47 |
| SmolLM2 135M Instruct Smallest that still holds a conversation. Good for narrow RAG bots. | 135M | t3.medium · CPU | ≈ $42.47 |
| TinyLlama 1.1B Chat Featherweight. Runs on CPU-only nodes, sub-second replies. | 1.1B | t3.large · CPU | ≈ $72.84 |
| SmolLM2 1.7B Instruct Tiny but instruction-tuned properly. Good CPU-node default for short replies. | 1.7B | t3.large · CPU | ≈ $72.84 |
| Qwen 2.5 1.5B Instruct The step up from 0.5B — noticeably better answers, still CPU-friendly. | 1.5B | t3.large · CPU | ≈ $72.84 |
| Qwen 2.5 Coder 1.5B Instruct Small code assistant — snippets, explanations and docs Q&A on a CPU node. | 1.5B | t3.large · CPU | ≈ $72.84 |
| Phi-2 Microsoft's compact reasoning model. Great for FAQ + support bots. | 2.8B | t3.xlarge · CPU | ≈ $134 |
| Qwen 2.5 3B Instruct Long context in a small footprint. Strong multilingual support. | 3.1B | t3.xlarge · CPU | ≈ $134 |
| Granite 3.1 2B Instruct IBM's enterprise-tuned small model. Long context, built for RAG and tools. | 2.5B | t3.xlarge · CPU | ≈ $134 |
| Phi-3 Mini 4K Instruct Punches far above its size on reasoning and instruction following. | 3.8B | g4dn.xlarge · GPU | ≈ $399 |
| Phi-3.5 Mini Instruct Phi-3 Mini with a 128K window — reads long documents whole. | 3.8B | g4dn.xlarge · GPU | ≈ $399 |
| Phi-4 Mini Instruct Newest Phi. Stronger maths and multilingual answers at the same size. | 3.8B | g4dn.xlarge · GPU | ≈ $399 |
| Zephyr 7B β DPO-aligned chat model with a natural conversational tone. | 7.2B | g5.xlarge · GPU | ≈ $750 |
| Mistral 7B Instruct v0.2 The general-purpose default — long documents, multi-turn support, RAG answers. | 7.2B | g5.xlarge · GPU | ≈ $750 |
| Qwen 2.5 7B Instruct Strongest all-rounder here — coding, maths, 29 languages, 32K context. | 7.6B | g5.xlarge · GPU | ≈ $750 |
| OLMo 2 7B Instruct Fully open — weights, data and training code. For auditable deployments. | 7.3B | g5.xlarge · GPU | ≈ $750 |
| Qwen 2.5 Coder 7B Instruct Best coding model here — generation, review and repo-docs Q&A. | 7.6B | g5.xlarge · GPU | ≈ $750 |
| Hermes 3 (Llama 3.1 8B) Ungated Llama 3.1 fine-tune. Steerable system prompts, strong tool use. | 8.0B | g5.xlarge · GPU | ≈ $750 |
| Granite 3.1 8B Instruct Largest in the catalogue. Enterprise RAG over long documents, on one GPU. | 8.2B | g5.xlarge · GPU | ≈ $750 |
How the monthly figure is worked out
EC2 instance hours (730 a month, on-demand, us-east-1), the root EBS volume, and one SSM interface endpoint. S3, Lambda and API Gateway stay inside AWS’s always-free allowances at typical chat volumes. ForgeStack adds no markup, and the dashboard shows a live estimate for your region before you deploy.
Deploy it into your own AWS account
Free to prototype. You pay AWS directly, and the CloudFormation template is yours to keep.