Can you run it?
This page is a projection of the one entry record, the Reliability and Data control factors that Implement covers. The full verdict is set by all four factors together, floor-weighted so the weakest caps the whole.
- AssessUse & modify + Transparency
- ImplementData control + Reliability
- UseReliability
- SupportTransparency
Install & run
Access is gated: accept the license on the model page, then authenticate with a Hugging Face token. Pin the revision and prefer safetensors.
# Hugging Face (gated) - accept the license first, then:
huggingface-cli login # paste your HF token
python -c "from transformers import AutoModelForCausalLM, AutoTokenizer; \
m='meta-llama/Llama-3.1-8B-Instruct'; \
AutoTokenizer.from_pretrained(m); \
AutoModelForCausalLM.from_pretrained(m, revision='main')"
# Ollama (community-packaged, quantized):
ollama run llama3.1
# vLLM (OpenAI-compatible server):
vllm serve meta-llama/Llama-3.1-8B-Instruct
# llama.cpp (GGUF):
llama-cli -hf bartowski/Meta-Llama-3.1-8B-Instruct-GGUF -p "Hello"
Hosted alternatives need no local weights: Llama is served on AWS Bedrock, Azure AI, Google Vertex AI, and third-party APIs (Together, Groq). Those are endpoints, not the weight source, so pin the canonical HF revision when self-hosting.
Hardware & VRAM requirements
The 8B model runs on a single modern GPU (roughly 16 GB in bf16, far less quantized) and is usable on high-end laptops via GGUF. The 70B models (3.1 / 3.3) need multi-GPU in full precision, or a single large GPU with 4-bit quantization.
The 405B model and the large Llama 4 MoE models (Maverick, Behemoth) require a serving cluster. Llama 4 Scout's mixture-of-experts keeps active parameters (~17B) small, but total weights and the very long context still demand substantial memory.
Serving stacks
Llama has first-class, day-0 support across the serving ecosystem: vLLM and TGI for high-throughput GPU serving, llama.cpp for CPU/edge and GGUF, Ollama for local developer workflows, MLX for Apple Silicon, and plain transformers for scripting. Nothing needs unlocking before serving.
Safe-deployment controls & Deployment Ceiling
The model-specific gate rows of the safe-deployment playbook are four. Pin the revision from
meta-llama and verify the checksum (supply chain). Confirm the license permits your use,
including the 700M-MAU trigger and attribution terms (legal). Deploy the Instruct variant with
Llama Guard on both input and output plus prompt-injection defences (safety). Add a deterministic
action-policy engine with a kill switch before granting any autonomy (action layer).
Deployment Ceiling: T3 (bounded autonomous), conditional. T2 is reachable with the Instruct variant, Llama Guard input/output classification, and prompt-injection defences, all available. T3 additionally requires a deterministic action-policy engine, per-action human approval for irreversible operations, a kill switch, and scheduled red-teaming. T4 (regulated high-stakes autonomous) is not recommended: the community license, the acceptable-use terms, and the unmet Article 55 documentation burden do not support high-stakes regulated use.
Available quantizations
Meta ships official low-bit releases for parts of the family. The community adds GGUF (bartowski, unsloth) for llama.cpp/Ollama, plus AWQ and GPTQ builds for GPU serving. Quantized checkpoints are third-party artifacts: verify the uploader, prefer well-known maintainers, and re-test quality at the chosen bit-width.
Fine-tuning & adaptation
Meta's torchtune library targets Llama directly, and the model works with the common LoRA/QLoRA stacks (PEFT + Transformers, Axolotl, Unsloth). QLoRA makes 8B and even 70B tunable on modest hardware. A derivative placed on the EU market under your own name can make you a provider with your own documentation obligations.
API / OpenAI-compatible usage
vLLM and TGI both expose /v1/chat/completions, so existing OpenAI-SDK code points at a
self-hosted Llama with only a base-URL change. Managed OpenAI-compatible APIs (Together, Groq, the
cloud providers) and Meta's own Llama API offer the same shape without self-hosting.
How this scores
The ownership factors this domain covers, drawn from the one entry record.
ReliabilityIs it reliable and good enough for the job?
StrongRuns dependably on the largest fine-tuning and serving ecosystem of any open family, with documented safety tuning and an available companion guard model, so misuse is controllable off the shelf.
Doesn't extract your dataDoes running it keep your knowledge and data yours?
StrongSelf-hosted, the weights run on your own infrastructure - the Community License adds naming and scale conditions but no telemetry or data clawback, so your data stays yours.
Sources
The same evidence records as the entry sheet. Read means the text was verified; unverified means it is known to exist but not yet read.