Meta: Llama 3.3 70B Instruct

meta-llama/llama-3-3-70b-instruct

The Meta Llama 3.3 multilingual large language model (LLM) is a pretrained and instruction tuned generative model in 70B (text in/text out).

Modalities
In / out per 1M$0.10 / $0.32
Context128K
ReleasedDec 6, 2024

Providers

11 providers serve this model upstream.

Providers serving this model
ProviderInput /MOutput /MCache read /MLatency P50Throughput P50Uptime
AkashMLfp8$0.20$0.52$0.100.64s34 tps100%
Groq$0.59$0.79$0.290.54s190 tps100%
CoreWeavefp16$0.71$0.71$0.710.21s72 tps99.69%
Cloudflarefp8$0.29$2.25Variable0.70s39 tps99.64%
Parasailfp8$0.22$0.50$0.110.65s39 tps99.58%
SambaNova Turbo$0.45$0.90Variable0.71s103 tps99.53%
NovitaAIbf16$0.14$0.40Variable1.31s34 tps99.47%
DeepInfra (Turbo)us · fp8$0.10$0.32Variable0.97s11 tps99.16%
Together$1.04$1.04Variable1.11s25 tps98.48%
Google Vertexus-central1$0.72$0.72Variable1.51s101 tps—
Google Vertex$0.72$0.72Variable———

Market-wide measurements, refreshed every 15 minutes. They describe who serves Llama 3.3 70B Instruct in general, not how ProjectCOZY routes your requests.

Pricing

List rates per 1M tokens — what Llama 3.3 70B Instruct costs pay-as-you-go elsewhere, not what you pay here.

Input
$0.10
per 1M tokens
Output
$0.32
per 1M tokens
Cache read
—
not published
Included in every pass
Llama 3.3 70B Instruct is included in every ProjectCOZY pass at no extra cost — one flat rate, no per-token metering. The rates above are there for comparison.

Performance

Time to first token and output speed across every provider, over the last 3 days.

0.0s2.5s5.0s7.5s10.0s
Oct 16, 12 AMOct 16, 8 AMOct 16, 4 PMOct 17, 12 AMOct 17, 8 AMOct 17, 4 PMOct 18, 12 AMOct 18, 8 AMOct 18, 4 PMOct 19, 12 AMOct 19, 8 AM
Together Cloudflare AkashML DeepInfra (Turbo) (us) CoreWeave NovitaAI Groq SambaNova Turbo Parasail
0 tok/s200 tok/s400 tok/s600 tok/s800 tok/s
Oct 16, 12 AMOct 16, 8 AMOct 16, 4 PMOct 17, 12 AMOct 17, 8 AMOct 17, 4 PMOct 18, 12 AMOct 18, 8 AMOct 18, 4 PMOct 19, 12 AMOct 19, 8 AM
Together Cloudflare AkashML DeepInfra (Turbo) (us) CoreWeave NovitaAI Groq SambaNova Turbo Parasail

Uptime

100% of requests were served successfully over the last 3 days, across all providers.

Last 3 daysAvailability 99.96%
Oct 16, 11:00 AMNow
Availability over the last 24 hoursAvailability 99.97%
100% 98% 97% 95%
Oct 18, 11:40 AMNow

Apps

Tools that most often run Llama 3.3 70B Instruct, by tokens this week.

Activity

167.5B tokens processed market-wide in the last month.

02.5B5.0B7.5B10.0B
Sep 19Sep 24Sep 29Oct 4Oct 9Oct 14Oct 19
Tokens

FAQ

Which model id do I use for Llama 3.3 70B Instruct?
meta-llama/llama-3.3-70b-instruct. Send it as the model field from any OpenAI-compatible client, against http://192.168.1.55/v1.
How much context does Llama 3.3 70B Instruct hold?
131,072 tokens per request, of which up to 16,384 can be the response.
What can it accept and return?
Input: text. Output: text.
What does Llama 3.3 70B Instruct cost on a pass?
Nothing beyond the pass itself. Pay-as-you-go elsewhere it lists at $0.10 per 1M input tokens and $0.32 per 1M output tokens.
How current is its knowledge?
Training data runs to Dec 2023. Anything later has to come from your prompt or a tool call.
Who built Llama 3.3 70B Instruct?
Meta, in the Llama family.

Explore

Other models your pass can call.