Start free trial

WizardLM-2 8x22B

microsoft/wizardlm-2-8x22b
CompareAPITry this model

WizardLM-2 8x22B is Microsoft AI's most advanced Wizard model. It demonstrates highly competitive performance compared to leading proprietary models, and it consistently outperforms all existing state-of-the-art opensource models.

Modalities
In / out per 1M$0.62 / $0.62
Context64K
ReleasedApr 16, 2024

Providers

1 provider serves this model upstream.

Providers serving this model
ProviderInput /MOutput /MCache read /MLatency P50Throughput P50Uptime
NovitaAIbf16$0.62$0.62Variable1.78s15 tps100%

Market-wide measurements, refreshed every 15 minutes. They describe who serves WizardLM-2 8x22B in general, not how ProjectCOZY routes your requests.

Pricing

List rates per 1M tokens — what WizardLM-2 8x22B costs pay-as-you-go elsewhere, not what you pay here.

Input
$0.62
per 1M tokens
Output
$0.62
per 1M tokens
Cache read
—
not published
Included in every pass
WizardLM-2 8x22B is included in every ProjectCOZY pass at no extra cost — one flat rate, no per-token metering. The rates above are there for comparison.

Performance

Time to first token and output speed across every provider, over the last 3 days.

0.0s10.0s20.0s30.0s40.0s
Oct 16, 12 AMOct 16, 8 AMOct 16, 4 PMOct 17, 12 AMOct 17, 8 AMOct 17, 4 PMOct 18, 12 AMOct 18, 8 AMOct 18, 4 PMOct 19, 12 AMOct 19, 8 AM
Median 90th percentile 99th percentile
0 tok/s20 tok/s40 tok/s60 tok/s80 tok/s
Oct 16, 12 AMOct 16, 8 AMOct 16, 4 PMOct 17, 12 AMOct 17, 8 AMOct 17, 4 PMOct 18, 12 AMOct 18, 8 AMOct 18, 4 PMOct 19, 12 AMOct 19, 8 AM
Median 90th percentile 99th percentile

Uptime

100% of requests were served successfully over the last 3 days, across all providers.

Last 3 daysAvailability 96.56%
Oct 16, 10:00 AMNow
Availability over the last 24 hoursAvailability 96.35%
100% 93% 87% 80%
Oct 18, 10:20 AMNow

Apps

Tools that most often run WizardLM-2 8x22B, by tokens this week.

HammerAIRoleplay · 42.9% of tokens553.3MJuicyChatRoleplay · 18.4% of tokens236.5MAgnaistic13.5% of tokens174.2MSillyTavernRoleplay · 13.5% of tokens173.5MJanitor AIRoleplay · 11.7% of tokens151.1M

Activity

5.1B tokens processed market-wide in the last month.

0100.0M200.0M300.0M400.0M
Sep 19Sep 24Sep 29Oct 4Oct 9Oct 14Oct 19
Tokens

FAQ

Which model id do I use for WizardLM-2 8x22B?
microsoft/wizardlm-2-8x22b. Send it as the model field from any OpenAI-compatible client, against http://192.168.1.55/v1.
How much context does WizardLM-2 8x22B hold?
65,535 tokens per request, of which up to 8,000 can be the response.
What can it accept and return?
Input: text. Output: text.
What does WizardLM-2 8x22B cost on a pass?
Nothing beyond the pass itself. Pay-as-you-go elsewhere it lists at $0.62 per 1M input tokens and $0.62 per 1M output tokens.
How current is its knowledge?
Training data runs to Apr 2024. Anything later has to come from your prompt or a tool call.

Explore

Other models your pass can call.

Aion 3.5aion-labs/aion-3.5256KAionAion 3.5 Miniaion-labs/aion-3.5-mini256KAionAion-2.0aion-labs/aion-2.0128KAionAion-3.0aion-labs/aion-3.0128KAionAion-3.0-Miniaion-labs/aion-3.0-mini128KAionAion-RP 1.0 (8B)aion-labs/aion-rp-llama-3.1-8b32KAion