Repository navigation
Strata Backend
#12514
Replies: 2 comments
|
@mudler do you plan to add native support of Strata into LocalAI? It works really fast - 60-80 tokens/sec on 32GB VRAM + 32GB RAM |
0 replies
|
I'm currently working on fork of your backend with optional support of Polystrata rewritten in Rust and made in native way. IDK, maybe it will be ready for being published in 1-3 weeks |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Strata is a new interference engine for Qwen3.8 Flash Next with almost 15k stars on Github: https://github.com/Niko1221/Strata
Strata pushend my local AI expirience to a new level, but I wanted to keep LocalAI as frontend. So I came up with a gRPC backend for Strata: https://github.com/jangatzke/strata-localai-backend/
I've been using Hermes Agent, OpenWebUI and Zoo Code all day yesterday consuming 50 million input and 0.6 million output tokens. So it is safe to say the backend is stable enough for release.
Does the LocalAI team plan to release a native backend for Strata?
All reactions