openjev-sglang

by ekzhang · AI Tool · ★ 333

About openjev-sglang

openjev-sglang A server implementing the TypeSafe/Jev HTTP API with Qwen3.6-35B-A3B on SGLang. Each container one B200 with SGLang 0.5.19's Rust frontend, radix caching, and breakable prefill CUDA graphs. A separate Python API process uses FastAPI, uvloop, the Rust-backed HF tokenizer, and pooled asynchronous HTTP connections to SGLang on localhost. CUDA dependencies stay in SGLang's container; on your laptop installs only the API, deployment tools, and tests. Run on Modal The deployment prints a URL. It uses a Modal Server, , , and .

jevllmstructured-generationsystemone

Quick Facts

Stars333
Forks46
LanguagePython
CategoryAI Tool
Quality Score66.269155881763/100
Open Issues5
Last Updated2026-10-01
Created2026-09-17
Platformspython
Est. Tokens~16k

Compatible Skills

These tools work well together with openjev-sglang for enhanced workflows:

  • reflex — semantic(0.27)+complementary+shared_fw(huggingface)+same_lang+similar_pop+shared_platform (67%)
  • jevmeter — semantic(0.32)+complementary+rare_topics+same_lang+similar_pop+shared_platform (66%)
  • NanoJev — semantic(0.21)+complementary+shared_fw(huggingface)+same_lang+similar_pop+shared_platform (65%)
  • jev-on-a-laptop — semantic(0.20)+complementary+shared_fw(huggingface)+same_lang+similar_pop+shared_platform (65%)

More AI Tool Tools

Explore other popular ai tool tools:

View all AI Tool tools →

Popular Python Agent Tools

Frequently Asked Questions

What is openjev-sglang?

openjev-sglang is Jev-compatible API endpoint based on open models (prefill-only). It is categorized as a AI Tool with 333 GitHub stars.

What programming language is openjev-sglang written in?

openjev-sglang is primarily written in Python. It covers topics such as jev, llm, structured-generation.

How do I install or use openjev-sglang?

You can find installation instructions and usage details in the openjev-sglang GitHub repository at github.com/ekzhang/openjev-sglang. The project has 333 stars and 46 forks, indicating an active community.

View on GitHub → Browse AI Tool tools