by AbdelStark · AI Tool · ★ 26
jev-benchmarks Probability-aware evaluation for typed decision models. measures more than whether a model selects the right label. It evaluates whether the reported probabilities are calibrated enough to support automation, how much work can be accepted at a fixed error budget, what resources each decision uses, and how long it takes end to end. The first study compares TypeSafe Jev with on their shared capability: zero-shot, single-label text classification with per-label probabilities.
| Stars | 26 |
| Forks | 5 |
| Language | Python |
| Category | AI Tool |
| License | Apache-2.0 |
| Quality Score | 67.8602262315025/100 |
| Open Issues | 5 |
| Last Updated | 2026-09-17 |
| Created | 2026-09-17 |
| Platforms | python |
| Est. Tokens | ~15k |
These tools work well together with jev-benchmarks for enhanced workflows:
Explore other popular ai tool tools:
jev-benchmarks is Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.. It is categorized as a AI Tool with 26 GitHub stars.
jev-benchmarks is primarily written in Python. It covers topics such as benchmarking, calibration, evaluation.
You can find installation instructions and usage details in the jev-benchmarks GitHub repository at github.com/AbdelStark/jev-benchmarks. The project has 26 stars and 5 forks, indicating an active community.
jev-benchmarks is released under the Apache-2.0 license, making it free to use and modify according to the license terms.