[MOSH-2320]: Update API docs for worker number exposure#240
[MOSH-2320]: Update API docs for worker number exposure#240jli-together merged 2 commits intomainfrom
Conversation
✱ Stainless preview buildsThis PR will update the go openapi python terraform typescript
|
openapi.yaml
Outdated
| type: integer | ||
| minimum: 1 | ||
| description: "Number of concurrent workers for inference requests. Overrides the default concurrency for this model. Useful for tuning throughput when using proxy endpoints (e.g. OpenRouter) or rate-limited external APIs." | ||
| example: 9 |
There was a problem hiding this comment.
the examples are a bit weird, maybe we can just say 5
Related to PR changes: https://github.com/togethercomputer/together-evaluation/pull/165