When I use scaleAI's deployment endpoint, if the generated text doesnt get sent back within ~30s, I get an error about a 504 error.
If the response comes back quickly, I dont.
This makes it very difficult to use for long text generation from GPTx's. Setting timeout in my request to 5mins does not change it, so I think its on Scale's end
When I use scaleAI's deployment endpoint, if the generated text doesnt get sent back within ~30s, I get an error about a 504 error.
If the response comes back quickly, I dont.
This makes it very difficult to use for long text generation from GPTx's. Setting timeout in my request to 5mins does not change it, so I think its on Scale's end