🏁 Response speed

Question:

We have a ChatGPT 4 account, and when I test there, it goes really quick to get a reply. ChatGPT 4 made a complete reply within 15-20 seconds. In that same time, the test account we are using, did you even start writing. It took about 2 minutes before it gave up. Second time we tested it took about 45 seconds before it started to write. Right now, it is just timing out all the time – maybe OpenAI have some problems? So, would performance be better using our own OpenAI key?”

I understand your question. Unfortunately, using your own API key won’t significantly address this issue. It doesn’t stop you from using it, but it won’t solve this particular problem.

The only thing that can speed up response generation is reducing the question’s size. In the widget settings, under the GPT tab, there’s an option called “Document,” which determines how many parts of your resource are included in the context for generating the answer.

Reducing this value will affect generation speed. Also, turning off additional Prompt settings will make response generation a bit faster, but not significantly.

Soon, in OmniMind, the GPT-4-Turbo model will become available, and with it, your Omni-bot’s work will be several times faster. That might be the best solution to this problem.

Was this article helpful to you? Yes No

How can we help?