With Max. Tokens, you determine the text length of the chatbot's response. This is only a guideline for chat models. The model then attempts to generate a text of approximately the desired length.
What are tokens?
Chat models break down text into segments (tokens) to calculate a possible response. The number of tokens generated from a text depends, among other things, on the language in which the model generates text. In English, 1 token is approximately 4-5 characters, or 1 word is approximately 1.5 tokens.
Which settings are useful?
For normal chat conversations, a value of 50-70 tokens per answer is usually sufficient. If more detailed answers are desired (e.g., Entertain Mode), then values between 80-120 are recommended. In role-playing games, where very detailed answers are often desired, values of 160-260 are usually good. These are good recommendations. However, chat models are very complex systems, so it's advisable to experiment to find out which values work best for you with specific models.
Often, it's also a very good solution to set the maximum tokens higher than suggested above and add a clear instruction to your bot's bio, such as: "Generate a maximum of 3-4 sentences per answer."
In any case, keep in mind that the more text your chatbot generates, the higher its token consumption will be.
Set the maximum tokens in the Chat AI menu:
Set the maximum tokens in the web interface: