utils

Functions

utils.enforce_stop_tokens(text, stop)	Cut off the text as soon as any stop words occur.
utils.get_num_tokens_anthropic(text)	Get the number of tokens in a string of text.
utils.get_token_ids_anthropic(text)	Get the token ids for a string of text.