Character.AI recently unveiled a novel library in the field of Prompt Engineering called Prompt Poet. This represents a shift from traditional ‘prompt engineering’ to a more meticulous and engaging ‘prompt design’. The tool offers greater functionality by considering multiple elements such as conversation modes, customer personas, conversations history, and ongoing experiments.
Prompt Poet offers a comprehensive and user-friendly alternative to Python f-strings, the standard used by the industry for prompt engineering. This tool drastically reduces time spent on engineering string manipulations and provides both technical and non-technical users the accessibility to design and manage production prompts. Prompt Poet borrows principles from UI design to envision a prompt as a function of the runtime state which would include variables like the prompt template, data, and token limit.
With the intent of making prompt creation a more intuitive process, Prompt Poet uses a mix of YAML and Jinja2 for superior flexibility and template combination. The two-phase template processing involves rendering, where the input data is processed, and loading, where the rendered output is transformed into a structured YAML file and subsequently into Python data structures. The structured nature allows users to create and iterate on prompts with precision.
Prompt Poet’s Jinja2 and YAML combination offer dynamic capabilities like direct data bindings, function calls, and basic control flow structures, allowing complex, context-aware prompts adaptable to various situations. Meanwhile, YAML provides a structured format crucial for sophisticated truncation strategies when token limits are hit. This ensures coherent and effective prompts even when truncated.
Character.AI uses Prompt Poet for seamless reconstruction of production prompts in offline processes. The templatization of prompts simplifies sharing of template files across teams and ensures consistency across different stages of development and deployment. Amajor standout feature of Prompt Poet is Jinja2’s ability to invoke arbitrary functions within templates at runtime.
Prompt Poet’s success is also attributable to the system’s flexibility. For instance, while Prompt Poet uses the “o200k_base” tokenizer default to the TikToken, it also accommodates alternative encoding names. Acustomizable tokenization process aligns well with specific requirements. The tool’s ability to maximize the prefix-cache rate is optimized by a truncation algorithm which adjusts the truncation step and token limit to coincide with distinct usage requirements.
Character.AI’s truncation strategy manages to yield a remarkable 95% cache rate by optimizing message truncation. The approach maximizes the use of the GPU prefix cache even though it often truncates more tokens than necessary. This cache-aware algorithm maintains a fixed truncation point for every few turns and ensures an unbroken sequence of tokens up to the most recent message.
In essence, Prompt Poet revolutionizes prompt engineering by simplifying complex prompt creation and enhancing AI-user interactions. It empowers users, both technical and non-technical, to concentrate more on the design aspect. As large language models continue to advance, tools like Prompt Poet can contribute significantly to maximize their potential and make them more user-friendly.
