Crafting Prompt Sandwiches for Generative AI

,

Large Language Models (LLMs) can give notoriously inconsistent responses when asked the same question multiple times. For example, if you ask for help writing an Elasticsearch query, sometimes the generated query may be wrapped by an API call, even though we didn’t ask for it. This sometimes subtle, other times dramatic variability adds complexity when integrating generative AI into analyst workflows that expect specifically-formatted responses, like queries. In the short term, LLM providers may offer a patchwork of vendor-specific techniques for improving the repeatability of responses.

Fortunately, in addition to these emerging vendor-specific techniques, there’s a universal pattern for structuring prompts at the start of a conversation to steer it in the right direction. We’ll craft a prompt sandwich to improve the quality and consistency of chats with your preferred LLM. We’ll explore the parts of a prompt sandwich and illustrate its practical application in the Elastic AI Assistant.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt432f5a33601f516a/64c93cd6a3f81b3b2f51929a/elastic-blog-1-adalle-ai.jpg,elastic-blog-1-adalle-ai.jpg,

The Elastic AI Assistant integrates Elastic Security detection alerts, events, rules, and data quality checks as context for generative AI chats. The assistant helps make prompt sandwiches and reduces privacy risks through field-level pseudonymization of values.

_________________________________________________
/ \
| You are an expert in Elastic Security. | (1)
\ _______________________________________________/
+————————————————+
| I have an alert for a user named alice. | (2)
+————————————————+
/ \
| How should I investigate it? | (3)
\_______________________________________________/,

Each layer of the prompt sandwich has a name and specific purpose:

1. System prompt

2. Context

3. User prompt

Each layer’s unique purpose is outlined below:

,

	Purpose	Example
System prompt	Steers the conversation and role of the assistant	You are an expert in Elastic Security.
Context	Data or metadata, which may be structured or freeform	I have an alert for a user named alice.
User prompt	Asks a question or makes a request	How should I investigate it?

,

When building and tuning prompts for repeatability, this three-layered format provides guidance on structuring prompts and which parts to modify first.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltc2eafab222154714/64c93ed785f91b7703336ead/elastic-blog-2-chat-button.jpg,elastic-blog-2-chat-button.jpg,

The three layers of the prompt sandwich — (1) System prompt, (2) Context, and (3) User prompt — are labeled in the screenshot below.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt18be903fe82af6f7/64c93f0002b0387127168a78/elastic-blog-3-annotated.jpg,elastic-blog-3-annotated.jpg,

Before any data is sent to the LLM, you may use the Elastic AI Assistant’s inline editing prompt preview (as shown above) to:

• Add new, edit, or remove system prompts
• Add additional context, remove context, and for some types of data, configure field level anonymization
• Preview or edit user prompts selected via the Quick prompts feature

Custom system prompts and user prompts are optionally created via the settings shown in the screenshot below:

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltc1038ea9ca07312e/64c93f32c53087b6e1bc3470/elastic-blog-4-system-prompts.jpg,elastic-blog-4-system-prompts.jpg,

The Settings view above also configures the defaults for data anonymization.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltd14f52285f53aafb/64c93f91ec43999f83ea15d4/elastic-blog-5-values.jpg,elastic-blog-5-values.jpg,

In the example above, the values of three fields will be anonymized when they are sent to the LLM. You may accept the configured defaults or interact with the preview to toggle specific fields.

,

Above: The Show anonymized toggle reveals the anonymized data sent and received from the LLM.

,

The release and timing of any features or functionality described in this post remain at Elastic’s sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

In this blog post, we may have used third party generative AI tools, which are owned and operated by their respective owners. Elastic does not have any control over the third party tools and we have no responsibility or liability for their content, operation or use, nor for any loss or damage that may arise from your use of such tools. Please exercise caution when using AI tools with personal, sensitive or confidential information. Any data you submit may be used for AI training or other purposes. There is no guarantee that information you provide will be kept secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use.

Elastic, Elasticsearch and associated marks are trademarks, logos or registered trademarks of Elasticsearch N.V. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.

Source link