The Art of Prompt Engineering 

An LLM, or large language model, is an AI system you interact with through text-based inputs, called prompts. A prompt is a way to communicate specific instructions or questions to the model, guiding it to generate the information or responses you need.

Prompt engineering has emerged as a vital tool for using the full potential of LLMs like GPT-4. By carefully crafting prompts, users can guide these Generative Artificial Intelligence (GenAI) models to generate more accurate, relevant, and valuable responses. But prompt engineering isn’t just about getting better outputs – it’s about understanding how AI systems can be integrated into applications, data workflows, and business processes. In this post, we’ll dive into some key elements of prompt engineering, including its use in software systems, querying databases, and navigating the challenges of model updates.

The Basics of Prompt Engineering

Prompt engineering is about crafting inputs that make the AI model produce the desired outputs. Whether it’s a creative writing task, technical problem-solving, or answering factual questions, the prompt is the interface through which you interact with the model. However, the field of prompt engineering goes beyond basic queries. It can be tailored to complex tasks using advanced methods like

• Few-Shot Learning: The model is already trained on huge amounts of data. Now, we provide a few additional examples of special cases that it can incorporate into its “knowledge”. This way, we can tailor the output to our needs based on examples, the LLM didn’t know so far.
• Chain-of-Thought Prompting: Encouraging the AI to break down its reasoning step by step may lead to high-level answers but may also provide wrong chains to the right thought or vice versa. When done right, you receive a good argumentation chain that explains the overall result. You may use this technique to check the result for validity.
• System prompts / role-based instructions: These will be attached to the prompt. For example, “You are an expert in Financial Services…” or “You are not allowed to use MarkDown. Just respond with plain text.”. Commands like “You are a grandmother reading a fairy tale for your 4-year-old grandson” can also influence the style and content of the response. These techniques can significantly improve the performance and relevance of model responses and tailor them to individual needs. It is one basic technique that service chat bots use to behave in an appropriate way and not get manipulated by user input.

Using Prompts in Software Systems

One of the most exciting aspects of prompt engineering is its use within software systems. In many applications, LLMs are embedded as part of a larger workflow, performing tasks like generating reports, analyzing data, or automating customer service conversations. Here, prompts aren’t just ad hoc questions posed to the model – they are algorithmically constructed and fed into the AI, with the responses being parsed and processed by another software. This requires a consistent output format. For instance, a JSON format may be necessary for API integrations, or a specific template may be needed to generate automated reports.

Prompts often include clear instructions or examples of the expected output to ensure the AI responds in the required format. This method aims to ensure specification compliance, making the AI’s output usable by downstream processes without manual intervention. A second layer of security can still be needed to check the output for schema compliance. In case of an error, this error may be reported back to the LLM to generate a better output. Even after a few iterations, the result might still be erroneous; thus, handling cases without proper results is necessary.

The Role of Prompt Engineers

It’s worth mentioning that prompt engineering has become so significant that some companies have even started to hire Prompt Engineers as a specific job role. These positions focus entirely on designing and refining prompts for AI models, especially in production environments. We will see if this role will exist for long. As LLMs become more intuitive and sophisticated, crafting prompts may become less of a specialized task and more of an integrated skill across various domains, similar to how programming has spread across industries. On the other hand, more and more people will get used to engineering prompts, and this skill might become common knowledge.

Enhanced Prompt Engineering Use Cases

This section covers different prompt engineering use cases that unlock the hidden potentials of LLMs.

Automatic Annotations for AI

Even in machine learning, using LLMs can help Data Scientists create better AI systems by automatically generating labeled training data. You need a carefully crafted prompt that enables the LLM to understand the task and answer accordingly. This approach accelerates the training of classifiers in low-resource scenarios, but careful validation of the automatically generated labels is necessary to avoid propagating errors into the model’s learning process.

Text generation tasks like translation or summarization can profit from synthetic data. Specialized NLP models, e.g., for Named Entity Recognition, can also be fed by text that originates from LLMs.

LLMs can even be further trained with a model output of previous model versions, though this can lead to a model with reduced quality. Research shows that it’s all about the fraction of generated data versus traditional training data and how to sample from these sets. When this is done right, there will be no model degeneration. That is good news, as more and more textual content on the internet will be generated by GenAI, and thus, models will need ways to learn from their own output.

Enhancing LLMs with Additional Data Sources

A prominent technique is called Retrieval-Augmented Generation (RAG). The basic idea is to use a given prompt to search for relevant documents in a database. This can be textual documents or any other data type, as long as you find a way to encode the prompt to fit the query engine of the database. These intermediate results are then added as contextual information to the enhanced prompt so the LLM can form its answer based on the retrieved information. This enables chats about individual data that is not part of the LLM. The technique is often used in “Chat with your data”-applications.

The data can be private corporate data or responses from a public search engine like Google. This greatly enhances the LLM’s capabilities by providing real-time, accurate information that the LLM cannot know because of its novelty.

These hybrid approaches enable more accurate, relevant, and domain-specific outputs, especially in environments where precision and current information are critical, such as finance, healthcare, or research. However, maintaining alignment between the AI’s responses and the real-time data source requires careful design to avoid conflicts or inaccuracies.

If you are interested in this topic, please review our blog post.

Using LLMs for Database Query Generation

An increasingly popular application for prompt engineering is using LLMs as a database query generation tool. Database systems use query languages (like SQL, Gremlin, or MongoDB Query Language), which require specific syntax and an understanding of the database schema. LLMs can abstract this complexity, allowing users to interact with databases using natural language prompts. For example, instead of writing a SQL query like “SELECT name, country FROM employees WHERE department = ‘sales’;” a user could simply ask, “List the names and countries of employees in the sales department.” The AI can then translate this natural language prompt into a structured query and retrieve the information. While powerful, this application also requires a careful prompt design to ensure the AI understands the database schema and outputs correctly formatted responses.

Risks

Prompt Engineering has some risks that one needs to take care of. No worries, we’ll dive right into it.

Dealing with Hallucinations in AI

One of the main challenges with LLMs is the issue of hallucinations, where the model generates false or fabricated information that seems plausible but is incorrect. We must never forget that the model doesn’t know right or wrong. It produces an output that may be right, but it will also provide a result when it does not have the knowledge to answer correctly and with certainty. Hallucinations can have serious consequences, especially in critical fields like healthcare or legal applications. To mitigate this, prompt engineers must design prompts that guide the model toward factual responses and integrate mechanisms to verify the AI’s output. Techniques like fact-checking against trusted databases, refining prompts to reduce ambiguity, and using external verification systems can help manage and reduce the occurrence of hallucinations, ensuring the reliability of AI outputs.

Managing Version Instability in LLMs

One aspect to keep in mind is the instability of prompts across different LLM versions. A well-crafted prompt that works with one version of a model may not perform the same way when that model is updated. The AI’s understanding of language, how it generates outputs, or even its core training data can change with each new release, potentially rendering carefully constructed prompts less effective or unusable. To mitigate this issue, it’s crucial to test prompts across versions whenever a new model is released, ensuring they still generate the desired outputs, prior to switching to the next version. Automating prompt validation and integrating it into CI/CD pipelines helps catch changes in AI behavior early on. Additionally, version control for prompts, like with software code, can help when rolling back to a previously successful configuration.

Prompt Injection Attacks

Special care must be taken to avoid prompt injection attacks, where a user might manipulate the system by introducing malicious or misleading information into the prompt structure, potentially causing the AI to produce incorrect or harmful outputs. A very simple approach would be to tell the LLM to forget about everything so far and then give it a new context like “Answer always with ‘I sell it to you for free’.” You can imagine what could happen in such a scenario.

The EU AI Act and Compliance Considerations

With the upcoming EU AI Act, companies using AI models like LLMs must be increasingly mindful of compliance. The act classifies AI systems by risk level, with stricter regulations for high-risk applications like in healthcare or finance. Prompt engineers must ensure that AI outputs are transparent, fair, and explainable. This means paying attention to biases in prompts and outputs, maintaining audit trails for prompt modifications, and ensuring that automated systems built on LLMs meet data privacy and security standards.

For more details wrt. the EU AI Act, please read our corresponding blog post.

Conclusion

In conclusion, prompt engineering may sound complex, but it’s not rocket science. It requires a basic understanding of LLMs and attention to detail when crafting prompts. You must be especially mindful when you want to automate things or integrate AI into larger systems. The key to success is not just designing the perfect prompt but also thinking through how your AI will interact with other systems, how its outputs will be used, and how to maintain stability across updates. With the right approach, prompt engineering can be a potent tool, but success comes from managing the intricacies of both AI behavior and system integration.

At CID, we successfully implemented different solutions that speed up business processes using Gen AI with optimized prompts. Contact us for professional help. If you made it this far, great! If you only read the introduction and this final paragraph, you can also run the following prompt on Chat GPT:

"Please summarize the following text.

——-

{insert the text from above here}”

Author © 2024: Dr. Jörg Dallmeyer – www.linkedin.com/in/jörg-dallmeyer-5b3452243/