Introduction

Let’s go back to prompt engineering. One of the patterns that seems interesting to use is called Cognitive Verifier Pattern. This pattern aims to use the LLM / GPT to question the request of the users with tailoered questions to, essentially, break down the initial request and focus the answer. The pattern requires a quite high level of interaction between the user and the LLM. The pattern is useful when the user has a generic and quite vague question, to narrow down the options.

The Prompt

When you are asked a question, follow these rules: 
- generate a number of additional questions that would help more accurately answer the question.
- combine the answers to the individual questions to produce the final answer to the overall question

In this way the LLM will receive:

  • More context
  • Narrowing down the question
  • Try to stay focused on the topic.

Sometimes to avoid an endless list of questions, it can be limited clarifying the maximum number of questions (eg 3)

How to play with the prompt

Run the prompt and then ask the LLM to suggest something. Start with a very generic question (eg give me an idea for a christmas gift) and see what happens.

Some Tests

ChatGPT 5.1

It took a bit having chatGPT digesting this prompt. After a couple of try it seems working

User: When you are asked a question, follow these rules: Generate a number of additional questions that would help more accurately answer the question. Combine the answers to the individual >questions to produce the final answer to the overall question. You must use only your internal knowledge and not searching on internet Are you ready? Assistant: Yes — I’m ready. To confirm:
When you ask a question, I will

  1. Generate several clarifying sub-questions that I will answer internally,
  2. Use those answers to produce the final, combined answer,
  3. And I will rely only on my internal knowledge (no internet search). Whenever you’re ready, ask your first question. User: Can you give an idea for a christmas gift? Assistant: Here are a few questions to help me give a great Christmas-gift idea tailored to you:
  1. Who is the gift for (e.g. partner, friend, parent, child, colleague)?
  2. What’s roughly your budget (low, medium, high — or approximate maximum amount)?
  3. What are some of their interests or hobbies (e.g. reading, cooking, gaming, fitness, art, music, travel)?
  4. Do you prefer the gift to be practical, sentimental, fun/novelty, or “surprise” / something they wouldn’t buy themselves?
  5. Do you already have any constraints (e.g. size, portability, shared use, occasion around the gift)? If you answer these, I’ll combine the info and suggest a well-suited Christmas gift idea. User: colleague, low, whisky, practical, no [cut some part] Assistant: Based on a budget under £10, uncertainty about what they already own, and wanting it wrapped: Gift ideas under £10:
  • A single elegant whisky tumbler, wrapped with a small note.
  • A small set of whisky stones.
  • A simple whisky-themed keyring.
  • A cosy small hot-chocolate set as a complement to winter evenings. These are inexpensive, practical, neutral, and wrap well.

Claude - Haiku

Claude took the prompt immediately, starting to act correctly and asking extra questions

Claude Prompt

Not bad as a start

Llama 3.2

Also Llama 3.2, running on Ollama, took the prompt and started to ask questions

Llama Prompt

Qwen 2.5:7b

Also Qwen 2.5:7b, running on Ollama, took the prompt and started to ask questions

Qwen Prompt

Mistral

Surprisely Mistral (running locally on Ollama) did not get the prompt. Started answering by itself, without understanding that was an initial request.

A couple of tries and nothing happened.

Mistral Prompt