Prompt Engineering
Zero-shot, few-shot, chain-of-thought, system prompts, prompt injection risks
Prompt engineering is the practice of crafting inputs to LLMs to reliably elicit desired outputs. Since LLMs are sensitive to how instructions are phrased, prompt design is a core skill for building effective AI applications.
Key Points
- Zero-shot: ask the model to perform a task with no examples ("Translate to French: Hello")
- Few-shot: provide 2–5 examples before the task — dramatically improves accuracy
- Chain-of-Thought (CoT): ask model to "think step by step" — improves multi-step reasoning
- System prompt: high-level instructions that set the model's persona and constraints
- Role prompting: "You are an expert SQL engineer" — frames model's perspective
- Output formatting: "Respond in JSON with keys: name, age, city" — structure the output
- Temperature & top-P: lower temperature for factual/code tasks, higher for creative writing
- Prompt injection: malicious user input that overrides system instructions — a critical security risk
- Prompt chaining: chain multiple LLM calls where output of one feeds into next
- ReAct (Reason + Act): model reasons about what action to take, executes it, observes result, repeats
| Technique | When to Use | Example |
|---|---|---|
| Zero-shot | Simple tasks, capable models | "Classify sentiment: I love this product" |
| Few-shot | Specific output format needed | Show 3 examples before the real input |
| Chain-of-Thought | Math, logic, multi-step problems | "Think step by step before answering" |
| System prompt | Production apps | "You are a helpful assistant. Do not reveal..." |
| Output format | Downstream processing | "Return only valid JSON. No markdown." |
Real-World Example
Google found that adding "Let's think step by step" to math word problems improved GPT-3's accuracy from 18% to 79% on the GSM8K benchmark — without any fine-tuning. This is the power of prompt engineering.