reasoning

reasoning

Covers how to reliably extract high-quality reasoning from LLMs: prompt design, decomposition strategies, output validation, and evaluation at scale.

1) Prompt architecture patterns

# Structure: System → Context → Task → Constraints → Output format
PROMPT_TEMPLATE = """
[SYSTEM]
You are {persona}. {core_capability_statement}.

[CONTEXT]
{relevant_background}

[TASK]
{specific_instruction_verb_first}

[CONSTRAINTS]
- {constraint_1}
- {constraint_2}

[OUTPUT FORMAT]
{exact_format_specification}
"""

# Verb-first task instructions improve instruction-following
GOOD = "Classify the following text as..."
BAD  = "I would like you to think about classifying..."

2) Chain-of-thought (CoT)

# Zero-shot CoT — append "Think step by step"
def zero_shot_cot(question: str) -> str:
    return f"{question}\n\nThink step by step before giving your final answer."

# Few-shot CoT — provide worked examples
FEW_SHOT_COT = """
Q: Is the keto diet safe for type 2 diabetics?
Thinking: Type 2 diabetes involves insulin resistance. Keto reduces carbs → lower blood glucose →
reduced insulin demand. Studies show HbA1c improvements. Risk: hypoglycaemia if on medication.
Answer: Generally beneficial but requires medical supervision and medication adjustment.

Q: {new_question}
Thinking:"""

# Scratchpad pattern — separate reasoning from answer
SCRATCHPAD = """
Work through this in a <scratchpad> block, then give your final answer after </scratchpad>.

{question}
"""

3) Self-consistency (ensemble reasoning)

# Generate N independent answers, take majority vote
async def self_consistent_answer(prompt: str, n: int = 5, temperature: float = 0.7) -> str:
    answers = await asyncio.gather(*[
        llm_call(prompt, temperature=temperature) for _ in range(n)
    ])
    # Extract final answers and vote
    finals = [extract_final_answer(a) for a in answers]
    from collections import Counter
    return Counter(finals).most_common(1)[0][0]

# Useful when: high-stakes classification, numeric estimation, factual Q&A
# Not useful when: creative writing, style tasks, open-ended generation