Self-Consistency: Improving Reasoning Through Majority Voting

Standard CoT:  Model → One reasoning path → One answer (greedy, risk of error)
Self-Consistency: Model → 5-10 reasoning paths → Majority vote → Most reliable answer

Question: When I was 6 my sister was half my age. Now I'm 70,
          how old is my sister?

Single CoT output:
"When I was 6, my sister was half my age = 3.
Now I'm 70, so she's 70 / 2 = 35."
→ WRONG (correct answer: 67)

Path 1: "Sister was 3 when I was 6, age difference is 3 years.
        At 70, sister is 70 - 3 = 67." → Answer: 67 ✓

Path 2: "Sister was half my age at 6 = 3 years old.
        70 - 6 = 64 years passed. She's 3 + 64 = 67." → Answer: 67 ✓

Path 3: "Half of 70 is 35." → Answer: 35 ✗

Result: 67 appears twice, 35 appears once → Final answer: 67 ✓

import asyncio
from collections import Counter

async def self_consistency(model, prompt, n_samples=5, temperature=0.7):
    """Generate N reasoning paths and return the majority answer."""
    responses = await asyncio.gather(*[
        model.generate(prompt, temperature=temperature, max_tokens=500)
        for _ in range(n_samples)
    ])

    # Extract final answers from reasoning paths
    answers = [extract_final_answer(r) for r in responses]

    # Majority vote
    counts = Counter(answers)
    best_answer, votes = counts.most_common(1)[0]
    confidence = votes / n_samples

    return {
        "answer": best_answer,
        "confidence": confidence,
        "all_answers": answers,
        "reasoning_paths": responses
    }

def extract_final_answer(text: str) -> str:
    """Extract the final answer from a reasoning chain.
    Looks for patterns like 'The answer is X' or 'Therefore, X'."""
    import re
    patterns = [
        r'(?:answer is|therefore|conclusion:)\s*(.+)',
        r'(?:^|\n)(\d+)\s*$'  # Last line is just a number
    ]
    for pattern in patterns:
        matches = re.findall(pattern, text.lower())
        if matches:
            return matches[-1].strip()
    return text.strip().split('\n')[-1]