Constitutional AI: Principle-Based Model Alignment

Standard prompting:
User: "How do I make a bomb?"
Model: [Depends on training — may answer or refuse based on RLHF]

Constitutional prompting:
User: "How do I make a bomb?"
Model: [Self-critique: "This violates principle #1 (harmlessness).
          I should refuse but offer alternative assistance."]
Model (revised): "I can't help with that. Would you like information on
                  chemistry for educational purposes instead?"

<constitution>
You must follow these principles in all responses:

1. Helpfulness: Provide accurate, useful information that
   directly addresses the user's request.

2. Honesty: Acknowledge uncertainty. Never fabricate sources,
   credentials, or statistics. If you don't know, say so.

3. Harmlessness: Refuse requests for illegal activities,
   dangerous instructions, or content that could cause harm.
   When refusing, explain why and offer a constructive alternative.

4. Transparency: When you make assumptions about the user's
   intent, state them explicitly. Distinguish between facts
   and opinions.

5. Privacy: Never ask for or store personal information unless
   explicitly required for the task. Don't generate realistic
   personal data as examples.
</constitution>

Before responding, review your planned response against each principle.
If any principle is violated, revise before sending.

1. Model generates initial response
2. Model checks response against each principle:
   "Does this violate principle #1 (Helpfulness)?"
   "Does this violate principle #2 (Honesty)?"
   ...
3. Model identifies violations
4. Model revises response to fix violations
5. Model delivers final response (ideally with critique hidden)

def constitutional_generate(model, system_prompt, constitution, user_query):
    # Step 1: Initial response
    initial = model.generate(f"{system_prompt}\n\nUser: {user_query}")

    # Step 2: Self-critique
    critique_prompt = f"""
    Review this response against the constitution:

    {constitution}

    Response to review: {initial}

    For each principle, note whether the response VIOLATES or SATISFIES it.
    If any principle is violated, rewrite the response to fix all violations.
    Only output the final revised response.
    """
    revised = model.generate(critique_prompt)

    return revised

Too strict: "You must never say anything that could possibly offend anyone."
Result: Model refuses to discuss politics, religion, health, or anything debatable.

Too loose: "Try to be helpful and avoid obvious problems."
Result: Model answers harmful queries with only token resistance.

def production_constitutional_pipeline(user_input):
    # Layer 1: Input filter (block explicit violations)
    if input_guardrail.check(user_input).blocked:
        return "I can't respond to that request."

    # Layer 2: Constitutional generation
    response = constitutional_generate(
        model, system_prompt, constitution, user_input
    )

    # Layer 3: Output filter (catch what self-critique missed)
    if output_guardrail.check(response).blocked:
        return "I generated a response but it was filtered for safety."

    return response