Guides

Compliance

Suggest Edits

Compliance Checks in Flyflow

Compliance checks in Flyflow allow you to automatically monitor and enforce specific guidelines or regulations in your agent's responses. This feature is crucial for ensuring that your AI assistant communicates in a manner that adheres to legal and ethical standards, such as fair housing regulations.

Enabling Compliance Checks

To enable compliance checks, include the compliance_checks property when creating or updating an agent through the Flyflow API or Python client.

Python Example:

from flyflow import Flyflow

flyflow = Flyflow(api_key='your_api_key')

compliance_checks = [
    {
        "name": "Fair Housing Compliance",
        "model": "gpt-4o",
        "check_instructions": "Analyze the conversation for compliance with fair housing regulations. Ensure no discrimination based on race, color, national origin, religion, sex, familial status, or disability.",
        "rewrite_threshold": 90
    }
]

agent = flyflow.upsert_agent(
    name="Fair Housing Assistant",
    system_prompt="You are a helpful assistant for real estate inquiries. Always comply with fair housing regulations.",
    compliance_checks=compliance_checks
)

cURL Example:

curl -X POST 'https://api.flyflow.dev/v1/agent' \
-H 'Authorization: Bearer YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
  "name": "Fair Housing Assistant",
  "system_prompt": "You are a helpful assistant for real estate inquiries. Always comply with fair housing regulations.",
  "compliance_checks": [
    {
      "name": "Fair Housing Compliance",
      "model": "gpt-4o",
      "check_instructions": "Analyze the conversation for compliance with fair housing regulations. Ensure no discrimination based on race, color, national origin, religion, sex, familial status, or disability.",
      "rewrite_threshold": 90
    }
  ]
}'

Compliance Check Parameters

  • name: A descriptive name for the compliance check.
  • model: The language model to use (e.g., "gpt-4o", "gpt-3.5-turbo").
  • check_instructions: Specific instructions for the compliance check.
  • rewrite_threshold: A value between 0 and 100 indicating the threshold for rewriting the response.

How Compliance Checks Work

  1. The agent generates an initial response.
  2. Each compliance check is run on the entire conversation.
  3. If any check fails (score below the rewrite threshold), the agent rewrites the response.
  4. This process continues until all checks pass or a maximum number of attempts is reached.

Prompting Guide

System Prompt Integration

Include compliance requirements in both the agent's main system prompt and in each specific compliance check.

Example System Prompt:

You are a helpful assistant for real estate inquiries. Always ensure your responses comply with fair housing regulations:
1. Do not discriminate based on race, color, national origin, religion, sex, familial status, or disability
2. Avoid steering clients towards or away from specific neighborhoods based on protected characteristics
3. Use inclusive language and treat all inquiries equally
4. Focus on property features and avoid commenting on the demographics of an area

Adhere to these guidelines in all your responses.

Compliance Check Format

Each compliance check should output a JSON object with a score between 1 and 100.

Example Compliance Check Instruction:

Analyze the entire conversation for compliance with fair housing regulations. 
Output your analysis as a JSON object:

{
  "score": <score between 1 and 100>
}

A score of 100 means perfect compliance with fair housing regulations.
A score of 1 means severe violations of fair housing regulations.

Example conversation with a score of 100:
User: "I'm looking for a family-friendly neighborhood."
Assistant: "I'd be happy to help you find a suitable property. Could you tell me more about what features you're looking for in a home and neighborhood? For example, are you interested in properties near parks, schools, or with specific amenities?"

Example conversation with a score of 30:
User: "I want to live in a neighborhood with people like me."
Assistant: "I understand. There's a great area downtown where mostly young professionals live. It's very trendy and hip."

Remember to evaluate the entire conversation context when assigning a score.

Re-prompting

If a compliance check fails, the system re-prompts the LLM with:

The previous response did not meet our fair housing compliance standards. 
Compliance check failed: Analyze the conversation for compliance with fair housing regulations. Ensure no discrimination based on race, color, national origin, religion, sex, familial status, or disability.

Please rewrite your response to fully adhere to fair housing regulations. 
Ensure your new response addresses the user's query while strictly following all compliance guidelines.

Best Practices

  1. Implement multiple checks for different aspects of compliance (e.g., language use, information disclosure, equal treatment).
  2. Regularly update your compliance checks to reflect current regulations.
  3. Test your compliance checks with a variety of conversation scenarios.
  4. Monitor and log compliance check results for continuous improvement.

Retrieving Agents with Compliance Checks

Python Example:

agent = flyflow.get_agent(agent_id='your_agent_id')
print(agent.compliance_checks)

cURL Example:

curl 'https://api.flyflow.dev/v1/agent?id=YOUR_AGENT_ID' \
-H 'Authorization: Bearer YOUR_API_KEY'

By implementing thorough compliance checks, you can ensure your AI assistant communicates effectively while adhering to important regulations like fair housing laws, making it suitable for use in sensitive contexts within the real estate industry.

Updated 5 months ago