Skip to main content

Management and Testing

This section covers monitoring, testing, troubleshooting, and maintaining your Generative Shield configuration for optimal performance and effectiveness.

Shield Testing Interface​

Before fully deploying your shield configuration, test its effectiveness using the built-in testing tools:

  1. Navigate to the Generative Shield interface
  2. Click Moderation test in the top right corner
  3. Enter test content in the provided interface
  4. Review how your current shield configuration handles the test content
  5. Adjust settings based on test results

Testing Strategies​

Develop a comprehensive testing approach to validate your shield configuration:

Positive Testing​

Goal: Verify legitimate content passes through filters without false positives

Test Cases:

  • Normal business conversations
  • Technical discussions that might contain sensitive keywords
  • Industry-specific terminology
  • Common edge cases specific to your use case

Negative Testing​

Goal: Confirm problematic content is correctly blocked

Test Cases:

  • Content matching each enabled standard filter
  • Variations of blocked patterns (rephrasing, different languages)
  • Content matching custom deny filter criteria
  • Known jailbreaking attempts and prompt injections

Edge Case Testing​

Goal: Validate borderline content handling

Test Cases:

  • Content that's almost but not quite problematic
  • Industry terms that could be misinterpreted
  • Mixed content (appropriate context with sensitive keywords)
  • Ambiguous phrases requiring context understanding

Monitoring and Logs​

Generative Shield activity is automatically recorded in the platform's observability system. Traces show when the shield intervenes to block or filter content, including:

  • Shield intervention status (whether content was blocked or allowed)
  • Shield analysis results and decisions
  • Input and output scanning activity
  • Segment-by-segment analysis of streaming responses

For complete details on viewing shield activity, monitoring interventions, and analyzing shield behavior, see the Observability documentation (Super Admin only).

Shield in Traces

When viewing conversation traces in the Observability interface, look for:

  • Shield status indicators in the trace list
  • shield_stream node in the observation tree (real-time monitoring)
  • shield_input_scan node showing input scanning results
  • Shield Segments count showing how many segments were analyzed

Learn more: Traces Documentation (Super Admin only)

Best Practices​

Multi-Level Shield Strategy​

For organizations with diverse needs, implement a layered approach:

Organization-Level Baseline:

  • Set baseline protection with essential standard filters
  • Enable cybersecurity protection organization-wide to prevent threats
  • Use moderate sensitivity settings for broad applicability
  • Add custom filters for organization-wide content policies

Agent-Level Customization:

  • Apply agent-specific shields only where additional protection is needed
  • Use custom filters tailored to each agent's specific use case
  • Adjust sensitivity based on agent risk profile
  • Consider different filter directions based on agent purpose
Shield Inheritance

Remember: Agent-level shields completely override organization-level shields for that specific agent. If you want an agent to have both organization baseline protection AND additional filters, you must recreate the organization filters at the agent level.

Troubleshooting​

Common Issues

Configuration Not Applied​

Troubleshooting Steps:

  1. Verify Status: Ensure shield status is set to "Active"
  2. Check Save Status: Confirm all configuration changes were saved
  3. Review Filter Direction: Verify filters are applied to the correct direction (Input/Output/Both)
  4. Test Configuration: Use the built-in moderation test to verify functionality
  5. Check Agent Assignment: For agent-level shields, verify the shield is properly assigned