cremote/docs/llm_ada_testing.md

LLM Agent Guide: ADA/WCAG Accessibility Testing with Cremote

Purpose

This document provides LLM coding agents with concrete, actionable guidance for using Cremote's accessibility testing tools to conduct ADA/WCAG compliance audits.

Quick Reference

Tool Selection Matrix

Testing Need	Primary Tool	Secondary Tool	WCAG Criteria
Comprehensive automated audit	web_run_axe_cremotemcp	-	~57% of WCAG 2.1 AA
Color contrast issues	web_contrast_check_cremotemcp	web_gradient_contrast_check_cremotemcp	1.4.3, 1.4.6, 1.4.11
Gradient backgrounds	web_gradient_contrast_check_cremotemcp	-	1.4.3, 1.4.6, 1.4.11
Video/audio captions	web_media_validation_cremotemcp	-	1.2.2, 1.2.5, 1.4.2
Hover/focus content	web_hover_focus_test_cremotemcp	-	1.4.13
Text in images	web_text_in_images_cremotemcp	-	1.4.5, 1.4.9, 1.1.1
Cross-page consistency	web_cross_page_consistency_cremotemcp	-	3.2.3, 3.2.4, 1.3.1
Sensory instructions	web_sensory_characteristics_cremotemcp	-	1.3.3
Animations/flashing	web_animation_flash_cremotemcp	-	2.3.1, 2.2.2, 2.3.2
ARIA validation	web_enhanced_accessibility_cremotemcp	web_run_axe_cremotemcp	1.3.1, 4.1.2, 2.4.6
Keyboard accessibility	web_keyboard_test_cremotemcp	web_run_axe_cremotemcp	2.1.1, 2.4.7
Zoom/resize functionality	web_zoom_test_cremotemcp	-	1.4.4
Responsive design	web_reflow_test_cremotemcp	-	1.4.10
Visual documentation	web_screenshot_cremotemcp	-	Evidence capture
Custom JavaScript testing	console_command_cremotemcp	-	Advanced scenarios

Standard Testing Sequence

1. web_inject_axe_cremotemcp                    # Inject axe-core library
2. web_run_axe_cremotemcp                       # Run comprehensive automated tests
3. web_contrast_check_cremotemcp                # Detailed contrast analysis
4. web_gradient_contrast_check_cremotemcp       # Gradient background contrast (NEW)
5. web_media_validation_cremotemcp              # Video/audio caption validation (NEW)
6. web_hover_focus_test_cremotemcp              # Hover/focus content testing (NEW)
7. web_text_in_images_cremotemcp                # Text-in-images detection (NEW)
8. web_sensory_characteristics_cremotemcp       # Sensory instruction detection (NEW)
9. web_animation_flash_cremotemcp               # Animation/flash detection (NEW)
10. web_enhanced_accessibility_cremotemcp       # Enhanced ARIA validation (NEW)
11. web_keyboard_test_cremotemcp                # Keyboard navigation testing
12. web_zoom_test_cremotemcp                    # Zoom functionality testing
13. web_reflow_test_cremotemcp                  # Responsive design testing

Note: For multi-page sites, also run:

14. web_cross_page_consistency_cremotemcp       # Cross-page consistency (NEW)

Tool Usage Patterns

Pattern 1: Initial Audit

// Step 1: Inject axe-core
{
  "tool": "web_inject_axe_cremotemcp",
  "arguments": {
    "timeout": 30
  }
}

// Step 2: Run comprehensive tests
{
  "tool": "web_run_axe_cremotemcp",
  "arguments": {
    "run_only": ["wcag2a", "wcag2aa", "wcag21aa"],
    "timeout": 30
  }
}

// Step 3: Analyze results and run specialized tests based on findings

Pattern 2: Contrast-Specific Testing

// For pages with known or suspected contrast issues
{
  "tool": "web_contrast_check_cremotemcp",
  "arguments": {
    "selector": "body",  // Test entire page
    "timeout": 10
  }
}

// For specific sections
{
  "tool": "web_contrast_check_cremotemcp",
  "arguments": {
    "selector": ".main-content",  // Test specific area
    "timeout": 10
  }
}

Pattern 3: Keyboard Navigation Testing

{
  "tool": "web_keyboard_test_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - not_focusable: Elements that should be keyboard accessible but aren't
// - no_focus_indicator: Elements missing visible focus indicators
// - keyboard_trap: Elements that trap keyboard focus

Pattern 4: Zoom and Responsive Testing

// Test zoom levels (WCAG 1.4.4)
{
  "tool": "web_zoom_test_cremotemcp",
  "arguments": {
    "zoom_levels": [1.0, 2.0, 4.0],
    "timeout": 10
  }
}

// Test responsive breakpoints (WCAG 1.4.10)
{
  "tool": "web_reflow_test_cremotemcp",
  "arguments": {
    "widths": [320, 768, 1280],
    "timeout": 10
  }
}

Pattern 5: Visual Documentation

// Capture baseline
{
  "tool": "web_screenshot_cremotemcp",
  "arguments": {
    "output": "/tmp/baseline.png",
    "timeout": 5
  }
}

// Capture at 200% zoom
{
  "tool": "web_screenshot_cremotemcp",
  "arguments": {
    "output": "/tmp/zoom-200.png",
    "zoom_level": 2.0,
    "timeout": 5
  }
}

// Capture mobile view
{
  "tool": "web_screenshot_cremotemcp",
  "arguments": {
    "output": "/tmp/mobile-320.png",
    "width": 320,
    "height": 568,
    "timeout": 5
  }
}

Pattern 6: Gradient Contrast Testing (NEW)

// Test specific element with gradient background
{
  "tool": "web_gradient_contrast_check_cremotemcp",
  "arguments": {
    "selector": ".hero-section",
    "timeout": 10
  }
}

// Test all elements with gradient backgrounds
{
  "tool": "web_gradient_contrast_check_cremotemcp",
  "arguments": {
    "selector": "body",  // Scans entire page
    "timeout": 10
  }
}

// Analyze output for:
// - worst_case_ratio: Minimum contrast found across gradient
// - best_case_ratio: Maximum contrast found across gradient
// - wcag_aa_pass: Whether it meets WCAG AA standards
// - wcag_aaa_pass: Whether it meets WCAG AAA standards

Pattern 7: Media Validation (NEW)

// Validate all video/audio elements on page
{
  "tool": "web_media_validation_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - missing_captions: Videos without caption tracks
// - missing_audio_descriptions: Videos without audio description tracks
// - inaccessible_tracks: Track files that cannot be loaded
// - autoplay_violations: Videos that autoplay without controls

Pattern 8: Hover/Focus Content Testing (NEW)

// Test all hover/focus triggered content
{
  "tool": "web_hover_focus_test_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - not_dismissible: Content that cannot be dismissed with Escape key
// - not_hoverable: Tooltip disappears when hovering over it
// - not_persistent: Content disappears too quickly
// - native_title_tooltip: Using native title attribute (violation)

Pattern 9: Text-in-Images Detection (NEW)

// Detect text embedded in images using OCR
{
  "tool": "web_text_in_images_cremotemcp",
  "arguments": {
    "timeout": 30  // OCR is CPU-intensive, allow more time
  }
}

// Analyze output for:
// - missing_alt: Images with text but no alt text
// - insufficient_alt: Images with text but inadequate alt text
// - detected_text: Actual text found in the image
// - recommendations: Specific suggestions for each image

Pattern 10: Cross-Page Consistency (NEW)

// Check consistency across multiple pages
{
  "tool": "web_cross_page_consistency_cremotemcp",
  "arguments": {
    "urls": [
      "https://example.com/",
      "https://example.com/about",
      "https://example.com/contact",
      "https://example.com/services"
    ],
    "timeout": 10  // Per page
  }
}

// Analyze output for:
// - common_navigation: Links present on all pages
// - inconsistent_pages: Pages missing common navigation
// - landmark_issues: Missing or multiple main/header/footer landmarks
// - navigation_issues: Inconsistent navigation structure

Pattern 11: Sensory Characteristics Detection (NEW)

// Detect instructions relying on sensory characteristics
{
  "tool": "web_sensory_characteristics_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - color_only: "Click the red button" (violation)
// - shape_only: "Press the round icon" (violation)
// - sound_only: "Listen for the beep" (violation)
// - location_visual: "See above" (warning)
// - size_only: "Click the large button" (warning)

Pattern 12: Animation/Flash Detection (NEW)

// Detect animations and flashing content
{
  "tool": "web_animation_flash_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - flashing_content: Content flashing > 3 times per second (violation)
// - no_pause_control: Autoplay animation > 5s without controls (violation)
// - rapid_animation: Fast infinite animations (warning)
// - animation_types: CSS, GIF, video, canvas, SVG

Pattern 13: Enhanced ARIA Validation (NEW)

// Perform enhanced accessibility tree analysis
{
  "tool": "web_enhanced_accessibility_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - missing_accessible_name: Interactive elements without labels
// - aria_hidden_interactive: Interactive elements with aria-hidden
// - invalid_tabindex: Elements with invalid tabindex values
// - landmark_issues: Multiple landmarks without distinguishing labels

Interpreting Results

Axe-Core Results

{
  "violations": [
    {
      "id": "color-contrast",
      "impact": "serious",  // critical, serious, moderate, minor
      "description": "Elements must have sufficient color contrast",
      "nodes": [
        {
          "html": "<p class=\"text-gray\">Low contrast text</p>",
          "target": [".text-gray"],
          "failureSummary": "Fix any of the following:\n  Element has insufficient color contrast of 3.2:1 (foreground color: #808080, background color: #ffffff, font size: 12.0pt (16px), font weight: normal). Expected contrast ratio of 4.5:1"
        }
      ]
    }
  ],
  "passes": [...],      // Tests that passed
  "incomplete": [...],  // Tests requiring manual review
  "inapplicable": [...]  // Tests not applicable to this page
}

Action Priority:

1. Critical/Serious violations - Fix immediately
2. Moderate violations - Fix in current sprint
3. Minor violations - Fix when convenient
4. Incomplete - Manually review and test
5. Passes - Document for compliance

Contrast Check Results

WCAG AA Violations:
  - p:nth-of-type(3): 3.2:1 (required: 4.5:1)
    Text: This text has insufficient contrast
    Colors: rgb(128, 128, 128) on rgb(255, 255, 255)

Remediation:

• Darken foreground color or lighten background
• Use contrast ratio calculator to find compliant colors
• Test with web_contrast_check_cremotemcp after fixes

Keyboard Test Results

High Severity Issues:
  - not_focusable: Interactive element is not keyboard focusable
    Element: div[role="button"]
  - no_focus_indicator: Interactive element lacks visible focus indicator
    Element: button.submit-btn

Remediation:

• not_focusable: Add tabindex="0" or use semantic HTML (<button> instead of <div>)
• no_focus_indicator: Add CSS :focus styles with visible outline/border
• keyboard_trap: Review JavaScript event handlers and focus management

Zoom Test Results

Zoom 400% ✗ FAIL:
  Horizontal Scroll: true
  Overflowing Elements: 3
  Text Readable: true

Remediation:

• Use responsive units (rem, em, %) instead of fixed pixels
• Implement CSS media queries for zoom levels
• Test with max-width: 100% on images and containers

Reflow Test Results

320px Width ✗ FAIL:
  Horizontal Scroll: true
  Responsive Layout: false
  Overflowing Elements: 5

Remediation:

• Implement mobile-first responsive design
• Use CSS Grid or Flexbox with flex-wrap
• Test with overflow-x: hidden cautiously (may hide content)

Common Workflows

Workflow 1: Comprehensive New Page Audit (UPDATED)

1. Navigate to page
2. Run web_inject_axe_cremotemcp
3. Run web_run_axe_cremotemcp with wcag2aa tags
4. Run specialized tests based on page content:
   a. web_contrast_check_cremotemcp for contrast issues
   b. web_gradient_contrast_check_cremotemcp for gradient backgrounds (NEW)
   c. web_media_validation_cremotemcp if page has video/audio (NEW)
   d. web_hover_focus_test_cremotemcp for tooltips/popovers (NEW)
   e. web_text_in_images_cremotemcp for infographics/charts (NEW)
   f. web_sensory_characteristics_cremotemcp for instructional content (NEW)
   g. web_animation_flash_cremotemcp for animated content (NEW)
   h. web_enhanced_accessibility_cremotemcp for ARIA validation (NEW)
   i. web_keyboard_test_cremotemcp for keyboard issues
5. Run web_zoom_test_cremotemcp
6. Run web_reflow_test_cremotemcp
7. Capture screenshots for documentation
8. Generate comprehensive report with all findings

Workflow 2: Regression Testing

1. Navigate to page
2. Run web_run_axe_cremotemcp
3. Compare results with baseline
4. If new violations:
   a. Run specialized tests for affected areas
   b. Capture screenshots showing issues
5. Report regressions

Workflow 3: Contrast-Focused Audit

1. Navigate to page
2. Run web_contrast_check_cremotemcp on entire page
3. For each violation:
   a. Capture screenshot of affected element
   b. Document current and required contrast ratios
   c. Suggest color adjustments
4. After fixes, re-run web_contrast_check_cremotemcp
5. Verify all violations resolved

Workflow 4: Keyboard Accessibility Audit

1. Navigate to page
2. Run web_keyboard_test_cremotemcp
3. Analyze tab order for logical sequence
4. For each issue:
   a. Document element and issue type
   b. Suggest remediation (tabindex, semantic HTML, focus styles)
5. After fixes, re-run web_keyboard_test_cremotemcp
6. Manually verify complex interactions

Workflow 5: Media Accessibility Audit (NEW)

1. Navigate to page with video/audio content
2. Run web_media_validation_cremotemcp
3. For each media element:
   a. Check for caption tracks (WCAG 1.2.2 Level A)
   b. Check for audio description tracks (WCAG 1.2.5 Level AA)
   c. Verify track files are accessible
   d. Check for autoplay violations (WCAG 1.4.2 Level A)
4. Document missing captions/descriptions
5. After fixes, re-run web_media_validation_cremotemcp
6. Manually verify caption accuracy (not automated)

Workflow 6: Text-in-Images Audit (NEW)

1. Navigate to page with images
2. Run web_text_in_images_cremotemcp (allow 30s timeout for OCR)
3. For each image with detected text:
   a. Review detected text vs alt text
   b. If alt text missing: Add comprehensive alt text
   c. If alt text insufficient: Expand to include all text
   d. Consider using real text instead of images
4. Capture screenshots of problematic images
5. After fixes, re-run web_text_in_images_cremotemcp
6. Verify all images with text have adequate alt text

Workflow 7: Multi-Page Consistency Audit (NEW)

1. Identify key pages to test (home, about, contact, services, etc.)
2. Run web_cross_page_consistency_cremotemcp with all URLs
3. Analyze common navigation elements
4. For each inconsistent page:
   a. Document missing navigation links
   b. Check landmark structure (header, footer, main, nav)
   c. Verify navigation order consistency
5. After fixes, re-run web_cross_page_consistency_cremotemcp
6. Verify all pages have consistent navigation

Workflow 8: Animation Safety Audit (NEW)

1. Navigate to page with animations
2. Run web_animation_flash_cremotemcp
3. For each animation:
   a. Check flash rate (must be ≤ 3 flashes/second)
   b. Check for pause/stop controls (if > 5 seconds)
   c. Verify autoplay behavior
4. For violations:
   a. Reduce flash rate or remove flashing
   b. Add pause/stop controls
   c. Disable autoplay or add controls
5. After fixes, re-run web_animation_flash_cremotemcp
6. Verify no flashing content exceeds 3 flashes/second

Workflow 9: ARIA Validation Audit (NEW)

1. Navigate to page with interactive elements
2. Run web_enhanced_accessibility_cremotemcp
3. For each element with issues:
   a. Missing accessible name: Add aria-label or visible text
   b. aria-hidden on interactive: Remove aria-hidden
   c. Invalid tabindex: Use 0 or -1
   d. Multiple landmarks: Add distinguishing labels
4. Capture screenshots of problematic elements
5. After fixes, re-run web_enhanced_accessibility_cremotemcp
6. Verify all interactive elements have accessible names

Error Handling

Common Errors and Solutions

Error: "Failed to inject library"

Solution: Check network connectivity, try alternative CDN, or increase timeout

Error: "Evaluation timed out"

Solution: Increase timeout parameter, test on smaller page sections

Error: "No tab available"

Solution: Navigate to a page first using web_navigate_cremotemcp

Error: "Failed to get page"

Solution: Verify tab ID is valid, check if page is still loaded

Best Practices for LLM Agents

1. Always Start with Axe-Core

Axe-core provides the broadest coverage (~57% of WCAG 2.1 AA). Use it as the foundation, then run specialized tests based on findings.

2. Test in Logical Order

Automated (axe) → Specialized (contrast, keyboard) → Visual (zoom, reflow) → Documentation (screenshots)

3. Provide Context in Reports

When reporting issues, include:

• WCAG criterion violated
• Severity/impact level
• Specific element(s) affected
• Current state (e.g., contrast ratio 3.2:1)
• Required state (e.g., contrast ratio 4.5:1)
• Suggested remediation

4. Capture Evidence

Always capture screenshots when documenting issues:

{
  "tool": "web_screenshot_cremotemcp",
  "arguments": {
    "output": "/tmp/issue-contrast-button.png"
  }
}

5. Verify Fixes

After suggesting fixes, re-run the relevant tests to verify resolution:

1. Suggest fix
2. Wait for implementation
3. Re-run specific test
4. Confirm issue resolved

6. Know the Limitations

These tools cannot test:

• Semantic meaning of content
• Cognitive load
• Caption accuracy (speech-to-text validation) - Only presence is checked
• Complex user interactions
• Context-dependent issues (some sensory characteristics need human judgment)
• Video frame-by-frame flash analysis (simplified estimation used)
• Complex ARIA widget validation (basic validation only)

Always recommend manual testing with assistive technologies for comprehensive audits.

NEW Tool Limitations:

• Gradient Contrast: Complex gradients (radial, conic) may not be fully analyzed
• Media Validation: Cannot verify caption accuracy, only presence
• Hover/Focus: May miss custom implementations using non-standard patterns
• Text-in-Images: OCR struggles with stylized fonts, handwriting, low contrast
• Cross-Page: Requires 2+ pages, may flag intentional variations
• Sensory Characteristics: Context-dependent, may have false positives
• Animation/Flash: Simplified flash rate estimation, no actual frame analysis
• Enhanced A11y: Simplified reference validation, doesn't check all ARIA states

Integration with Development Workflow

Pre-Commit Testing

# Run quick accessibility check before commit
cremote inject-axe && cremote run-axe --run-only wcag2aa

CI/CD Integration

# Run full accessibility suite in CI
- cremote inject-axe
- cremote run-axe > axe-results.json
- cremote contrast-check > contrast-results.txt
- cremote keyboard-test > keyboard-results.txt
- cremote zoom-test > zoom-results.txt
- cremote reflow-test > reflow-results.txt

Pull Request Reviews

1. Run accessibility tests on PR branch
2. Compare with main branch baseline
3. Report new violations in PR comments
4. Block merge if critical violations found

Quick Command Reference

# Inject axe-core
cremote inject-axe

# Run comprehensive tests
cremote run-axe --run-only wcag2a,wcag2aa,wcag21aa

# Check contrast
cremote contrast-check --selector body

# Check gradient contrast (NEW)
cremote gradient-contrast-check --selector .hero-section

# Validate media captions/descriptions (NEW)
cremote media-validation

# Test hover/focus content (NEW)
cremote hover-focus-test

# Detect text in images with OCR (NEW)
cremote text-in-images

# Check cross-page consistency (NEW)
cremote cross-page-consistency --urls "https://example.com/,https://example.com/about"

# Detect sensory characteristics (NEW)
cremote sensory-characteristics

# Detect animations and flashing (NEW)
cremote animation-flash

# Enhanced ARIA validation (NEW)
cremote enhanced-accessibility

# Test keyboard navigation
cremote keyboard-test

# Test zoom levels
cremote zoom-test --zoom-levels 1.0,2.0,4.0

# Test responsive design
cremote reflow-test --widths 320,768,1280

# Capture screenshot
cremote screenshot --output /tmp/screenshot.png

# Capture screenshot at zoom level
cremote screenshot --output /tmp/zoom-200.png --zoom-level 2.0

# Capture screenshot at specific viewport
cremote screenshot --output /tmp/mobile.png --width 320 --height 568

# Get accessibility tree
cremote get-accessibility-tree --include-contrast true

# Execute custom JavaScript with library injection
cremote console-command --command "axe.run()" --inject-library axe

Resources

• WCAG 2.1 Quick Reference: https://www.w3.org/WAI/WCAG21/quickref/
• Axe-Core Rules: https://github.com/dequelabs/axe-core/blob/develop/doc/rule-descriptions.md
• Contrast Ratio Calculator: https://contrast-ratio.com/
• WebAIM Resources: https://webaim.org/resources/

---

Coverage Summary

Automated WCAG 2.1 Level AA Coverage: ~93%

The cremote platform now provides comprehensive automated testing across:

• Phase 1 Tools: Gradient contrast, media validation, hover/focus testing
• Phase 2 Tools: Text-in-images, cross-page consistency, sensory characteristics
• Phase 3 Tools: Animation/flash detection, enhanced ARIA validation
• Core Tools: Axe-core, contrast checking, keyboard testing, zoom/reflow testing

What's Automated (93%):

• HTML structure and semantics
• Color contrast (simple and gradient backgrounds)
• Form labels and validation
• Heading structure
• Link text and purpose
• Image alt text (presence and adequacy)
• Keyboard accessibility
• Focus indicators
• ARIA attributes and roles
• Landmark structure
• Video/audio captions (presence)
• Text-in-images detection
• Cross-page consistency
• Sensory characteristics
• Animation/flash safety
• Zoom and responsive design

What Requires Manual Testing (7%):

• Caption accuracy (speech-to-text)
• Complex cognitive assessments
• Subjective content quality
• Advanced ARIA widget validation
• Video content frame analysis
• Context-dependent sensory instructions

---

For LLM Agents: This guide is designed for programmatic use. Always provide specific, actionable recommendations based on test results. Include WCAG criterion numbers, severity levels, and concrete remediation steps in your reports.

NEW TOOLS (2025-10-02): Eight new automated testing tools have been added, increasing coverage from 70% to 93%. These tools provide enhanced detection for gradient contrast, media accessibility, hover/focus content, text-in-images, cross-page consistency, sensory characteristics, animations/flashing, and ARIA validation. See the updated Tool Selection Matrix and usage patterns above for details.