cremote/docs/llm_ada_testing.md

# LLM Agent Guide: ADA/WCAG Accessibility Testing with Cremote

## Purpose
This document provides LLM coding agents with concrete, actionable guidance for using Cremote's accessibility testing tools to conduct ADA/WCAG compliance audits.

## Quick Reference

### Tool Selection Matrix

| Testing Need | Primary Tool | Secondary Tool | WCAG Criteria |
|--------------|--------------|----------------|---------------|
| Comprehensive automated audit | `web_run_axe_cremotemcp` | - | ~57% of WCAG 2.1 AA |
| Color contrast issues | `web_contrast_check_cremotemcp` | `web_gradient_contrast_check_cremotemcp` | 1.4.3, 1.4.6, 1.4.11 |
| Gradient backgrounds | `web_gradient_contrast_check_cremotemcp` | - | 1.4.3, 1.4.6, 1.4.11 |
| Video/audio captions | `web_media_validation_cremotemcp` | - | 1.2.2, 1.2.5, 1.4.2 |
| Hover/focus content | `web_hover_focus_test_cremotemcp` | - | 1.4.13 |
| Text in images | `web_text_in_images_cremotemcp` | - | 1.4.5, 1.4.9, 1.1.1 |
| Cross-page consistency | `web_cross_page_consistency_cremotemcp` | - | 3.2.3, 3.2.4, 1.3.1 |
| Sensory instructions | `web_sensory_characteristics_cremotemcp` | - | 1.3.3 |
| Animations/flashing | `web_animation_flash_cremotemcp` | - | 2.3.1, 2.2.2, 2.3.2 |
| ARIA validation | `web_enhanced_accessibility_cremotemcp` | `web_run_axe_cremotemcp` | 1.3.1, 4.1.2, 2.4.6 |
| Keyboard accessibility | `web_keyboard_test_cremotemcp` | `web_run_axe_cremotemcp` | 2.1.1, 2.4.7 |
| Zoom/resize functionality | `web_zoom_test_cremotemcp` | - | 1.4.4 |
| Responsive design | `web_reflow_test_cremotemcp` | - | 1.4.10 |
| Visual documentation | `web_screenshot_cremotemcp` | - | Evidence capture |
| Custom JavaScript testing | `console_command_cremotemcp` | - | Advanced scenarios |

### Standard Testing Sequence

```
1. web_inject_axe_cremotemcp                    # Inject axe-core library
2. web_run_axe_cremotemcp                       # Run comprehensive automated tests
3. web_contrast_check_cremotemcp                # Detailed contrast analysis
4. web_gradient_contrast_check_cremotemcp       # Gradient background contrast (NEW)
5. web_media_validation_cremotemcp              # Video/audio caption validation (NEW)
6. web_hover_focus_test_cremotemcp              # Hover/focus content testing (NEW)
7. web_text_in_images_cremotemcp                # Text-in-images detection (NEW)
8. web_sensory_characteristics_cremotemcp       # Sensory instruction detection (NEW)
9. web_animation_flash_cremotemcp               # Animation/flash detection (NEW)
10. web_enhanced_accessibility_cremotemcp       # Enhanced ARIA validation (NEW)
11. web_keyboard_test_cremotemcp                # Keyboard navigation testing
12. web_zoom_test_cremotemcp                    # Zoom functionality testing
13. web_reflow_test_cremotemcp                  # Responsive design testing
```

**Note:** For multi-page sites, also run:
```
14. web_cross_page_consistency_cremotemcp       # Cross-page consistency (NEW)
```

## Tool Usage Patterns

### Pattern 1: Initial Audit

```json
// Step 1: Inject axe-core
{
  "tool": "web_inject_axe_cremotemcp",
  "arguments": {
    "timeout": 30
  }
}

// Step 2: Run comprehensive tests
{
  "tool": "web_run_axe_cremotemcp",
  "arguments": {
    "run_only": ["wcag2a", "wcag2aa", "wcag21aa"],
    "timeout": 30
  }
}

// Step 3: Analyze results and run specialized tests based on findings
```

### Pattern 2: Contrast-Specific Testing

```json
// For pages with known or suspected contrast issues
{
  "tool": "web_contrast_check_cremotemcp",
  "arguments": {
    "selector": "body",  // Test entire page
    "timeout": 10
  }
}

// For specific sections
{
  "tool": "web_contrast_check_cremotemcp",
  "arguments": {
    "selector": ".main-content",  // Test specific area
    "timeout": 10
  }
}
```

### Pattern 3: Keyboard Navigation Testing

```json
{
  "tool": "web_keyboard_test_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - not_focusable: Elements that should be keyboard accessible but aren't
// - no_focus_indicator: Elements missing visible focus indicators
// - keyboard_trap: Elements that trap keyboard focus
```

### Pattern 4: Zoom and Responsive Testing

```json
// Test zoom levels (WCAG 1.4.4)
{
  "tool": "web_zoom_test_cremotemcp",
  "arguments": {
    "zoom_levels": [1.0, 2.0, 4.0],
    "timeout": 10
  }
}

// Test responsive breakpoints (WCAG 1.4.10)
{
  "tool": "web_reflow_test_cremotemcp",
  "arguments": {
    "widths": [320, 768, 1280],
    "timeout": 10
  }
}
```

### Pattern 5: Visual Documentation

```json
// Capture baseline
{
  "tool": "web_screenshot_cremotemcp",
  "arguments": {
    "output": "/tmp/baseline.png",
    "timeout": 5
  }
}

// Capture at 200% zoom
{
  "tool": "web_screenshot_cremotemcp",
  "arguments": {
    "output": "/tmp/zoom-200.png",
    "zoom_level": 2.0,
    "timeout": 5
  }
}

// Capture mobile view
{
  "tool": "web_screenshot_cremotemcp",
  "arguments": {
    "output": "/tmp/mobile-320.png",
    "width": 320,
    "height": 568,
    "timeout": 5
  }
}
```

### Pattern 6: Gradient Contrast Testing (NEW)

```json
// Test specific element with gradient background
{
  "tool": "web_gradient_contrast_check_cremotemcp",
  "arguments": {
    "selector": ".hero-section",
    "timeout": 10
  }
}

// Test all elements with gradient backgrounds
{
  "tool": "web_gradient_contrast_check_cremotemcp",
  "arguments": {
    "selector": "body",  // Scans entire page
    "timeout": 10
  }
}

// Analyze output for:
// - worst_case_ratio: Minimum contrast found across gradient
// - best_case_ratio: Maximum contrast found across gradient
// - wcag_aa_pass: Whether it meets WCAG AA standards
// - wcag_aaa_pass: Whether it meets WCAG AAA standards
```

### Pattern 7: Media Validation (NEW)

```json
// Validate all video/audio elements on page
{
  "tool": "web_media_validation_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - missing_captions: Videos without caption tracks
// - missing_audio_descriptions: Videos without audio description tracks
// - inaccessible_tracks: Track files that cannot be loaded
// - autoplay_violations: Videos that autoplay without controls
```

### Pattern 8: Hover/Focus Content Testing (NEW)

```json
// Test all hover/focus triggered content
{
  "tool": "web_hover_focus_test_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - not_dismissible: Content that cannot be dismissed with Escape key
// - not_hoverable: Tooltip disappears when hovering over it
// - not_persistent: Content disappears too quickly
// - native_title_tooltip: Using native title attribute (violation)
```

### Pattern 9: Text-in-Images Detection (NEW)

```json
// Detect text embedded in images using OCR
{
  "tool": "web_text_in_images_cremotemcp",
  "arguments": {
    "timeout": 30  // OCR is CPU-intensive, allow more time
  }
}

// Analyze output for:
// - missing_alt: Images with text but no alt text
// - insufficient_alt: Images with text but inadequate alt text
// - detected_text: Actual text found in the image
// - recommendations: Specific suggestions for each image
```

### Pattern 10: Cross-Page Consistency (NEW)

```json
// Check consistency across multiple pages
{
  "tool": "web_cross_page_consistency_cremotemcp",
  "arguments": {
    "urls": [
      "https://example.com/",
      "https://example.com/about",
      "https://example.com/contact",
      "https://example.com/services"
    ],
    "timeout": 10  // Per page
  }
}

// Analyze output for:
// - common_navigation: Links present on all pages
// - inconsistent_pages: Pages missing common navigation
// - landmark_issues: Missing or multiple main/header/footer landmarks
// - navigation_issues: Inconsistent navigation structure
```

### Pattern 11: Sensory Characteristics Detection (NEW)

```json
// Detect instructions relying on sensory characteristics
{
  "tool": "web_sensory_characteristics_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - color_only: "Click the red button" (violation)
// - shape_only: "Press the round icon" (violation)
// - sound_only: "Listen for the beep" (violation)
// - location_visual: "See above" (warning)
// - size_only: "Click the large button" (warning)
```

### Pattern 12: Animation/Flash Detection (NEW)

```json
// Detect animations and flashing content
{
  "tool": "web_animation_flash_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - flashing_content: Content flashing > 3 times per second (violation)
// - no_pause_control: Autoplay animation > 5s without controls (violation)
// - rapid_animation: Fast infinite animations (warning)
// - animation_types: CSS, GIF, video, canvas, SVG
```

### Pattern 13: Enhanced ARIA Validation (NEW)

```json
// Perform enhanced accessibility tree analysis
{
  "tool": "web_enhanced_accessibility_cremotemcp",
  "arguments": {
    "timeout": 10
  }
}

// Analyze output for:
// - missing_accessible_name: Interactive elements without labels
// - aria_hidden_interactive: Interactive elements with aria-hidden
// - invalid_tabindex: Elements with invalid tabindex values
// - landmark_issues: Multiple landmarks without distinguishing labels
```

## Interpreting Results

### Axe-Core Results

```json
{
  "violations": [
    {
      "id": "color-contrast",
      "impact": "serious",  // critical, serious, moderate, minor
      "description": "Elements must have sufficient color contrast",
      "nodes": [
        {
          "html": "<p class=\"text-gray\">Low contrast text</p>",
          "target": [".text-gray"],
          "failureSummary": "Fix any of the following:\n  Element has insufficient color contrast of 3.2:1 (foreground color: #808080, background color: #ffffff, font size: 12.0pt (16px), font weight: normal). Expected contrast ratio of 4.5:1"
        }
      ]
    }
  ],
  "passes": [...],      // Tests that passed
  "incomplete": [...],  // Tests requiring manual review
  "inapplicable": [...]  // Tests not applicable to this page
}
```

**Action Priority:**
1. **Critical/Serious violations** - Fix immediately
2. **Moderate violations** - Fix in current sprint
3. **Minor violations** - Fix when convenient
4. **Incomplete** - Manually review and test
5. **Passes** - Document for compliance

### Contrast Check Results

```
WCAG AA Violations:
  - p:nth-of-type(3): 3.2:1 (required: 4.5:1)
    Text: This text has insufficient contrast
    Colors: rgb(128, 128, 128) on rgb(255, 255, 255)
```

**Remediation:**
- Darken foreground color or lighten background
- Use contrast ratio calculator to find compliant colors
- Test with `web_contrast_check_cremotemcp` after fixes

### Keyboard Test Results

```
High Severity Issues:
  - not_focusable: Interactive element is not keyboard focusable
    Element: div[role="button"]
  - no_focus_indicator: Interactive element lacks visible focus indicator
    Element: button.submit-btn
```

**Remediation:**
- `not_focusable`: Add `tabindex="0"` or use semantic HTML (`<button>` instead of `<div>`)
- `no_focus_indicator`: Add CSS `:focus` styles with visible outline/border
- `keyboard_trap`: Review JavaScript event handlers and focus management

### Zoom Test Results

```
Zoom 400% ✗ FAIL:
  Horizontal Scroll: true
  Overflowing Elements: 3
  Text Readable: true
```

**Remediation:**
- Use responsive units (rem, em, %) instead of fixed pixels
- Implement CSS media queries for zoom levels
- Test with `max-width: 100%` on images and containers

### Reflow Test Results

```
320px Width ✗ FAIL:
  Horizontal Scroll: true
  Responsive Layout: false
  Overflowing Elements: 5
```

**Remediation:**
- Implement mobile-first responsive design
- Use CSS Grid or Flexbox with `flex-wrap`
- Test with `overflow-x: hidden` cautiously (may hide content)

## Common Workflows

### Workflow 1: Comprehensive New Page Audit (UPDATED)

```
1. Navigate to page
2. Run web_inject_axe_cremotemcp
3. Run web_run_axe_cremotemcp with wcag2aa tags
4. Run specialized tests based on page content:
   a. web_contrast_check_cremotemcp for contrast issues
   b. web_gradient_contrast_check_cremotemcp for gradient backgrounds (NEW)
   c. web_media_validation_cremotemcp if page has video/audio (NEW)
   d. web_hover_focus_test_cremotemcp for tooltips/popovers (NEW)
   e. web_text_in_images_cremotemcp for infographics/charts (NEW)
   f. web_sensory_characteristics_cremotemcp for instructional content (NEW)
   g. web_animation_flash_cremotemcp for animated content (NEW)
   h. web_enhanced_accessibility_cremotemcp for ARIA validation (NEW)
   i. web_keyboard_test_cremotemcp for keyboard issues
5. Run web_zoom_test_cremotemcp
6. Run web_reflow_test_cremotemcp
7. Capture screenshots for documentation
8. Generate comprehensive report with all findings
```

### Workflow 2: Regression Testing

```
1. Navigate to page
2. Run web_run_axe_cremotemcp
3. Compare results with baseline
4. If new violations:
   a. Run specialized tests for affected areas
   b. Capture screenshots showing issues
5. Report regressions
```

### Workflow 3: Contrast-Focused Audit

```
1. Navigate to page
2. Run web_contrast_check_cremotemcp on entire page
3. For each violation:
   a. Capture screenshot of affected element
   b. Document current and required contrast ratios
   c. Suggest color adjustments
4. After fixes, re-run web_contrast_check_cremotemcp
5. Verify all violations resolved
```

### Workflow 4: Keyboard Accessibility Audit

```
1. Navigate to page
2. Run web_keyboard_test_cremotemcp
3. Analyze tab order for logical sequence
4. For each issue:
   a. Document element and issue type
   b. Suggest remediation (tabindex, semantic HTML, focus styles)
5. After fixes, re-run web_keyboard_test_cremotemcp
6. Manually verify complex interactions
```

### Workflow 5: Media Accessibility Audit (NEW)

```
1. Navigate to page with video/audio content
2. Run web_media_validation_cremotemcp
3. For each media element:
   a. Check for caption tracks (WCAG 1.2.2 Level A)
   b. Check for audio description tracks (WCAG 1.2.5 Level AA)
   c. Verify track files are accessible
   d. Check for autoplay violations (WCAG 1.4.2 Level A)
4. Document missing captions/descriptions
5. After fixes, re-run web_media_validation_cremotemcp
6. Manually verify caption accuracy (not automated)
```

### Workflow 6: Text-in-Images Audit (NEW)

```
1. Navigate to page with images
2. Run web_text_in_images_cremotemcp (allow 30s timeout for OCR)
3. For each image with detected text:
   a. Review detected text vs alt text
   b. If alt text missing: Add comprehensive alt text
   c. If alt text insufficient: Expand to include all text
   d. Consider using real text instead of images
4. Capture screenshots of problematic images
5. After fixes, re-run web_text_in_images_cremotemcp
6. Verify all images with text have adequate alt text
```

### Workflow 7: Multi-Page Consistency Audit (NEW)

```
1. Identify key pages to test (home, about, contact, services, etc.)
2. Run web_cross_page_consistency_cremotemcp with all URLs
3. Analyze common navigation elements
4. For each inconsistent page:
   a. Document missing navigation links
   b. Check landmark structure (header, footer, main, nav)
   c. Verify navigation order consistency
5. After fixes, re-run web_cross_page_consistency_cremotemcp
6. Verify all pages have consistent navigation
```

### Workflow 8: Animation Safety Audit (NEW)

```
1. Navigate to page with animations
2. Run web_animation_flash_cremotemcp
3. For each animation:
   a. Check flash rate (must be ≤ 3 flashes/second)
   b. Check for pause/stop controls (if > 5 seconds)
   c. Verify autoplay behavior
4. For violations:
   a. Reduce flash rate or remove flashing
   b. Add pause/stop controls
   c. Disable autoplay or add controls
5. After fixes, re-run web_animation_flash_cremotemcp
6. Verify no flashing content exceeds 3 flashes/second
```

### Workflow 9: ARIA Validation Audit (NEW)

```
1. Navigate to page with interactive elements
2. Run web_enhanced_accessibility_cremotemcp
3. For each element with issues:
   a. Missing accessible name: Add aria-label or visible text
   b. aria-hidden on interactive: Remove aria-hidden
   c. Invalid tabindex: Use 0 or -1
   d. Multiple landmarks: Add distinguishing labels
4. Capture screenshots of problematic elements
5. After fixes, re-run web_enhanced_accessibility_cremotemcp
6. Verify all interactive elements have accessible names
```

## Error Handling

### Common Errors and Solutions

**Error:** "Failed to inject library"
```
Solution: Check network connectivity, try alternative CDN, or increase timeout
```

**Error:** "Evaluation timed out"
```
Solution: Increase timeout parameter, test on smaller page sections
```

**Error:** "No tab available"
```
Solution: Navigate to a page first using web_navigate_cremotemcp
```

**Error:** "Failed to get page"
```
Solution: Verify tab ID is valid, check if page is still loaded
```

## Best Practices for LLM Agents

### 1. Always Start with Axe-Core
Axe-core provides the broadest coverage (~57% of WCAG 2.1 AA). Use it as the foundation, then run specialized tests based on findings.

### 2. Test in Logical Order
```
Automated (axe) → Specialized (contrast, keyboard) → Visual (zoom, reflow) → Documentation (screenshots)
```

### 3. Provide Context in Reports
When reporting issues, include:
- WCAG criterion violated
- Severity/impact level
- Specific element(s) affected
- Current state (e.g., contrast ratio 3.2:1)
- Required state (e.g., contrast ratio 4.5:1)
- Suggested remediation

### 4. Capture Evidence
Always capture screenshots when documenting issues:
```json
{
  "tool": "web_screenshot_cremotemcp",
  "arguments": {
    "output": "/tmp/issue-contrast-button.png"
  }
}
```

### 5. Verify Fixes
After suggesting fixes, re-run the relevant tests to verify resolution:
```
1. Suggest fix
2. Wait for implementation
3. Re-run specific test
4. Confirm issue resolved
```

### 6. Know the Limitations
These tools cannot test:
- Semantic meaning of content
- Cognitive load
- Caption accuracy (speech-to-text validation) - Only presence is checked
- Complex user interactions
- Context-dependent issues (some sensory characteristics need human judgment)
- Video frame-by-frame flash analysis (simplified estimation used)
- Complex ARIA widget validation (basic validation only)

Always recommend manual testing with assistive technologies for comprehensive audits.

**NEW Tool Limitations:**
- **Gradient Contrast:** Complex gradients (radial, conic) may not be fully analyzed
- **Media Validation:** Cannot verify caption accuracy, only presence
- **Hover/Focus:** May miss custom implementations using non-standard patterns
- **Text-in-Images:** OCR struggles with stylized fonts, handwriting, low contrast
- **Cross-Page:** Requires 2+ pages, may flag intentional variations
- **Sensory Characteristics:** Context-dependent, may have false positives
- **Animation/Flash:** Simplified flash rate estimation, no actual frame analysis
- **Enhanced A11y:** Simplified reference validation, doesn't check all ARIA states

## Integration with Development Workflow

### Pre-Commit Testing
```bash
# Run quick accessibility check before commit
cremote inject-axe && cremote run-axe --run-only wcag2aa
```

### CI/CD Integration
```yaml
# Run full accessibility suite in CI
- cremote inject-axe
- cremote run-axe > axe-results.json
- cremote contrast-check > contrast-results.txt
- cremote keyboard-test > keyboard-results.txt
- cremote zoom-test > zoom-results.txt
- cremote reflow-test > reflow-results.txt
```

### Pull Request Reviews
```
1. Run accessibility tests on PR branch
2. Compare with main branch baseline
3. Report new violations in PR comments
4. Block merge if critical violations found
```

## Quick Command Reference

```bash
# Inject axe-core
cremote inject-axe

# Run comprehensive tests
cremote run-axe --run-only wcag2a,wcag2aa,wcag21aa

# Check contrast
cremote contrast-check --selector body

# Check gradient contrast (NEW)
cremote gradient-contrast-check --selector .hero-section

# Validate media captions/descriptions (NEW)
cremote media-validation

# Test hover/focus content (NEW)
cremote hover-focus-test

# Detect text in images with OCR (NEW)
cremote text-in-images

# Check cross-page consistency (NEW)
cremote cross-page-consistency --urls "https://example.com/,https://example.com/about"

# Detect sensory characteristics (NEW)
cremote sensory-characteristics

# Detect animations and flashing (NEW)
cremote animation-flash

# Enhanced ARIA validation (NEW)
cremote enhanced-accessibility

# Test keyboard navigation
cremote keyboard-test

# Test zoom levels
cremote zoom-test --zoom-levels 1.0,2.0,4.0

# Test responsive design
cremote reflow-test --widths 320,768,1280

# Capture screenshot
cremote screenshot --output /tmp/screenshot.png

# Capture screenshot at zoom level
cremote screenshot --output /tmp/zoom-200.png --zoom-level 2.0

# Capture screenshot at specific viewport
cremote screenshot --output /tmp/mobile.png --width 320 --height 568

# Get accessibility tree
cremote get-accessibility-tree --include-contrast true

# Execute custom JavaScript with library injection
cremote console-command --command "axe.run()" --inject-library axe
```

## Resources

- **WCAG 2.1 Quick Reference:** https://www.w3.org/WAI/WCAG21/quickref/
- **Axe-Core Rules:** https://github.com/dequelabs/axe-core/blob/develop/doc/rule-descriptions.md
- **Contrast Ratio Calculator:** https://contrast-ratio.com/
- **WebAIM Resources:** https://webaim.org/resources/

---

## Coverage Summary

**Automated WCAG 2.1 Level AA Coverage: ~93%**

The cremote platform now provides comprehensive automated testing across:
- **Phase 1 Tools:** Gradient contrast, media validation, hover/focus testing
- **Phase 2 Tools:** Text-in-images, cross-page consistency, sensory characteristics
- **Phase 3 Tools:** Animation/flash detection, enhanced ARIA validation
- **Core Tools:** Axe-core, contrast checking, keyboard testing, zoom/reflow testing

**What's Automated (93%):**
- HTML structure and semantics
- Color contrast (simple and gradient backgrounds)
- Form labels and validation
- Heading structure
- Link text and purpose
- Image alt text (presence and adequacy)
- Keyboard accessibility
- Focus indicators
- ARIA attributes and roles
- Landmark structure
- Video/audio captions (presence)
- Text-in-images detection
- Cross-page consistency
- Sensory characteristics
- Animation/flash safety
- Zoom and responsive design

**What Requires Manual Testing (7%):**
- Caption accuracy (speech-to-text)
- Complex cognitive assessments
- Subjective content quality
- Advanced ARIA widget validation
- Video content frame analysis
- Context-dependent sensory instructions

---

**For LLM Agents:** This guide is designed for programmatic use. Always provide specific, actionable recommendations based on test results. Include WCAG criterion numbers, severity levels, and concrete remediation steps in your reports.

**NEW TOOLS (2025-10-02):** Eight new automated testing tools have been added, increasing coverage from 70% to 93%. These tools provide enhanced detection for gradient contrast, media accessibility, hover/focus content, text-in-images, cross-page consistency, sensory characteristics, animations/flashing, and ARIA validation. See the updated Tool Selection Matrix and usage patterns above for details.