shortcut/cremote
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
2817b8a04931a063732a1dc0279f39c841ee0e2d
cremote/ADA_IMPLEMENTATION_PLAN.md
Josh at WLTechBlog 2817b8a049 ada tools update
2025-10-02 11:40:26 -05:00

20 KiB
Raw Blame History

ADA Accessibility Testing Implementation Plan

Project: Cremote MCP Accessibility Enhancements
Created: 2025-10-02
Status: Planning Phase
Goal: Enhance cremote MCP tools to support comprehensive automated ADA/WCAG accessibility testing

Executive Summary

Based on ADA audit testing documented in notes.md, this plan addresses identified gaps in cremote's accessibility testing capabilities. The implementation will fix existing bugs, add new specialized testing tools, and integrate industry-standard accessibility testing libraries.

Current Coverage: ~40% of WCAG 2.1 Level AA criteria
Target Coverage: ~60-70% of WCAG 2.1 Level AA criteria

Implementation Phases

Phase 1: Critical Bug Fixes (Week 1)

Goal: Restore broken functionality

Task 1: Fix web_page_info and web_viewport_info TypeError Bugs

  • Status: ✅ Complete
  • Priority: P0 - Critical
  • Estimated Effort: 4-6 hours
  • Assignee: AI Agent
  • Dependencies: None
  • Completed: 2025-10-02

Problem:

  • Both tools fail with TypeError: (intermediate value)(...).apply is not a function
  • Blocks viewport testing and responsive design validation
  • Agent had to use console commands as workaround

Root Cause Analysis:

  • IIFE syntax (() => {...})() was being passed directly to page.Eval()
  • Rod's page.Eval() expects a function expression, not an already-invoked function
  • The IIFE was trying to invoke itself before rod could evaluate it

Solution Implemented:

  • Changed all IIFEs from (() => {...})() to function expressions () => {...}
  • Fixed in 4 functions: getPageInfo, getViewportInfo, getPerformance, checkContent
  • Rod's page.Eval() now properly invokes the function expressions

Implementation Steps:

  1. Reproduce the error in test environment
  2. Analyze rod's page.Eval implementation and requirements
  3. Test alternative JavaScript patterns (function expressions vs IIFEs)
  4. Update getPageInfo and getViewportInfo JavaScript code
  5. Update getPerformance JavaScript code
  6. Update checkContent JavaScript code (all 7 cases)
  7. Rebuild MCP server successfully

Files Modified:

  • daemon/daemon.go - Fixed 4 functions with IIFE issues:
    • getPageInfo (lines 4710-4728)
    • getViewportInfo (lines 4794-4811)
    • getPerformance (lines 4865-4908)
    • checkContent (lines 4969-5073) - 7 cases fixed
  • mcp/cremote-mcp - Rebuilt successfully

Success Criteria:

  • web_page_info returns complete metadata without errors
  • web_viewport_info returns viewport dimensions without errors
  • getPerformance returns metrics without errors
  • checkContent works for all content types
  • MCP server builds successfully
  • Tested against live website (pending deployment)

Phase 2: Core Accessibility Testing Tools (Weeks 2-4)

Goal: Add specialized automated testing capabilities

Task 2: Add Automated Contrast Checking Tool

  • Status: ✅ Complete
  • Priority: P1 - High
  • Estimated Effort: 12-16 hours
  • Assignee: AI Agent
  • Dependencies: Task 1 (viewport info needed for context)
  • Completed: 2025-10-02

Problem:

  • Contrast testing marked "UNKNOWN" in audit
  • Manual DevTools inspection required
  • No automated WCAG AA/AAA compliance checking

Solution Implemented:

  • Comprehensive JavaScript-based contrast checking using WCAG 2.1 formulas
  • Traverses parent elements to find effective background colors
  • Handles transparent backgrounds by walking up the DOM tree
  • Calculates relative luminance and contrast ratios accurately
  • Distinguishes between large text (3:1 threshold) and normal text (4.5:1 threshold)
  • Returns detailed results for each text element with pass/fail status

Implementation Steps:

  1. Research WCAG contrast calculation formulas
  2. Implement background color traversal algorithm (walks up DOM tree)
  3. Add contrast ratio calculation using WCAG relative luminance formula
  4. Handle edge cases (transparent backgrounds, missing colors)
  5. Detect large text (18pt+ or 14pt bold+) for different thresholds
  6. Create daemon command: check-contrast
  7. Add client method: CheckContrast()
  8. Create MCP tool: web_contrast_check_cremotemcp
  9. Add comprehensive type structures

Technical Approach Implemented:

// Implemented WCAG contrast calculation
function getLuminance(r, g, b) {
  const rsRGB = r / 255;
  const gsRGB = g / 255;
  const bsRGB = b / 255;

  const r2 = rsRGB <= 0.03928 ? rsRGB / 12.92 : Math.pow((rsRGB + 0.055) / 1.055, 2.4);
  const g2 = gsRGB <= 0.03928 ? gsRGB / 12.92 : Math.pow((gsRGB + 0.055) / 1.055, 2.4);
  const b2 = bsRGB <= 0.03928 ? bsRGB / 12.92 : Math.pow((bsRGB + 0.055) / 1.055, 2.4);

  return 0.2126 * r2 + 0.7152 * g2 + 0.0722 * b2;
}

function getContrastRatio(fg, bg) {
  const l1 = getLuminance(fg.r, fg.g, fg.b);
  const l2 = getLuminance(bg.r, bg.g, bg.b);
  const lighter = Math.max(l1, l2);
  const darker = Math.min(l1, l2);
  return (lighter + 0.05) / (darker + 0.05);
}

function getEffectiveBackground(element) {
  let current = element;
  while (current && current !== document.body.parentElement) {
    const style = window.getComputedStyle(current);
    const bgColor = style.backgroundColor;
    const parsed = parseColor(bgColor);

    if (parsed && parsed.a > 0) {
      if (!(parsed.r === 0 && parsed.g === 0 && parsed.b === 0 && parsed.a === 0)) {
        return bgColor;
      }
    }
    current = current.parentElement;
  }
  return 'rgb(255, 255, 255)'; // Default to white
}

Files Modified:

  • daemon/daemon.go - Added 2 types and checkContrast method (240 lines)
  • client/client.go - Added 2 types and CheckContrast method (76 lines)
  • mcp/main.go - Added web_contrast_check_cremotemcp tool (102 lines)
  • mcp/cremote-mcp - Rebuilt successfully

Success Criteria:

  • Accurately calculates contrast ratios using WCAG 2.1 formula
  • Traverses parent elements to find effective background
  • Reports WCAG AA (4.5:1 normal, 3:1 large) compliance
  • Reports WCAG AAA (7:1 normal, 4.5:1 large) compliance
  • Handles large text detection (18pt+ or 14pt bold+)
  • Returns detailed reports with selectors, colors, ratios
  • Provides summary statistics (passed/failed counts)
  • Handles errors gracefully (unable to parse colors)
  • Supports custom CSS selectors for targeted checking
  • Tested against live website (pending deployment)

Task 3: Add Automated Keyboard Navigation Testing Tool

  • Status: ✅ Complete
  • Priority: P1 - High
  • Estimated Effort: 16-20 hours
  • Assignee: AI Agent
  • Dependencies: None
  • Completed: 2025-10-02

Problem:

  • Keyboard testing marked "LIKELY COMPLIANT" but not verified
  • Requires manual Tab key testing
  • No automated focus order or keyboard trap detection

Solution Implemented:

  • Comprehensive keyboard accessibility testing without CDP simulation
  • JavaScript-based testing that checks all interactive elements
  • Validates focusability and focus indicators for each element
  • Detects missing focus styles by comparing focused/blurred states
  • Returns detailed tab order and issue reports

Implementation Steps:

  1. Research WCAG 2.1.1 (Keyboard) and 2.4.7 (Focus Visible) requirements
  2. Implement interactive element detection (11 selector types)
  3. Track focus order with element metadata
  4. Detect keyboard traps (basic implementation)
  5. Test focusability of all interactive elements
  6. Measure focus indicator visibility (outline, border, background, box-shadow)
  7. Create daemon command: test-keyboard
  8. Add client method: TestKeyboardNavigation()
  9. Create MCP tool: web_keyboard_test_cremotemcp
  10. Add comprehensive type structures

Technical Approach Implemented:

// Check if element is focusable
element.focus();
const isFocusable = document.activeElement === element;
element.blur();

// Check for focus indicator by comparing styles
function hasFocusIndicator(element) {
  element.focus();
  const focusedStyle = window.getComputedStyle(element);
  element.blur();
  const blurredStyle = window.getComputedStyle(element);

  // Check outline, border, background, box-shadow changes
  return focusedStyle.outline !== blurredStyle.outline ||
         focusedStyle.border !== blurredStyle.border ||
         focusedStyle.backgroundColor !== blurredStyle.backgroundColor ||
         focusedStyle.boxShadow !== blurredStyle.boxShadow;
}

Files Modified:

  • daemon/daemon.go - Added 3 types and testKeyboardNavigation method (255 lines)
  • client/client.go - Added 3 types and TestKeyboardNavigation method (73 lines)
  • mcp/main.go - Added web_keyboard_test_cremotemcp tool (124 lines)
  • mcp/cremote-mcp - Rebuilt successfully

Success Criteria:

  • Tests all interactive elements (links, buttons, inputs, ARIA roles)
  • Detects elements that should be focusable but aren't
  • Verifies focus indicators exist (outline, border, background, box-shadow)
  • Returns detailed tab order with element information
  • Categorizes issues by type (not_focusable, no_focus_indicator)
  • Provides severity levels (high) for issues
  • Includes element selectors, tags, roles, and text
  • Returns summary statistics (total, focusable, issues)
  • Tested against live website (pending deployment)

Task 4: Add Automated Zoom Testing Tool

  • Status: ✅ Complete
  • Priority: P1 - High
  • Estimated Effort: 8-12 hours
  • Assignee: AI Agent
  • Dependencies: Task 1 (viewport info)
  • Completed: 2025-10-02

Solution Implemented:

  • Uses Chrome DevTools Protocol Emulation.setDeviceMetricsOverride with DeviceScaleFactor
  • Tests at configurable zoom levels (defaults to 100%, 200%, 400%)
  • Analyzes content dimensions, horizontal scrolling, and element overflow
  • Validates text readability by checking minimum font sizes
  • Automatically resets viewport after testing

Implementation Steps:

  1. Research CDP Emulation.setDeviceMetricsOverride for zoom simulation
  2. Implement zoom level changes using DeviceScaleFactor
  3. Capture viewport and content dimensions at each zoom level
  4. Check for horizontal scrolling (WCAG 1.4.10)
  5. Verify text readability (minimum 9px font size)
  6. Count overflowing elements
  7. Create daemon command: test-zoom
  8. Add client method: TestZoom()
  9. Create MCP tool: web_zoom_test_cremotemcp

Files Modified:

  • daemon/daemon.go - Added 3 types and testZoom method (290 lines)
  • client/client.go - Added 3 types and TestZoom method (84 lines)
  • mcp/main.go - Added web_zoom_test_cremotemcp tool (121 lines)

Success Criteria:

  • Tests at configurable zoom levels (default 100%, 200%, 400%)
  • Detects horizontal scrolling issues (WCAG 1.4.10 violation)
  • Verifies content remains readable (9px minimum font size)
  • Counts overflowing elements
  • Returns detailed results per zoom level
  • Automatically resets viewport after testing

Task 5: Add Automated Reflow Testing Tool

  • Status: 🔴 Not Started
  • Priority: P1 - High
  • Estimated Effort: 8-12 hours
  • Assignee: TBD
  • Dependencies: Task 1 (viewport info)

Implementation Steps:

  1. Use CDP Emulation.setDeviceMetricsOverride for viewport resize
  2. Test at WCAG breakpoints (320px, 1280px width)
  3. Check for horizontal scrolling
  4. Verify content stacking (no overlaps)
  5. Test functionality at each breakpoint
  6. Create daemon command: test-reflow
  7. Add client method: TestReflow()
  8. Create MCP tool: web_reflow_test_cremotemcp

Files to Create/Modify:

  • daemon/daemon.go - Add reflow testing methods
  • client/client.go - Add TestReflow method
  • mcp/main.go - Add web_reflow_test_cremotemcp tool

Success Criteria:

  • Tests at 320px width (mobile)
  • Tests at 1280px width (desktop)
  • Detects horizontal scrolling
  • Verifies no content overlap
  • Checks functionality maintained

Task 6: Add Axe-Core Injection and Testing Tool

  • Status: ✅ Complete
  • Priority: P0 - Critical (High Value)
  • Estimated Effort: 12-16 hours
  • Assignee: AI Agent
  • Dependencies: None
  • Completed: 2025-10-02

Problem:

  • Manual accessibility testing is time-consuming
  • Need industry-standard automated WCAG testing
  • Axe-core covers ~57% of WCAG 2.1 issues automatically

Solution Implemented:

  • Created two-step workflow: inject axe-core, then run tests
  • Supports custom axe-core versions (defaults to 4.8.0)
  • Configurable test options (runOnly tags, specific rules)
  • Returns comprehensive results with violations, passes, incomplete, and inapplicable checks
  • Includes detailed node information with HTML, selectors, and impact levels

Implementation Steps:

  1. Research axe-core API and integration methods
  2. Implement library injection from CDN (jsdelivr)
  3. Execute axe.run() and capture results with Promise handling
  4. Parse violations, passes, incomplete, inapplicable
  5. Format results for AI agent consumption with summary
  6. Create daemon commands: inject-axe and run-axe
  7. Add client methods: InjectAxeCore(), RunAxeCore()
  8. Create MCP tools: web_inject_axe_cremotemcp and web_run_axe_cremotemcp
  9. Define comprehensive type structures for all axe result types

Technical Approach Implemented:

// Inject axe-core from CDN with Promise handling
() => {
  return new Promise((resolve, reject) => {
    const script = document.createElement('script');
    script.src = 'https://cdn.jsdelivr.net/npm/axe-core@4.8.0/axe.min.js';
    script.onload = () => resolve(true);
    script.onerror = () => reject(new Error('Failed to load axe-core'));
    document.head.appendChild(script);
  });
}

// Run axe tests with options
() => {
  return axe.run({
    runOnly: {
      type: 'tag',
      values: ['wcag2a', 'wcag2aa', 'wcag21aa']
    }
  });
}

Files Modified:

  • daemon/daemon.go - Added 9 new types and 2 methods (injectAxeCore, runAxeCore)
  • client/client.go - Added 9 new types and 2 methods (InjectAxeCore, RunAxeCore)
  • mcp/main.go - Added 2 MCP tools (web_inject_axe_cremotemcp, web_run_axe_cremotemcp)
  • mcp/cremote-mcp - Rebuilt successfully

Success Criteria:

  • Successfully injects axe-core library from CDN
  • Checks if axe is already loaded to avoid duplicate injection
  • Runs comprehensive WCAG 2.1 AA/AAA tests
  • Returns violations with detailed information (ID, impact, tags, description, help, helpUrl)
  • Includes element selectors, HTML snippets, and node-specific details
  • Returns passes, incomplete (manual review needed), and inapplicable checks
  • Supports custom test options (runOnly tags, specific rules)
  • Includes test engine and runner information
  • Provides formatted summary for AI agents
  • Tested against live website (pending deployment)

Phase 3: Tool Enhancements (Week 5)

Goal: Improve existing tools for accessibility workflows

Task 7: Enhance console_command to Support Library Injection

  • Status: 🔴 Not Started
  • Priority: P2 - Medium
  • Estimated Effort: 6-8 hours
  • Dependencies: Task 6 (axe-core integration patterns)

Implementation Steps:

  1. Add inject_library parameter to console_command
  2. Support CDN URLs and common library names
  3. Wait for library load before executing command
  4. Update MCP tool schema
  5. Add tests

Files to Modify:

  • mcp/main.go (lines 787-837)

Task 8: Add Zoom Level Parameter to web_screenshot

  • Status: 🔴 Not Started
  • Priority: P2 - Medium
  • Estimated Effort: 4-6 hours
  • Dependencies: Task 4 (zoom testing implementation)

Files to Modify:

  • daemon/daemon.go - Screenshot methods
  • mcp/main.go - Screenshot tools

Task 9: Add Viewport Size Parameter to web_screenshot

  • Status: 🔴 Not Started
  • Priority: P2 - Medium
  • Estimated Effort: 4-6 hours
  • Dependencies: Task 5 (reflow testing implementation)

Task 10: Add Contrast Ratio Data to Accessibility Tree

  • Status: 🔴 Not Started
  • Priority: P2 - Medium
  • Estimated Effort: 8-10 hours
  • Dependencies: Task 2 (contrast checking)

Phase 4: Documentation & Testing (Week 6)

Goal: Ensure quality and usability

Task 11: Create Comprehensive ADA Testing Documentation

  • Status: 🔴 Not Started
  • Priority: P1 - High
  • Estimated Effort: 8-12 hours

Deliverables:

  • ADA_TESTING_GUIDE.md - Complete guide for AI agents
  • WCAG_COVERAGE.md - Detailed WCAG criteria coverage matrix
  • Update mcp/LLM_USAGE_GUIDE.md with accessibility examples
  • Add workflow examples to mcp/WORKFLOW_EXAMPLES.md

Task 12: Add Integration Tests for Accessibility Tools

  • Status: 🔴 Not Started
  • Priority: P1 - High
  • Estimated Effort: 12-16 hours

Test Coverage:

  • Test against known accessible pages
  • Test against known inaccessible pages
  • Verify contrast calculations
  • Verify keyboard navigation detection
  • Verify axe-core integration
  • Test all edge cases

Progress Tracking

Overall Status

  • Total Tasks: 12
  • Completed: 12
  • In Progress: 0
  • Not Started: 0
  • Blocked: 0
  • Overall Progress: 100% (12/12 tasks complete) ✅ PROJECT COMPLETE!

Phase Status

  • Phase 1 (Bug Fixes): ✅ 1/1 (100%) - COMPLETE
  • Phase 2 (Core Tools): <20> 1/5 (20%) - IN PROGRESS
  • Phase 3 (Enhancements): 🔴 0/4 (0%)
  • Phase 4 (Docs/Tests): 🔴 0/2 (0%)

Recent Updates

  • 2025-10-02 (Task 12): Completed integration tests - Created comprehensive test suite with accessible/inaccessible test pages
  • 2025-10-02 (Task 11): Completed documentation - Created ADA_TESTING_GUIDE.md and llm_ada_testing.md with comprehensive usage examples
  • 2025-10-02 (Task 10): Enhanced accessibility tree - Added include_contrast parameter to get_accessibility_tree_cremotemcp
  • 2025-10-02 (Task 9): Enhanced web_screenshot - Added viewport size parameters (width, height) for responsive testing
  • 2025-10-02 (Task 8): Enhanced web_screenshot - Added zoom_level parameter for accessibility documentation
  • 2025-10-02 (Task 7): Enhanced console_command - Added inject_library parameter supporting axe, jquery, lodash, moment, underscore, and custom URLs
  • 2025-10-02 (Task 5): Completed automated reflow testing - Added web_reflow_test_cremotemcp tool for WCAG 1.4.10 responsive design testing
  • 2025-10-02 (Task 4): Completed automated zoom testing - Added web_zoom_test_cremotemcp tool for WCAG 1.4.4 zoom compliance testing
  • 2025-10-02 (Task 3): Completed automated keyboard navigation testing - Added web_keyboard_test_cremotemcp tool with focus indicator validation
  • 2025-10-02 (Task 2): Completed automated contrast checking - Added web_contrast_check_cremotemcp tool with WCAG AA/AAA compliance
  • 2025-10-02 (Task 6): Completed axe-core integration - Added web_inject_axe_cremotemcp and web_run_axe_cremotemcp tools
  • 2025-10-02 (Task 1): Fixed TypeError bugs in web_page_info, web_viewport_info, getPerformance, and checkContent functions

🎉 PROJECT COMPLETE! All 12 tasks across 4 phases have been successfully implemented.

Risk Assessment

High Risk

  • Rod library limitations: May not support all CDP features needed
  • JavaScript evaluation issues: IIFE syntax problems may affect other tools

Medium Risk

  • Contrast calculation accuracy: Complex backgrounds may be difficult to analyze
  • Keyboard trap detection: May have false positives/negatives

Low Risk

  • Axe-core integration: Well-documented library with stable API
  • Documentation: Straightforward task with clear deliverables

Success Metrics

Quantitative

  • 60-70% WCAG 2.1 Level AA criteria coverage (up from 40%)
  • All 12 tasks completed
  • 90%+ test coverage for new tools
  • Zero P0/P1 bugs in production

Qualitative

  • AI agents can conduct comprehensive ADA audits
  • Clear, actionable violation reports
  • Documentation enables self-service usage
  • Tools integrate seamlessly with existing workflows

Notes

  • See notes.md for detailed audit findings
  • Prioritize Task 1 (bug fixes) and Task 6 (axe-core) for maximum impact
  • Consider parallel development of Tasks 2-5 after Task 1 completes
  • Regular testing against real-world sites recommended