Files
cremote/ADA_IMPLEMENTATION_PLAN.md
Josh at WLTechBlog 2817b8a049 ada tools update
2025-10-02 11:40:26 -05:00

542 lines
20 KiB
Markdown
Raw Blame History

# ADA Accessibility Testing Implementation Plan
**Project:** Cremote MCP Accessibility Enhancements
**Created:** 2025-10-02
**Status:** Planning Phase
**Goal:** Enhance cremote MCP tools to support comprehensive automated ADA/WCAG accessibility testing
## Executive Summary
Based on ADA audit testing documented in `notes.md`, this plan addresses identified gaps in cremote's accessibility testing capabilities. The implementation will fix existing bugs, add new specialized testing tools, and integrate industry-standard accessibility testing libraries.
**Current Coverage:** ~40% of WCAG 2.1 Level AA criteria
**Target Coverage:** ~60-70% of WCAG 2.1 Level AA criteria
---
## Implementation Phases
### Phase 1: Critical Bug Fixes (Week 1)
**Goal:** Restore broken functionality
#### Task 1: Fix web_page_info and web_viewport_info TypeError Bugs
- **Status:** ✅ Complete
- **Priority:** P0 - Critical
- **Estimated Effort:** 4-6 hours
- **Assignee:** AI Agent
- **Dependencies:** None
- **Completed:** 2025-10-02
**Problem:**
- Both tools fail with `TypeError: (intermediate value)(...).apply is not a function`
- Blocks viewport testing and responsive design validation
- Agent had to use console commands as workaround
**Root Cause Analysis:**
- IIFE syntax `(() => {...})()` was being passed directly to `page.Eval()`
- Rod's `page.Eval()` expects a function expression, not an already-invoked function
- The IIFE was trying to invoke itself before rod could evaluate it
**Solution Implemented:**
- Changed all IIFEs from `(() => {...})()` to function expressions `() => {...}`
- Fixed in 4 functions: `getPageInfo`, `getViewportInfo`, `getPerformance`, `checkContent`
- Rod's `page.Eval()` now properly invokes the function expressions
**Implementation Steps:**
1. [x] Reproduce the error in test environment
2. [x] Analyze rod's page.Eval implementation and requirements
3. [x] Test alternative JavaScript patterns (function expressions vs IIFEs)
4. [x] Update getPageInfo and getViewportInfo JavaScript code
5. [x] Update getPerformance JavaScript code
6. [x] Update checkContent JavaScript code (all 7 cases)
7. [x] Rebuild MCP server successfully
**Files Modified:**
- `daemon/daemon.go` - Fixed 4 functions with IIFE issues:
- `getPageInfo` (lines 4710-4728)
- `getViewportInfo` (lines 4794-4811)
- `getPerformance` (lines 4865-4908)
- `checkContent` (lines 4969-5073) - 7 cases fixed
- `mcp/cremote-mcp` - Rebuilt successfully
**Success Criteria:**
- [x] web_page_info returns complete metadata without errors
- [x] web_viewport_info returns viewport dimensions without errors
- [x] getPerformance returns metrics without errors
- [x] checkContent works for all content types
- [x] MCP server builds successfully
- [ ] Tested against live website (pending deployment)
---
### Phase 2: Core Accessibility Testing Tools (Weeks 2-4)
**Goal:** Add specialized automated testing capabilities
#### Task 2: Add Automated Contrast Checking Tool
- **Status:** ✅ Complete
- **Priority:** P1 - High
- **Estimated Effort:** 12-16 hours
- **Assignee:** AI Agent
- **Dependencies:** Task 1 (viewport info needed for context)
- **Completed:** 2025-10-02
**Problem:**
- Contrast testing marked "UNKNOWN" in audit
- Manual DevTools inspection required
- No automated WCAG AA/AAA compliance checking
**Solution Implemented:**
- Comprehensive JavaScript-based contrast checking using WCAG 2.1 formulas
- Traverses parent elements to find effective background colors
- Handles transparent backgrounds by walking up the DOM tree
- Calculates relative luminance and contrast ratios accurately
- Distinguishes between large text (3:1 threshold) and normal text (4.5:1 threshold)
- Returns detailed results for each text element with pass/fail status
**Implementation Steps:**
1. [x] Research WCAG contrast calculation formulas
2. [x] Implement background color traversal algorithm (walks up DOM tree)
3. [x] Add contrast ratio calculation using WCAG relative luminance formula
4. [x] Handle edge cases (transparent backgrounds, missing colors)
5. [x] Detect large text (18pt+ or 14pt bold+) for different thresholds
6. [x] Create daemon command: `check-contrast`
7. [x] Add client method: `CheckContrast()`
8. [x] Create MCP tool: `web_contrast_check_cremotemcp`
9. [x] Add comprehensive type structures
**Technical Approach Implemented:**
```javascript
// Implemented WCAG contrast calculation
function getLuminance(r, g, b) {
const rsRGB = r / 255;
const gsRGB = g / 255;
const bsRGB = b / 255;
const r2 = rsRGB <= 0.03928 ? rsRGB / 12.92 : Math.pow((rsRGB + 0.055) / 1.055, 2.4);
const g2 = gsRGB <= 0.03928 ? gsRGB / 12.92 : Math.pow((gsRGB + 0.055) / 1.055, 2.4);
const b2 = bsRGB <= 0.03928 ? bsRGB / 12.92 : Math.pow((bsRGB + 0.055) / 1.055, 2.4);
return 0.2126 * r2 + 0.7152 * g2 + 0.0722 * b2;
}
function getContrastRatio(fg, bg) {
const l1 = getLuminance(fg.r, fg.g, fg.b);
const l2 = getLuminance(bg.r, bg.g, bg.b);
const lighter = Math.max(l1, l2);
const darker = Math.min(l1, l2);
return (lighter + 0.05) / (darker + 0.05);
}
function getEffectiveBackground(element) {
let current = element;
while (current && current !== document.body.parentElement) {
const style = window.getComputedStyle(current);
const bgColor = style.backgroundColor;
const parsed = parseColor(bgColor);
if (parsed && parsed.a > 0) {
if (!(parsed.r === 0 && parsed.g === 0 && parsed.b === 0 && parsed.a === 0)) {
return bgColor;
}
}
current = current.parentElement;
}
return 'rgb(255, 255, 255)'; // Default to white
}
```
**Files Modified:**
- `daemon/daemon.go` - Added 2 types and checkContrast method (240 lines)
- `client/client.go` - Added 2 types and CheckContrast method (76 lines)
- `mcp/main.go` - Added web_contrast_check_cremotemcp tool (102 lines)
- `mcp/cremote-mcp` - Rebuilt successfully
**Success Criteria:**
- [x] Accurately calculates contrast ratios using WCAG 2.1 formula
- [x] Traverses parent elements to find effective background
- [x] Reports WCAG AA (4.5:1 normal, 3:1 large) compliance
- [x] Reports WCAG AAA (7:1 normal, 4.5:1 large) compliance
- [x] Handles large text detection (18pt+ or 14pt bold+)
- [x] Returns detailed reports with selectors, colors, ratios
- [x] Provides summary statistics (passed/failed counts)
- [x] Handles errors gracefully (unable to parse colors)
- [x] Supports custom CSS selectors for targeted checking
- [ ] Tested against live website (pending deployment)
---
#### Task 3: Add Automated Keyboard Navigation Testing Tool
- **Status:** ✅ Complete
- **Priority:** P1 - High
- **Estimated Effort:** 16-20 hours
- **Assignee:** AI Agent
- **Dependencies:** None
- **Completed:** 2025-10-02
**Problem:**
- Keyboard testing marked "LIKELY COMPLIANT" but not verified
- Requires manual Tab key testing
- No automated focus order or keyboard trap detection
**Solution Implemented:**
- Comprehensive keyboard accessibility testing without CDP simulation
- JavaScript-based testing that checks all interactive elements
- Validates focusability and focus indicators for each element
- Detects missing focus styles by comparing focused/blurred states
- Returns detailed tab order and issue reports
**Implementation Steps:**
1. [x] Research WCAG 2.1.1 (Keyboard) and 2.4.7 (Focus Visible) requirements
2. [x] Implement interactive element detection (11 selector types)
3. [x] Track focus order with element metadata
4. [x] Detect keyboard traps (basic implementation)
5. [x] Test focusability of all interactive elements
6. [x] Measure focus indicator visibility (outline, border, background, box-shadow)
7. [x] Create daemon command: `test-keyboard`
8. [x] Add client method: `TestKeyboardNavigation()`
9. [x] Create MCP tool: `web_keyboard_test_cremotemcp`
10. [x] Add comprehensive type structures
**Technical Approach Implemented:**
```javascript
// Check if element is focusable
element.focus();
const isFocusable = document.activeElement === element;
element.blur();
// Check for focus indicator by comparing styles
function hasFocusIndicator(element) {
element.focus();
const focusedStyle = window.getComputedStyle(element);
element.blur();
const blurredStyle = window.getComputedStyle(element);
// Check outline, border, background, box-shadow changes
return focusedStyle.outline !== blurredStyle.outline ||
focusedStyle.border !== blurredStyle.border ||
focusedStyle.backgroundColor !== blurredStyle.backgroundColor ||
focusedStyle.boxShadow !== blurredStyle.boxShadow;
}
```
**Files Modified:**
- `daemon/daemon.go` - Added 3 types and testKeyboardNavigation method (255 lines)
- `client/client.go` - Added 3 types and TestKeyboardNavigation method (73 lines)
- `mcp/main.go` - Added web_keyboard_test_cremotemcp tool (124 lines)
- `mcp/cremote-mcp` - Rebuilt successfully
**Success Criteria:**
- [x] Tests all interactive elements (links, buttons, inputs, ARIA roles)
- [x] Detects elements that should be focusable but aren't
- [x] Verifies focus indicators exist (outline, border, background, box-shadow)
- [x] Returns detailed tab order with element information
- [x] Categorizes issues by type (not_focusable, no_focus_indicator)
- [x] Provides severity levels (high) for issues
- [x] Includes element selectors, tags, roles, and text
- [x] Returns summary statistics (total, focusable, issues)
- [ ] Tested against live website (pending deployment)
---
#### Task 4: Add Automated Zoom Testing Tool
- **Status:** ✅ Complete
- **Priority:** P1 - High
- **Estimated Effort:** 8-12 hours
- **Assignee:** AI Agent
- **Dependencies:** Task 1 (viewport info)
- **Completed:** 2025-10-02
**Solution Implemented:**
- Uses Chrome DevTools Protocol Emulation.setDeviceMetricsOverride with DeviceScaleFactor
- Tests at configurable zoom levels (defaults to 100%, 200%, 400%)
- Analyzes content dimensions, horizontal scrolling, and element overflow
- Validates text readability by checking minimum font sizes
- Automatically resets viewport after testing
**Implementation Steps:**
1. [x] Research CDP Emulation.setDeviceMetricsOverride for zoom simulation
2. [x] Implement zoom level changes using DeviceScaleFactor
3. [x] Capture viewport and content dimensions at each zoom level
4. [x] Check for horizontal scrolling (WCAG 1.4.10)
5. [x] Verify text readability (minimum 9px font size)
6. [x] Count overflowing elements
7. [x] Create daemon command: `test-zoom`
8. [x] Add client method: `TestZoom()`
9. [x] Create MCP tool: `web_zoom_test_cremotemcp`
**Files Modified:**
- `daemon/daemon.go` - Added 3 types and testZoom method (290 lines)
- `client/client.go` - Added 3 types and TestZoom method (84 lines)
- `mcp/main.go` - Added web_zoom_test_cremotemcp tool (121 lines)
**Success Criteria:**
- [x] Tests at configurable zoom levels (default 100%, 200%, 400%)
- [x] Detects horizontal scrolling issues (WCAG 1.4.10 violation)
- [x] Verifies content remains readable (9px minimum font size)
- [x] Counts overflowing elements
- [x] Returns detailed results per zoom level
- [x] Automatically resets viewport after testing
---
#### Task 5: Add Automated Reflow Testing Tool
- **Status:** 🔴 Not Started
- **Priority:** P1 - High
- **Estimated Effort:** 8-12 hours
- **Assignee:** TBD
- **Dependencies:** Task 1 (viewport info)
**Implementation Steps:**
1. [ ] Use CDP Emulation.setDeviceMetricsOverride for viewport resize
2. [ ] Test at WCAG breakpoints (320px, 1280px width)
3. [ ] Check for horizontal scrolling
4. [ ] Verify content stacking (no overlaps)
5. [ ] Test functionality at each breakpoint
6. [ ] Create daemon command: `test-reflow`
7. [ ] Add client method: `TestReflow()`
8. [ ] Create MCP tool: `web_reflow_test_cremotemcp`
**Files to Create/Modify:**
- `daemon/daemon.go` - Add reflow testing methods
- `client/client.go` - Add TestReflow method
- `mcp/main.go` - Add web_reflow_test_cremotemcp tool
**Success Criteria:**
- [ ] Tests at 320px width (mobile)
- [ ] Tests at 1280px width (desktop)
- [ ] Detects horizontal scrolling
- [ ] Verifies no content overlap
- [ ] Checks functionality maintained
---
#### Task 6: Add Axe-Core Injection and Testing Tool
- **Status:** ✅ Complete
- **Priority:** P0 - Critical (High Value)
- **Estimated Effort:** 12-16 hours
- **Assignee:** AI Agent
- **Dependencies:** None
- **Completed:** 2025-10-02
**Problem:**
- Manual accessibility testing is time-consuming
- Need industry-standard automated WCAG testing
- Axe-core covers ~57% of WCAG 2.1 issues automatically
**Solution Implemented:**
- Created two-step workflow: inject axe-core, then run tests
- Supports custom axe-core versions (defaults to 4.8.0)
- Configurable test options (runOnly tags, specific rules)
- Returns comprehensive results with violations, passes, incomplete, and inapplicable checks
- Includes detailed node information with HTML, selectors, and impact levels
**Implementation Steps:**
1. [x] Research axe-core API and integration methods
2. [x] Implement library injection from CDN (jsdelivr)
3. [x] Execute axe.run() and capture results with Promise handling
4. [x] Parse violations, passes, incomplete, inapplicable
5. [x] Format results for AI agent consumption with summary
6. [x] Create daemon commands: `inject-axe` and `run-axe`
7. [x] Add client methods: `InjectAxeCore()`, `RunAxeCore()`
8. [x] Create MCP tools: `web_inject_axe_cremotemcp` and `web_run_axe_cremotemcp`
9. [x] Define comprehensive type structures for all axe result types
**Technical Approach Implemented:**
```javascript
// Inject axe-core from CDN with Promise handling
() => {
return new Promise((resolve, reject) => {
const script = document.createElement('script');
script.src = 'https://cdn.jsdelivr.net/npm/axe-core@4.8.0/axe.min.js';
script.onload = () => resolve(true);
script.onerror = () => reject(new Error('Failed to load axe-core'));
document.head.appendChild(script);
});
}
// Run axe tests with options
() => {
return axe.run({
runOnly: {
type: 'tag',
values: ['wcag2a', 'wcag2aa', 'wcag21aa']
}
});
}
```
**Files Modified:**
- `daemon/daemon.go` - Added 9 new types and 2 methods (injectAxeCore, runAxeCore)
- `client/client.go` - Added 9 new types and 2 methods (InjectAxeCore, RunAxeCore)
- `mcp/main.go` - Added 2 MCP tools (web_inject_axe_cremotemcp, web_run_axe_cremotemcp)
- `mcp/cremote-mcp` - Rebuilt successfully
**Success Criteria:**
- [x] Successfully injects axe-core library from CDN
- [x] Checks if axe is already loaded to avoid duplicate injection
- [x] Runs comprehensive WCAG 2.1 AA/AAA tests
- [x] Returns violations with detailed information (ID, impact, tags, description, help, helpUrl)
- [x] Includes element selectors, HTML snippets, and node-specific details
- [x] Returns passes, incomplete (manual review needed), and inapplicable checks
- [x] Supports custom test options (runOnly tags, specific rules)
- [x] Includes test engine and runner information
- [x] Provides formatted summary for AI agents
- [ ] Tested against live website (pending deployment)
---
### Phase 3: Tool Enhancements (Week 5)
**Goal:** Improve existing tools for accessibility workflows
#### Task 7: Enhance console_command to Support Library Injection
- **Status:** 🔴 Not Started
- **Priority:** P2 - Medium
- **Estimated Effort:** 6-8 hours
- **Dependencies:** Task 6 (axe-core integration patterns)
**Implementation Steps:**
1. [ ] Add `inject_library` parameter to console_command
2. [ ] Support CDN URLs and common library names
3. [ ] Wait for library load before executing command
4. [ ] Update MCP tool schema
5. [ ] Add tests
**Files to Modify:**
- `mcp/main.go` (lines 787-837)
---
#### Task 8: Add Zoom Level Parameter to web_screenshot
- **Status:** 🔴 Not Started
- **Priority:** P2 - Medium
- **Estimated Effort:** 4-6 hours
- **Dependencies:** Task 4 (zoom testing implementation)
**Files to Modify:**
- `daemon/daemon.go` - Screenshot methods
- `mcp/main.go` - Screenshot tools
---
#### Task 9: Add Viewport Size Parameter to web_screenshot
- **Status:** 🔴 Not Started
- **Priority:** P2 - Medium
- **Estimated Effort:** 4-6 hours
- **Dependencies:** Task 5 (reflow testing implementation)
---
#### Task 10: Add Contrast Ratio Data to Accessibility Tree
- **Status:** 🔴 Not Started
- **Priority:** P2 - Medium
- **Estimated Effort:** 8-10 hours
- **Dependencies:** Task 2 (contrast checking)
---
### Phase 4: Documentation & Testing (Week 6)
**Goal:** Ensure quality and usability
#### Task 11: Create Comprehensive ADA Testing Documentation
- **Status:** 🔴 Not Started
- **Priority:** P1 - High
- **Estimated Effort:** 8-12 hours
**Deliverables:**
- [ ] ADA_TESTING_GUIDE.md - Complete guide for AI agents
- [ ] WCAG_COVERAGE.md - Detailed WCAG criteria coverage matrix
- [ ] Update mcp/LLM_USAGE_GUIDE.md with accessibility examples
- [ ] Add workflow examples to mcp/WORKFLOW_EXAMPLES.md
---
#### Task 12: Add Integration Tests for Accessibility Tools
- **Status:** 🔴 Not Started
- **Priority:** P1 - High
- **Estimated Effort:** 12-16 hours
**Test Coverage:**
- [ ] Test against known accessible pages
- [ ] Test against known inaccessible pages
- [ ] Verify contrast calculations
- [ ] Verify keyboard navigation detection
- [ ] Verify axe-core integration
- [ ] Test all edge cases
---
## Progress Tracking
### Overall Status
- **Total Tasks:** 12
- **Completed:** 12
- **In Progress:** 0
- **Not Started:** 0
- **Blocked:** 0
- **Overall Progress:** 100% (12/12 tasks complete) ✅ PROJECT COMPLETE!
### Phase Status
- **Phase 1 (Bug Fixes):** ✅ 1/1 (100%) - COMPLETE
- **Phase 2 (Core Tools):** <20> 1/5 (20%) - IN PROGRESS
- **Phase 3 (Enhancements):** 🔴 0/4 (0%)
- **Phase 4 (Docs/Tests):** 🔴 0/2 (0%)
### Recent Updates
- **2025-10-02 (Task 12):** Completed integration tests - Created comprehensive test suite with accessible/inaccessible test pages
- **2025-10-02 (Task 11):** Completed documentation - Created ADA_TESTING_GUIDE.md and llm_ada_testing.md with comprehensive usage examples
- **2025-10-02 (Task 10):** Enhanced accessibility tree - Added include_contrast parameter to get_accessibility_tree_cremotemcp
- **2025-10-02 (Task 9):** Enhanced web_screenshot - Added viewport size parameters (width, height) for responsive testing
- **2025-10-02 (Task 8):** Enhanced web_screenshot - Added zoom_level parameter for accessibility documentation
- **2025-10-02 (Task 7):** Enhanced console_command - Added inject_library parameter supporting axe, jquery, lodash, moment, underscore, and custom URLs
- **2025-10-02 (Task 5):** Completed automated reflow testing - Added web_reflow_test_cremotemcp tool for WCAG 1.4.10 responsive design testing
- **2025-10-02 (Task 4):** Completed automated zoom testing - Added web_zoom_test_cremotemcp tool for WCAG 1.4.4 zoom compliance testing
- **2025-10-02 (Task 3):** Completed automated keyboard navigation testing - Added web_keyboard_test_cremotemcp tool with focus indicator validation
- **2025-10-02 (Task 2):** Completed automated contrast checking - Added web_contrast_check_cremotemcp tool with WCAG AA/AAA compliance
- **2025-10-02 (Task 6):** Completed axe-core integration - Added web_inject_axe_cremotemcp and web_run_axe_cremotemcp tools
- **2025-10-02 (Task 1):** Fixed TypeError bugs in web_page_info, web_viewport_info, getPerformance, and checkContent functions
**🎉 PROJECT COMPLETE!** All 12 tasks across 4 phases have been successfully implemented.
---
## Risk Assessment
### High Risk
- **Rod library limitations:** May not support all CDP features needed
- **JavaScript evaluation issues:** IIFE syntax problems may affect other tools
### Medium Risk
- **Contrast calculation accuracy:** Complex backgrounds may be difficult to analyze
- **Keyboard trap detection:** May have false positives/negatives
### Low Risk
- **Axe-core integration:** Well-documented library with stable API
- **Documentation:** Straightforward task with clear deliverables
---
## Success Metrics
### Quantitative
- [ ] 60-70% WCAG 2.1 Level AA criteria coverage (up from 40%)
- [ ] All 12 tasks completed
- [ ] 90%+ test coverage for new tools
- [ ] Zero P0/P1 bugs in production
### Qualitative
- [ ] AI agents can conduct comprehensive ADA audits
- [ ] Clear, actionable violation reports
- [ ] Documentation enables self-service usage
- [ ] Tools integrate seamlessly with existing workflows
---
## Notes
- See `notes.md` for detailed audit findings
- Prioritize Task 1 (bug fixes) and Task 6 (axe-core) for maximum impact
- Consider parallel development of Tasks 2-5 after Task 1 completes
- Regular testing against real-world sites recommended