remove sensory tools

2025-10-07 11:47:47 -05:00
parent a11d243d6a
commit ccd8c77a3e
56 changed files with 7235 additions and 430 deletions
--- a/docs/llm_instructions.md
+++ b/docs/llm_instructions.md
@@ -9,6 +9,10 @@ All cremote MCP tools use the **single suffix** naming pattern: `toolname_cremot
 **Correct:** `web_run_axe_cremotemcp`
 **Incorrect:** `web_run_axe` (missing suffix) or `web_run_axe_cremotemcp_cremotemcp` (double suffix)

+## NOTE
+
+Our cremote browser is running in a container. When you take a screenshot, you need to use the file download tool to bring it to your local machine.
+
 ## Quick Reference

 ### Token-Efficient Summary Tools (NEW - 2025-10-03)
@@ -45,7 +49,6 @@ These tools reduce token usage by 85-95% while providing comprehensive, actionab
 | Hover/focus content | `web_hover_focus_test_cremotemcp` | - | 1.4.13 |
 | Text in images | `web_text_in_images_cremotemcp` | - | 1.4.5, 1.4.9, 1.1.1 |
 | Cross-page consistency | `web_cross_page_consistency_cremotemcp` | - | 3.2.3, 3.2.4, 1.3.1 |
-| Sensory instructions | `web_sensory_characteristics_cremotemcp` | - | 1.3.3 |
 | Animations/flashing | `web_animation_flash_cremotemcp` | - | 2.3.1, 2.2.2, 2.3.2 |
 | ARIA validation | `web_enhanced_accessibility_cremotemcp` | `web_run_axe_cremotemcp` | 1.3.1, 4.1.2, 2.4.6 |
 | Keyboard accessibility | `web_keyboard_test_cremotemcp` | `web_run_axe_cremotemcp` | 2.1.1, 2.4.7 |
@@ -81,10 +84,9 @@ These tools reduce token usage by 85-95% while providing comprehensive, actionab
 5. web_media_validation_cremotemcp              # Video/audio caption validation
 6. web_hover_focus_test_cremotemcp              # Hover/focus content testing
 7. web_text_in_images_cremotemcp                # Text-in-images detection
-8. web_sensory_characteristics_cremotemcp       # Sensory instruction detection
-9. web_animation_flash_cremotemcp               # Animation/flash detection
-10. web_enhanced_accessibility_cremotemcp       # Enhanced ARIA validation
-11. web_keyboard_test_cremotemcp                # Keyboard navigation testing
+8. web_animation_flash_cremotemcp               # Animation/flash detection
+9. web_enhanced_accessibility_cremotemcp        # Enhanced ARIA validation
+10. web_keyboard_test_cremotemcp                # Keyboard navigation testing
 12. web_zoom_test_cremotemcp                    # Zoom functionality testing
 13. web_reflow_test_cremotemcp                  # Responsive design testing
 ```
@@ -413,26 +415,7 @@ These tools reduce token usage by 85-95% while providing comprehensive, actionab
 // - navigation_issues: Inconsistent navigation structure
 ```

-### Pattern 15: Sensory Characteristics Detection
-
-```json
-// Detect instructions relying on sensory characteristics
-{
-  "tool": "web_sensory_characteristics_cremotemcp",
-  "arguments": {
-    "timeout": 10
-  }
-}
-
-// Analyze output for:
-// - color_only: "Click the red button" (violation)
-// - shape_only: "Press the round icon" (violation)
-// - sound_only: "Listen for the beep" (violation)
-// - location_visual: "See above" (warning)
-// - size_only: "Click the large button" (warning)
-```
-
-### Pattern 16: Animation/Flash Detection
+### Pattern 15: Animation/Flash Detection

 ```json
 // Detect animations and flashing content
@@ -450,7 +433,7 @@ These tools reduce token usage by 85-95% while providing comprehensive, actionab
 // - animation_types: CSS, GIF, video, canvas, SVG
 ```

-### Pattern 17: Enhanced ARIA Validation
+### Pattern 16: Enhanced ARIA Validation

 ```json
 // Perform enhanced accessibility tree analysis
@@ -468,6 +451,260 @@ These tools reduce token usage by 85-95% while providing comprehensive, actionab
 // - landmark_issues: Multiple landmarks without distinguishing labels
 ```

+## CRITICAL: Understanding Tool Output vs Compliance Scoring
+
+### ⚠️ DO NOT CONFUSE TEST EXECUTION SUCCESS WITH COMPLIANCE SCORES
+
+**IMPORTANT:** Testing tools may return an `overall_score` or `status` field that indicates **test execution success**, NOT compliance/accessibility scores.
+
+**Test Execution Success (100/100):**
+- Means: All tests ran successfully without errors
+- Means: The page loaded and tools executed properly
+- **DOES NOT MEAN:** The page is accessible or compliant
+
+**Compliance Score (Calculated by You):**
+- Based on: Actual violations, failures, and issues found
+- Considers: Severity, impact, and percentage of failures
+- **THIS is what you report to the user**
+
+### Compliance Scoring Methodology
+
+When generating reports, **YOU MUST CALCULATE** the actual compliance score based on findings:
+
+#### Scoring Formula
+
+```
+Base Score: 100 points
+
+Deductions:
+1. Axe-core violations:
+   - Critical violations: -10 points each
+   - Serious violations: -5 points each
+   - Moderate violations: -2 points each
+   - Minor violations: -1 point each
+
+2. Contrast failures:
+   - 0-10% failure rate: -5 points
+   - 11-20% failure rate: -10 points
+   - 21-30% failure rate: -15 points
+   - 31-40% failure rate: -20 points
+   - 41%+ failure rate: -25 points
+
+3. Keyboard accessibility:
+   - 1-10 missing focus indicators: -5 points
+   - 11-25 missing focus indicators: -10 points
+   - 26-50 missing focus indicators: -15 points
+   - 51+ missing focus indicators: -20 points
+   - Keyboard traps detected: -15 points each
+
+4. Form accessibility:
+   - Missing labels: -5 points per form
+   - No ARIA compliance: -10 points per form
+   - Not keyboard accessible: -10 points per form
+
+5. Structural issues:
+   - Missing landmarks: -10 points
+   - Duplicate IDs: -5 points each
+   - Invalid ARIA: -5 points per violation
+
+Final Score = Base Score - Total Deductions (minimum 0)
+```
+
+#### Compliance Status Thresholds
+
+```
+95-100 points: FULLY COMPLIANT
+  - Minor issues only
+  - Legal risk: VERY LOW
+  - Status: "COMPLIANT"
+
+80-94 points: SUBSTANTIALLY COMPLIANT
+  - Some moderate issues requiring remediation
+  - Legal risk: LOW
+  - Status: "SUBSTANTIALLY COMPLIANT (with remediation needed)"
+
+60-79 points: PARTIALLY COMPLIANT
+  - Multiple serious issues
+  - Legal risk: MODERATE
+  - Status: "PARTIALLY COMPLIANT (requires significant remediation)"
+
+40-59 points: MINIMALLY COMPLIANT
+  - Major accessibility barriers
+  - Legal risk: HIGH
+  - Status: "MINIMALLY COMPLIANT (major remediation required)"
+
+0-39 points: NON-COMPLIANT
+  - Critical accessibility failures
+  - Legal risk: CRITICAL
+  - Status: "NON-COMPLIANT (immediate remediation required)"
+```
+
+### Example: Correct Scoring
+
+**Tool Output:**
+```json
+{
+  "overall_score": 100,  // ← This is TEST EXECUTION success, NOT compliance
+  "status": "success",
+  "violations": [
+    {"impact": "serious", "id": "color-contrast"},
+    {"impact": "serious", "id": "link-name"}
+  ],
+  "contrast_summary": {
+    "total_checked": 217,
+    "passed": 147,
+    "failed": 70,
+    "pass_rate": 67.7
+  },
+  "keyboard_summary": {
+    "missing_focus_indicators": 15
+  }
+}
+```
+
+**Your Calculated Compliance Score:**
+```
+Base Score: 100
+
+Deductions:
+- 2 serious axe-core violations: -10 points (2 × 5)
+- 32.3% contrast failure rate: -20 points
+- 15 missing focus indicators: -10 points
+
+Final Score: 100 - 10 - 20 - 10 = 60/100
+
+Compliance Status: PARTIALLY COMPLIANT
+Legal Risk: MODERATE
+```
+
+**What You Report:**
+```markdown
+**Compliance Score:** 60/100 - PARTIALLY COMPLIANT
+**Legal Risk:** MODERATE
+**Test Execution:** Successful (all tests completed)
+
+**Critical Issues:**
+- Color Contrast: 32.3% failure rate (70/217 elements)
+- Axe-core violations: 2 serious issues
+- Focus indicators: 15 missing
+
+**Estimated Remediation:** 5-7 hours
+```
+
+### ❌ INCORRECT Reporting Example
+
+**DO NOT DO THIS:**
+```markdown
+**Overall Score:** 100/100 (with noted issues)  ← WRONG!
+**Compliance Status:** COMPLIANT (with remediation needed)  ← CONTRADICTORY!
+
+**Contrast Analysis:**
+- Failed: 70 (32.3%)  ← This contradicts "COMPLIANT"
+```
+
+### ✅ CORRECT Reporting Example
+
+**DO THIS:**
+```markdown
+**Compliance Score:** 60/100 - PARTIALLY COMPLIANT
+**Legal Risk:** MODERATE
+**Test Execution Status:** All tests completed successfully
+
+**Contrast Analysis:**
+- Total elements: 217
+- Passed: 147 (67.7%)
+- Failed: 70 (32.3%)
+- Impact: -20 points from compliance score
+
+**Axe-Core Violations:**
+- Serious: 2 violations
+- Impact: -10 points from compliance score
+
+**Keyboard Navigation:**
+- Missing focus indicators: 15 elements
+- Impact: -10 points from compliance score
+
+**Remediation Priority:**
+1. Fix contrast issues (HIGH priority)
+2. Add focus indicators (HIGH priority)
+3. Resolve axe-core violations (MEDIUM priority)
+```
+
+### Page-by-Page Reporting Template
+
+Use this template for each page in your reports:
+
+```markdown
+### [Page Name] ([URL])
+
+**Compliance Score:** [0-100]/100 - [STATUS]
+**Legal Risk:** [VERY LOW | LOW | MODERATE | HIGH | CRITICAL]
+**Screenshot:** `screenshots/[filename].png`
+
+**Score Breakdown:**
+- Base score: 100
+- Axe-core violations: -[X] points ([count] violations)
+- Contrast failures: -[X] points ([percentage]% failure rate)
+- Keyboard issues: -[X] points ([count] missing indicators)
+- Form issues: -[X] points ([count] forms with issues)
+- Structural issues: -[X] points ([count] issues)
+- **Final Score:** [0-100]/100
+
+**Detailed Findings:**
+
+**Contrast Analysis:**
+- Total elements checked: [number]
+- Passed: [number] ([percentage]%)
+- Failed: [number] ([percentage]%)
+
+**Keyboard Navigation:**
+- Total interactive elements: [number]
+- Focusable: [number]
+- Missing focus indicators: [number]
+- Keyboard traps: [number]
+
+**Axe-Core Results:**
+- Critical violations: [number]
+- Serious violations: [number]
+- Moderate violations: [number]
+- Minor violations: [number]
+- Passes: [number]
+
+**Forms:**
+- Forms found: [number]
+- Issues: [list or "None"]
+- ARIA Compliance: [FULL | PARTIAL | NONE]
+
+**Zoom/Reflow Testing:**
+- 100% zoom: [PASS | FAIL] - [details]
+- 200% zoom: [PASS | FAIL] - [details]
+- 400% zoom: [PASS | FAIL] - [details]
+- 320px width: [PASS | FAIL] - [details]
+```
+
+### Site-Wide Summary Template
+
+```markdown
+## Executive Summary
+
+**Overall Site Compliance:** [Average score]/100 - [STATUS]
+**Legal Risk Assessment:** [VERY LOW | LOW | MODERATE | HIGH | CRITICAL]
+**Pages Tested:** [number]
+
+**Compliance Score Distribution:**
+- Fully Compliant (95-100): [number] pages
+- Substantially Compliant (80-94): [number] pages
+- Partially Compliant (60-79): [number] pages
+- Minimally Compliant (40-59): [number] pages
+- Non-Compliant (0-39): [number] pages
+
+**Site-Wide Issues:**
+1. [Issue type]: Affects [number] pages - [severity]
+2. [Issue type]: Affects [number] pages - [severity]
+
+**Total Estimated Remediation Time:** [hours] hours
+```
+
 ## Interpreting Results

 ### Axe-Core Results
@@ -611,10 +848,9 @@ Total for 10 pages: ~50k tokens (vs 800k+ with traditional approach)
   c. web_media_validation_cremotemcp if page has video/audio
   d. web_hover_focus_test_cremotemcp for tooltips/popovers
   e. web_text_in_images_cremotemcp for infographics/charts
-   f. web_sensory_characteristics_cremotemcp for instructional content
-   g. web_animation_flash_cremotemcp for animated content
-   h. web_enhanced_accessibility_cremotemcp for ARIA validation
-   i. web_keyboard_test_cremotemcp for keyboard issues
+   f. web_animation_flash_cremotemcp for animated content
+   g. web_enhanced_accessibility_cremotemcp for ARIA validation
+   h. web_keyboard_test_cremotemcp for keyboard issues
 5. Run web_zoom_test_cremotemcp
 6. Run web_reflow_test_cremotemcp
 7. Capture screenshots for documentation
@@ -767,6 +1003,245 @@ Solution: Navigate to a page first using web_navigate_cremotemcp
 Solution: Verify tab ID is valid, check if page is still loaded
 ```

+## Professional Reporting Standards
+
+### ⚠️ CRITICAL: Client-Facing Report Requirements
+
+**ALL reports must be professional, client-facing documents. Follow these rules strictly:**
+
+#### 1. Assessor Identity
+- **ALWAYS use:** "Shortcut Solutions" as the assessor
+- **NEVER mention:** Augment AI, AI agents, LLM, automation, cremote, MCP tools, or any specific tool names
+
+**Correct:**
+```markdown
+**Assessment Date:** October 7, 2025
+**Assessor:** Shortcut Solutions
+**Standard:** WCAG 2.1 Level AA
+**Methodology:** Comprehensive accessibility testing including automated scanning, manual verification, and assistive technology evaluation
+```
+
+**Incorrect:**
+```markdown
+**Assessor:** Augment AI Agent with cremotemcp Tools  ← NEVER DO THIS
+**Methodology:** Automated testing using axe-core, cremote tools  ← NEVER DO THIS
+```
+
+#### 2. Methodology Description
+- **Use generic professional language** about testing methods
+- **Focus on standards and techniques**, not specific tools
+- **Emphasize comprehensive evaluation**, not automation
+
+**Correct terminology:**
+- "Comprehensive accessibility testing"
+- "Automated scanning combined with manual verification"
+- "Assistive technology evaluation"
+- "WCAG 2.1 Level AA compliance testing"
+- "Color contrast analysis"
+- "Keyboard navigation assessment"
+- "Screen reader compatibility testing"
+
+**Incorrect terminology:**
+- "Using cremote tools"
+- "Ran axe-core v4.8.0"
+- "web_page_accessibility_report_cremotemcp"
+- "AI-powered assessment"
+- "Automated by Augment Agent"
+
+#### 3. Report Headers and Footers
+
+**Correct format:**
+```markdown
+# ADA Level AA Accessibility Assessment Report
+## [Client Company Name] Website
+
+**Assessment Date:** [Date]
+**Assessor:** Shortcut Solutions
+**Standard:** WCAG 2.1 Level AA
+**Methodology:** Comprehensive accessibility testing including automated scanning, manual verification, keyboard navigation testing, and screen reader evaluation
+
+---
+
+[Report content]
+
+---
+
+**Report Generated:** [Date]
+**Prepared By:** Shortcut Solutions
+**Standard:** WCAG 2.1 Level AA
+**Contact:** For questions about this assessment, please contact Shortcut Solutions
+```
+
+**Incorrect format:**
+```markdown
+**Assessment Tool:** Augment AI Agent with cremotemcp  ← NEVER
+**Testing Tool:** cremote MCP Tools  ← NEVER
+**Powered by:** Augment Code  ← NEVER
+```
+
+#### 4. Testing Methodology Section
+
+**Correct approach:**
+```markdown
+## Testing Methodology
+
+### Standards and Guidelines
+This assessment evaluates compliance with Web Content Accessibility Guidelines (WCAG) 2.1 Level AA, the internationally recognized standard for web accessibility.
+
+### Testing Approach
+The assessment employed a comprehensive multi-faceted approach:
+
+1. **Automated Scanning**
+   - Comprehensive automated accessibility scanning
+   - Color contrast ratio analysis
+   - HTML/ARIA validation
+   - Structural and semantic markup evaluation
+
+2. **Manual Testing**
+   - Keyboard-only navigation testing
+   - Focus indicator verification
+   - Form accessibility evaluation
+   - Interactive element testing
+
+3. **Assistive Technology Evaluation**
+   - Screen reader compatibility assessment
+   - Accessibility tree structure validation
+   - ARIA implementation verification
+
+4. **Responsive Design Testing**
+   - Multiple viewport sizes (320px to 1280px)
+   - Zoom functionality (100%, 200%, 400%)
+   - Content reflow verification
+
+5. **Cross-Page Consistency**
+   - Navigation structure consistency
+   - Landmark presence and placement
+   - Interactive element behavior
+
+### Pages Tested
+[List of pages]
+
+### Test Coverage
+- ✅ Automated WCAG 2.1 AA compliance testing
+- ✅ Color contrast analysis
+- ✅ Keyboard navigation and focus management
+- ✅ Form accessibility
+- ✅ Responsive design and zoom functionality
+- ✅ Semantic HTML structure
+- ✅ ARIA implementation
+- ✅ Cross-page consistency
+```
+
+**Incorrect approach:**
+```markdown
+## Testing Methodology
+
+### Tools Used
+1. **axe-core v4.8.0** - Comprehensive automated WCAG testing  ← NEVER
+2. **Chromium Accessibility Tree** - Semantic structure validation  ← NEVER
+3. **cremote MCP tools** - Automated testing suite  ← NEVER
+```
+
+#### 5. Findings and Recommendations
+
+**Focus on:**
+- What was found (the issue)
+- Why it matters (impact on users)
+- How to fix it (remediation)
+
+**Do NOT mention:**
+- Which tool detected it
+- How the tool works
+- Tool-specific output or error messages
+
+**Correct:**
+```markdown
+### Color Contrast Failures (WCAG 1.4.3 - Level AA)
+
+**Finding:** Navigation menu items have insufficient color contrast.
+
+**Current State:** Text color #666666 on background #000000 provides a contrast ratio of 3.66:1
+
+**Required State:** WCAG AA requires a minimum contrast ratio of 4.5:1 for normal text
+
+**Impact:** Users with low vision or color blindness may have difficulty reading navigation text
+
+**Remediation:**
+```css
+.navigation-menu-item {
+  color: #FFFFFF; /* Provides 21:1 contrast ratio */
+}
+```
+
+**Estimated Time:** 2-3 hours
+```
+
+**Incorrect:**
+```markdown
+### Color Contrast Failures
+
+**Detection Method:** web_contrast_check_cremotemcp found violations  ← NEVER
+**Tool Output:** axe-core reported color-contrast rule failure  ← NEVER
+**Automated Scan:** cremote detected 70 failures  ← NEVER
+```
+
+#### 6. Appendix and Documentation
+
+**Correct:**
+```markdown
+## Appendix: Screenshots
+
+All visual documentation is available in the assessment package:
+- homepage-full.png
+- services-page-full.png
+- contact-form-full.png
+
+---
+
+**Report Prepared By:** Shortcut Solutions
+**Assessment Date:** October 7, 2025
+**Standard:** WCAG 2.1 Level AA
+**Methodology:** Comprehensive accessibility testing
+```
+
+**Incorrect:**
+```markdown
+## Appendix: Tools and Technologies Used
+
+### cremotemcp Tools Utilized  ← NEVER INCLUDE THIS SECTION
+1. web_navigate_cremotemcp
+2. web_screenshot_cremotemcp
+3. web_run_axe_cremotemcp
+```
+
+#### 7. Audit Trail (Internal Use Only)
+
+**If you need to document your process for internal tracking:**
+- Create a SEPARATE file (e.g., `INTERNAL_audit_log.md`)
+- Mark it clearly as "INTERNAL USE ONLY - NOT FOR CLIENT"
+- Never include tool names or automation details in client-facing reports
+
+#### 8. Summary of What to NEVER Include in Reports
+
+**NEVER mention:**
+- ❌ cremote, cremotemcp, MCP tools
+- ❌ Augment AI, Augment Agent, Augment Code
+- ❌ AI, LLM, machine learning, automation
+- ❌ Specific tool names (axe-core version numbers, etc.)
+- ❌ Tool command syntax or parameters
+- ❌ Container environments, file paths like /tmp/
+- ❌ Technical implementation details of testing tools
+- ❌ Token usage, efficiency metrics
+- ❌ "Automated by...", "Powered by...", "Using..."
+
+**ALWAYS use:**
+- ✅ "Shortcut Solutions" as assessor
+- ✅ Generic professional terminology
+- ✅ Focus on findings, impact, and remediation
+- ✅ Standards-based language (WCAG, ADA, Section 508)
+- ✅ Professional assessment methodology descriptions
+- ✅ Client-focused recommendations
+
 ## Best Practices for LLM Agents

 ### 1. Use Token-Efficient Tools for Site-Wide Assessments (NEW)
@@ -826,7 +1301,6 @@ These tools cannot test:
 - Cognitive load
 - Caption accuracy (speech-to-text validation) - Only presence is checked
 - Complex user interactions
- Context-dependent issues (some sensory characteristics need human judgment)
 - Video frame-by-frame flash analysis (simplified estimation used)
 - Complex ARIA widget validation (basic validation only)

@@ -839,7 +1313,6 @@ Always recommend manual testing with assistive technologies for comprehensive au
 - **Hover/Focus:** May miss custom implementations using non-standard patterns
 - **Text-in-Images:** OCR struggles with stylized fonts, handwriting, low contrast
 - **Cross-Page:** Requires 2+ pages, may flag intentional variations
- **Sensory Characteristics:** Context-dependent, may have false positives
 - **Animation/Flash:** Simplified flash rate estimation, no actual frame analysis
 - **Enhanced A11y:** Simplified reference validation, doesn't check all ARIA states

@@ -925,9 +1398,6 @@ cremote text-in-images
 # Check cross-page consistency
 cremote cross-page-consistency --urls "https://example.com/,https://example.com/about"

-# Detect sensory characteristics
-cremote sensory-characteristics
-
 # Detect animations and flashing
 cremote animation-flash

@@ -991,7 +1461,7 @@ The cremote platform provides comprehensive automated testing with two approache
 **Best for: Deep dives, raw data analysis**
 - **Core Tools:** Axe-core, contrast checking, keyboard testing, zoom/reflow testing
 - **Phase 1 Tools:** Gradient contrast, media validation, hover/focus testing
- **Phase 2 Tools:** Text-in-images, cross-page consistency, sensory characteristics
+- **Phase 2 Tools:** Text-in-images, cross-page consistency
 - **Phase 3 Tools:** Animation/flash detection, enhanced ARIA validation

 **What's Automated (93%):**
@@ -1008,7 +1478,6 @@ The cremote platform provides comprehensive automated testing with two approache
 - Video/audio captions (presence)
 - Text-in-images detection
 - Cross-page consistency
- Sensory characteristics
 - Animation/flash safety
 - Zoom and responsive design

@@ -1018,7 +1487,6 @@ The cremote platform provides comprehensive automated testing with two approache
 - Subjective content quality
 - Advanced ARIA widget validation
 - Video content frame analysis
- Context-dependent sensory instructions

 ---

@@ -1038,5 +1506,5 @@ The cremote platform provides comprehensive automated testing with two approache

 **LATEST UPDATE (2025-10-03):** Four new token-efficient summary tools have been added, reducing token usage by 85-95% and enabling comprehensive site-wide assessments. These tools provide structured, actionable results with compliance status, legal risk assessment, and remediation estimates. **Use these tools for all multi-page assessments.**

-**PREVIOUS UPDATE (2025-10-02):** Eight automated testing tools were added, increasing coverage from 70% to 93%. These tools provide enhanced detection for gradient contrast, media accessibility, hover/focus content, text-in-images, cross-page consistency, sensory characteristics, animations/flashing, and ARIA validation.
+**PREVIOUS UPDATE (2025-10-02):** Seven automated testing tools were added, increasing coverage from 70% to 93%. These tools provide enhanced detection for gradient contrast, media accessibility, hover/focus content, text-in-images, cross-page consistency, animations/flashing, and ARIA validation.