cremote/test/README.md

# Cremote Accessibility Integration Tests

## Overview

This directory contains comprehensive integration tests for Cremote's accessibility testing tools. The tests validate all ADA/WCAG compliance features against known accessible and inaccessible test pages.

## Test Coverage

### Tools Tested

1. **Axe-Core Integration** (`web_inject_axe_cremotemcp`, `web_run_axe_cremotemcp`)
   - Library injection
   - WCAG 2.1 AA/AAA automated testing
   - Violation detection and reporting

2. **Contrast Checking** (`web_contrast_check_cremotemcp`)
   - WCAG contrast ratio calculations
   - AA/AAA compliance validation
   - Violation reporting with element details

3. **Keyboard Navigation** (`web_keyboard_test_cremotemcp`)
   - Focusability testing
   - Focus indicator validation
   - Keyboard trap detection
   - Tab order verification

4. **Zoom Testing** (`web_zoom_test_cremotemcp`)
   - Multi-level zoom testing (100%, 200%, 400%)
   - Horizontal scroll detection
   - Text readability validation
   - Overflow detection

5. **Reflow Testing** (`web_reflow_test_cremotemcp`)
   - Responsive breakpoint testing (320px, 1280px)
   - Horizontal scroll detection
   - Responsive layout validation
   - Overflow detection

6. **Screenshot Enhancements** (`web_screenshot_cremotemcp`)
   - Zoom level screenshots
   - Viewport size screenshots
   - Full page screenshots

7. **Library Injection** (`console_command_cremotemcp`)
   - External library injection (jQuery, Lodash, etc.)
   - Custom URL injection
   - Command execution with injected libraries

8. **Accessibility Tree** (`get_accessibility_tree_cremotemcp`)
   - Full tree retrieval
   - Depth limiting
   - Contrast annotation support

## Test Files

### Test Pages

- **`testdata/test-accessible.html`** - WCAG 2.1 AA compliant page
  - High contrast colors (4.5:1+ for normal text)
  - Visible focus indicators
  - Semantic HTML
  - Proper ARIA labels
  - Responsive design (no horizontal scroll at 320px)
  - Keyboard accessible controls
  - Proper heading hierarchy
  - Form labels and associations

- **`testdata/test-inaccessible.html`** - Page with known violations
  - Low contrast text (3.2:1, 1.5:1)
  - Missing focus indicators
  - Non-keyboard accessible elements
  - Missing alt text
  - Fixed width causing horizontal scroll
  - Missing form labels
  - Heading hierarchy violations
  - Duplicate IDs
  - Empty links and buttons
  - Incorrect ARIA usage

### Test Suite

- **`accessibility_integration_test.go`** - Main test suite
  - `TestAccessibilityToolsIntegration` - Tests all tools on accessible page
  - `TestInaccessiblePage` - Tests violation detection on inaccessible page
  - `BenchmarkAxeCore` - Performance benchmarks

## Prerequisites

### 1. Start Cremote Daemon

```bash
# Start the daemon on default port 9223
cremote-daemon
```

### 2. Start Test HTTP Server

You need a simple HTTP server to serve the test HTML files:

```bash
# Option 1: Python 3
cd test/testdata
python3 -m http.server 8080

# Option 2: Python 2
cd test/testdata
python -m SimpleHTTPServer 8080

# Option 3: Node.js (http-server)
cd test/testdata
npx http-server -p 8080

# Option 4: Go
cd test/testdata
go run -m http.server 8080
```

The test server should serve files at `http://localhost:8080/`.

### 3. Set Environment Variable

```bash
export INTEGRATION_TESTS=true
```

## Running Tests

### Run All Integration Tests

```bash
cd test
INTEGRATION_TESTS=true go test -v
```

### Run Specific Test

```bash
cd test
INTEGRATION_TESTS=true go test -v -run TestAccessibilityToolsIntegration
```

### Run Specific Sub-Test

```bash
cd test
INTEGRATION_TESTS=true go test -v -run TestAccessibilityToolsIntegration/AxeCoreIntegration
```

### Run Benchmarks

```bash
cd test
INTEGRATION_TESTS=true go test -v -bench=. -benchmem
```

### Run with Coverage

```bash
cd test
INTEGRATION_TESTS=true go test -v -cover -coverprofile=coverage.out
go tool cover -html=coverage.out
```

## Expected Results

### Accessible Page Tests

When testing `test-accessible.html`, expect:

- **Axe-Core**: Few or no violations (< 5)
- **Contrast**: High AA pass rate (> 80%)
- **Keyboard**: High focusability rate (> 80%)
- **Zoom**: No horizontal scroll at 200%
- **Reflow**: No horizontal scroll at 320px

### Inaccessible Page Tests

When testing `test-inaccessible.html`, expect:

- **Axe-Core**: Multiple violations detected
- **Contrast**: Multiple AA failures
- **Keyboard**: Multiple focusability issues
- **Zoom**: Potential horizontal scroll issues
- **Reflow**: Horizontal scroll at 320px

## Test Output Example

```
=== RUN   TestAccessibilityToolsIntegration
=== RUN   TestAccessibilityToolsIntegration/AxeCoreIntegration
    accessibility_integration_test.go:72: Axe-core results: 2 violations, 45 passes, 3 incomplete, 12 inapplicable
=== RUN   TestAccessibilityToolsIntegration/ContrastChecking
    accessibility_integration_test.go:98: Contrast check: 38 elements, 36 AA pass, 2 AA fail, 28 AAA pass, 10 AAA fail
=== RUN   TestAccessibilityToolsIntegration/KeyboardNavigation
    accessibility_integration_test.go:124: Keyboard test: 25 elements, 23 focusable, 2 not focusable, 1 missing focus indicator, 0 keyboard traps
=== RUN   TestAccessibilityToolsIntegration/ZoomTesting
    accessibility_integration_test.go:152: Zoom test: tested 3 levels, found 0 issues
=== RUN   TestAccessibilityToolsIntegration/ReflowTesting
    accessibility_integration_test.go:180: Reflow test: tested 2 breakpoints, found 0 issues
=== RUN   TestAccessibilityToolsIntegration/ScreenshotEnhancements
    accessibility_integration_test.go:208: Screenshot functionality verified
=== RUN   TestAccessibilityToolsIntegration/LibraryInjection
    accessibility_integration_test.go:230: Library injection functionality verified
=== RUN   TestAccessibilityToolsIntegration/AccessibilityTree
    accessibility_integration_test.go:248: Accessibility tree: 156 nodes
--- PASS: TestAccessibilityToolsIntegration (15.23s)
    --- PASS: TestAccessibilityToolsIntegration/AxeCoreIntegration (3.45s)
    --- PASS: TestAccessibilityToolsIntegration/ContrastChecking (2.12s)
    --- PASS: TestAccessibilityToolsIntegration/KeyboardNavigation (1.89s)
    --- PASS: TestAccessibilityToolsIntegration/ZoomTesting (2.34s)
    --- PASS: TestAccessibilityToolsIntegration/ReflowTesting (1.98s)
    --- PASS: TestAccessibilityToolsIntegration/ScreenshotEnhancements (0.45s)
    --- PASS: TestAccessibilityToolsIntegration/LibraryInjection (1.23s)
    --- PASS: TestAccessibilityToolsIntegration/AccessibilityTree (1.77s)
PASS
ok      git.teamworkapps.com/shortcut/cremote/test      15.234s
```

## Troubleshooting

### Tests Skip with "Skipping integration tests"

**Problem:** Tests are being skipped.

**Solution:** Set the environment variable:
```bash
export INTEGRATION_TESTS=true
```

### "Failed to navigate to test page"

**Problem:** Test HTTP server is not running or not accessible.

**Solution:** 
1. Start HTTP server on port 8080
2. Verify server is accessible: `curl http://localhost:8080/test-accessible.html`
3. Check firewall settings

### "Failed to connect to daemon"

**Problem:** Cremote daemon is not running.

**Solution:**
```bash
# Start daemon
cremote-daemon

# Verify it's running
ps aux | grep cremote-daemon
```

### "Failed to inject axe-core"

**Problem:** Network issues or CDN unavailable.

**Solution:**
1. Check internet connectivity
2. Try alternative axe-core version
3. Check if CDN is accessible: `curl https://cdn.jsdelivr.net/npm/axe-core@4.8.0/axe.min.js`

### Tests Timeout

**Problem:** Tests are timing out.

**Solution:**
1. Increase timeout values in test code
2. Check system resources (CPU, memory)
3. Verify Chromium is not hanging

## CI/CD Integration

### GitHub Actions Example

```yaml
name: Accessibility Integration Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Set up Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.21
      
      - name: Install dependencies
        run: |
          sudo apt-get update
          sudo apt-get install -y chromium-browser
      
      - name: Start cremote daemon
        run: |
          ./cremote-daemon &
          sleep 5
      
      - name: Start test server
        run: |
          cd test/testdata
          python3 -m http.server 8080 &
          sleep 2
      
      - name: Run integration tests
        run: |
          cd test
          INTEGRATION_TESTS=true go test -v -cover
      
      - name: Upload coverage
        uses: codecov/codecov-action@v2
        with:
          files: ./test/coverage.out
```

## Adding New Tests

### 1. Create Test Function

```go
func testNewFeature(c *client.Client) func(*testing.T) {
    return func(t *testing.T) {
        // Test implementation
        result, err := c.NewFeature("", params, 10)
        if err != nil {
            t.Fatalf("Failed: %v", err)
        }
        
        // Assertions
        if result == nil {
            t.Error("Result is nil")
        }
    }
}
```

### 2. Add to Test Suite

```go
func TestAccessibilityToolsIntegration(t *testing.T) {
    // ... existing setup ...
    
    t.Run("NewFeature", testNewFeature(c))
}
```

### 3. Update Documentation

Update this README with:
- Tool description
- Expected results
- Common issues

## Performance Benchmarks

Expected performance on modern hardware:

- **Axe-Core**: ~2-4 seconds per run
- **Contrast Check**: ~1-2 seconds for 50 elements
- **Keyboard Test**: ~1-2 seconds for 30 elements
- **Zoom Test**: ~2-3 seconds for 3 zoom levels
- **Reflow Test**: ~1-2 seconds for 2 breakpoints

## Support

For issues or questions:
- Check the main documentation: `docs/ADA_TESTING_GUIDE.md`
- Review LLM guide: `docs/llm_ada_testing.md`
- File an issue: git.teamworkapps.com/shortcut/cremote/issues

---

**Last Updated:** 2025-10-02
**Version:** 1.0.0