
## Table of Contents

1. [Introduction](#introduction)
2. [The Problem](#the-problem)
3. [Why Lighthouse in CI/CD Matters](#why-lighthouse-in-cicd-matters)
4. [Issues Encountered](#issues-encountered)
5. [Solutions and Implementation](#solutions-and-implementation)
6. [Best Practices](#best-practices)
7. [Conclusion](#conclusion)

## Introduction

Website performance is critical to user experience, SEO rankings, and conversion rates. However, performance regressions often go unnoticed until they reach production. This post covers how to implement Lighthouse performance testing as part of your CI/CD pipeline to catch performance issues before they impact your users.

## The Problem

Performance regression detection is challenging because:

- **Manual Testing is Inefficient**: Running Lighthouse manually for every build wastes developer time and is easily forgotten.
- **No Baseline Comparison**: Without automated comparison, it's hard to know if a change degrades performance.
- **Silent Performance Degradation**: Small performance issues accumulate and go unnoticed until users complain.
- **Inconsistent Environments**: Local testing environments differ from production, making results unreliable.
- **Lack of Visibility**: Teams don't have a clear view of performance trends over time.

## Why Lighthouse in CI/CD Matters

Integrating Lighthouse into your CI/CD pipeline provides:

- **Automated Performance Checks**: Every pull request gets a performance audit automatically.
- **Baseline Comparisons**: Detect when performance scores drop relative to the previous build.
- **Fast Feedback Loop**: Developers get immediate performance feedback while making changes.
- **Enforced Standards**: Set minimum performance thresholds that block merges if not met.
- **Historical Tracking**: Build a performance history to understand trends and improvements.

## Issues Encountered

### 1. **Lighthouse Variability and Flaky Results**

Lighthouse results vary between runs due to:

- Network throttling differences
- CPU throttling inconsistencies
- Background processes on the CI runner

**Solution**: Run Lighthouse multiple times and average the results, or use median values. Set realistic thresholds (e.g., 85/100 instead of 95/100) to account for variability.

```yaml
# Example: Run Lighthouse 3 times and average scores
- name: Run Lighthouse Audit
  uses: actions/checkout@v3
- name: Setup Node.js
  uses: actions/setup-node@v3
  with:
    node-version: '18'
- name: Install dependencies
  run: npm install -g @lhci/cli@latest
- name: Run LHCI
  run: lhci autorun
```

### 2. **Lighthouse CLI Timeout Issues**

Large pages or slow networks can cause Lighthouse to timeout during CI runs.

**Solution**:

- Increase the timeout setting in `lighthouserc.js`:
  ```javascript
  module.exports = {
    ci: {
      collect: {
        numRetries: 3,
        settings: {
          chromeFlags: ['--disable-dev-shm-usage'], // Use disk instead of shared memory
          onlyCategories: ['performance', 'accessibility'], // Only audit needed categories
        },
      },
    },
  };
  ```
- Use `--disable-dev-shm-usage` flag to prevent memory issues on CI runners with limited shared memory.

### 3. **Authentication and Protected Pages**

Lighthouse cannot audit pages behind authentication or paywalls.

**Solution**:

- Test public pages only
- Use a test account for authenticated pages:
  ```javascript
  settings: {
    settings: {
      formFactor: 'mobile',
      screenEmulation: {
        mobile: true,
        width: 412,
        height: 823,
      }
    }
  }
  ```

### 4. **Comparison Baseline Not Available**

New projects don't have a baseline to compare against initially.

**Solution**:

- Store initial results as a baseline in your repository
- Use Lighthouse CI storage API to store historical results
- Start with informational thresholds and enforce stricter ones over time

### 5. **CI Runner Performance Constraints**

CI runners may have different hardware than production, causing misleading results.

**Solution**:

- Use consistent CI runners (e.g., GitHub Actions on the same machine type)
- Normalize CPU and network throttling settings
- Focus on relative changes rather than absolute scores

## Solutions and Implementation

### Step 1: Install Lighthouse CI

```bash
npm install -g @lhci/cli@latest
```

### Step 2: Create `lighthouserc.js` Configuration

```javascript
// lighthouserc.js
module.exports = {
  ci: {
    collect: {
      url: ['http://localhost:3000', 'http://localhost:3000/about'],
      numberOfRuns: 3,
      settings: {
        chromeFlags: ['--disable-dev-shm-usage', '--no-sandbox'],
      },
    },
    upload: {
      target: 'temporary-public-storage',
    },
    assert: {
      preset: 'lighthouse:recommended',
      assertions: {
        'categories:performance': ['error', { minScore: 0.85 }],
        'categories:accessibility': ['error', { minScore: 0.9 }],
        'categories:best-practices': ['error', { minScore: 0.85 }],
        'categories:seo': ['error', { minScore: 0.9 }],
      },
    },
  },
};
```

### Step 3: GitHub Actions Workflow

```yaml
# .github/workflows/lighthouse.yml
name: Lighthouse CI

on: [push, pull_request]

jobs:
  lighthouse:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'

      - name: Install dependencies
        run: npm ci

      - name: Build application
        run: npm run build

      - name: Install Lighthouse CI
        run: npm install -g @lhci/cli@latest

      - name: Start local server
        run: npm start &
        env:
          NODE_ENV: production

      - name: Wait for server
        run: sleep 5

      - name: Run Lighthouse CI
        run: lhci autorun

      - name: Upload Lighthouse results
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: lighthouse-results
          path: .lighthouseci/
```

### Step 4: Store and Compare Results

Use Lighthouse CI's temporary public storage or set up persistent storage:

```javascript
upload: {
  target: 'temporary-public-storage', // For free tier
  // OR for persistent storage:
  // target: 'lhci',
  // serverBaseUrl: 'https://your-lhci-server.com',
  // token: process.env.LHCI_TOKEN,
}
```

## Best Practices

1. **Start with Loose Thresholds**: Begin with achievable thresholds (e.g., 70-80) and gradually increase them as performance improves.

2. **Test Multiple Routes**: Audit both your homepage and key user paths to get comprehensive coverage.

3. **Monitor Trends, Not Just Pass/Fail**: Set up dashboards to visualize performance over time using tools like:

   - Lighthouse CI dashboard
   - Custom metrics stored in your database
   - GitHub Actions workflow visualizations

4. **Use Performance Budgets**: Define maximum file sizes, JavaScript budgets, and resource counts:

   ```javascript
   'resource-summary': ['error', {
     'script': [{ budget: 170 }],
     'total': [{ budget: 300 }]
   }]
   ```

5. **Exclude External Dependencies**: Focus on your own code by excluding third-party scripts when possible:

   ```yaml
   settings:
     skipAudits: ['uses-rel-preconnect']
   ```

6. **Schedule Regular Full Audits**: Run comprehensive Lighthouse audits on a schedule to catch performance issues that PR-level audits might miss:

   ```yaml
   schedule:
     - cron: '0 0 * * 0' # Weekly on Sunday
   ```

7. **Communicate Results**: Post Lighthouse results as comments on pull requests for visibility:
   ```yaml
   - name: Comment PR with results
     uses: actions/github-script@v6
     with:
       script: |
         github.rest.issues.createComment({
           issue_number: context.issue.number,
           owner: context.repo.owner,
           repo: context.repo.repo,
           body: 'Performance Audit: [View results](https://your-storage.com/results)'
         })
   ```

## Conclusion

Implementing Lighthouse in your CI/CD pipeline transforms performance from an afterthought into a first-class requirement. While you'll encounter variability, timeouts, and baseline challenges, these are solvable with proper configuration and realistic thresholds.

The key is to start simple, automate early, and gradually increase strictness as your team's performance culture matures. By catching performance regressions before they reach production, you protect your users' experience and maintain the speed that drives engagement and conversions.

For more resources:

- [Lighthouse CI Official Docs](https://github.com/GoogleChrome/lighthouse-ci)
- [Web Vitals Guide](https://web.dev/vitals/)
- [Performance Budgets](https://www.smashingmagazine.com/2015/09/3-types-of-performance-budgets/)


Implement automated Lighthouse performance testing in your CI/CD pipeline to catch performance regressions early. Learn the problems, challenges, and solutions for integrating Lighthouse into GitHub Actions or other CI systems.

CI/CD

Lighthouse Pipeline - Automate Performance Testing in CI/CD

Subscribe for Updates