Lighthouse Pipeline - Automate Performance Testing in CI/CD
Table of Contents
- Introduction
- The Problem
- Why Lighthouse in CI/CD Matters
- Issues Encountered
- Solutions and Implementation
- Best Practices
- Conclusion
Introduction
Website performance is critical to user experience, SEO rankings, and conversion rates. However, performance regressions often go unnoticed until they reach production. This post covers how to implement Lighthouse performance testing as part of your CI/CD pipeline to catch performance issues before they impact your users.
The Problem
Performance regression detection is challenging because:
- Manual Testing is Inefficient: Running Lighthouse manually for every build wastes developer time and is easily forgotten.
- No Baseline Comparison: Without automated comparison, it's hard to know if a change degrades performance.
- Silent Performance Degradation: Small performance issues accumulate and go unnoticed until users complain.
- Inconsistent Environments: Local testing environments differ from production, making results unreliable.
- Lack of Visibility: Teams don't have a clear view of performance trends over time.
Why Lighthouse in CI/CD Matters
Integrating Lighthouse into your CI/CD pipeline provides:
- Automated Performance Checks: Every pull request gets a performance audit automatically.
- Baseline Comparisons: Detect when performance scores drop relative to the previous build.
- Fast Feedback Loop: Developers get immediate performance feedback while making changes.
- Enforced Standards: Set minimum performance thresholds that block merges if not met.
- Historical Tracking: Build a performance history to understand trends and improvements.
Issues Encountered
1. Lighthouse Variability and Flaky Results
Lighthouse results vary between runs due to:
- Network throttling differences
- CPU throttling inconsistencies
- Background processes on the CI runner
Solution: Run Lighthouse multiple times and average the results, or use median values. Set realistic thresholds (e.g., 85/100 instead of 95/100) to account for variability.
# Example: Run Lighthouse 3 times and average scores
- name: Run Lighthouse Audit
uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: '18'
- name: Install dependencies
run: npm install -g @lhci/cli@latest
- name: Run LHCI
run: lhci autorun
2. Lighthouse CLI Timeout Issues
Large pages or slow networks can cause Lighthouse to timeout during CI runs.
Solution:
- Increase the timeout setting in
lighthouserc.js:module.exports = { ci: { collect: { numRetries: 3, settings: { chromeFlags: ['--disable-dev-shm-usage'], // Use disk instead of shared memory onlyCategories: ['performance', 'accessibility'], // Only audit needed categories }, }, }, }; - Use
--disable-dev-shm-usageflag to prevent memory issues on CI runners with limited shared memory.
3. Authentication and Protected Pages
Lighthouse cannot audit pages behind authentication or paywalls.
Solution:
- Test public pages only
- Use a test account for authenticated pages:
settings: { settings: { formFactor: 'mobile', screenEmulation: { mobile: true, width: 412, height: 823, } } }
4. Comparison Baseline Not Available
New projects don't have a baseline to compare against initially.
Solution:
- Store initial results as a baseline in your repository
- Use Lighthouse CI storage API to store historical results
- Start with informational thresholds and enforce stricter ones over time
5. CI Runner Performance Constraints
CI runners may have different hardware than production, causing misleading results.
Solution:
- Use consistent CI runners (e.g., GitHub Actions on the same machine type)
- Normalize CPU and network throttling settings
- Focus on relative changes rather than absolute scores
Solutions and Implementation
Step 1: Install Lighthouse CI
npm install -g @lhci/cli@latest
Step 2: Create lighthouserc.js Configuration
// lighthouserc.js
module.exports = {
ci: {
collect: {
url: ['http://localhost:3000', 'http://localhost:3000/about'],
numberOfRuns: 3,
settings: {
chromeFlags: ['--disable-dev-shm-usage', '--no-sandbox'],
},
},
upload: {
target: 'temporary-public-storage',
},
assert: {
preset: 'lighthouse:recommended',
assertions: {
'categories:performance': ['error', { minScore: 0.85 }],
'categories:accessibility': ['error', { minScore: 0.9 }],
'categories:best-practices': ['error', { minScore: 0.85 }],
'categories:seo': ['error', { minScore: 0.9 }],
},
},
},
};
Step 3: GitHub Actions Workflow
# .github/workflows/lighthouse.yml
name: Lighthouse CI
on: [push, pull_request]
jobs:
lighthouse:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: '18'
- name: Install dependencies
run: npm ci
- name: Build application
run: npm run build
- name: Install Lighthouse CI
run: npm install -g @lhci/cli@latest
- name: Start local server
run: npm start &
env:
NODE_ENV: production
- name: Wait for server
run: sleep 5
- name: Run Lighthouse CI
run: lhci autorun
- name: Upload Lighthouse results
if: always()
uses: actions/upload-artifact@v3
with:
name: lighthouse-results
path: .lighthouseci/
Step 4: Store and Compare Results
Use Lighthouse CI's temporary public storage or set up persistent storage:
upload: {
target: 'temporary-public-storage', // For free tier
// OR for persistent storage:
// target: 'lhci',
// serverBaseUrl: 'https://your-lhci-server.com',
// token: process.env.LHCI_TOKEN,
}
Best Practices
-
Start with Loose Thresholds: Begin with achievable thresholds (e.g., 70-80) and gradually increase them as performance improves.
-
Test Multiple Routes: Audit both your homepage and key user paths to get comprehensive coverage.
-
Monitor Trends, Not Just Pass/Fail: Set up dashboards to visualize performance over time using tools like:
- Lighthouse CI dashboard
- Custom metrics stored in your database
- GitHub Actions workflow visualizations
-
Use Performance Budgets: Define maximum file sizes, JavaScript budgets, and resource counts:
'resource-summary': ['error', { 'script': [{ budget: 170 }], 'total': [{ budget: 300 }] }] -
Exclude External Dependencies: Focus on your own code by excluding third-party scripts when possible:
settings: skipAudits: ['uses-rel-preconnect'] -
Schedule Regular Full Audits: Run comprehensive Lighthouse audits on a schedule to catch performance issues that PR-level audits might miss:
schedule: - cron: '0 0 * * 0' # Weekly on Sunday -
Communicate Results: Post Lighthouse results as comments on pull requests for visibility:
- name: Comment PR with results uses: actions/github-script@v6 with: script: | github.rest.issues.createComment({ issue_number: context.issue.number, owner: context.repo.owner, repo: context.repo.repo, body: 'Performance Audit: [View results](https://your-storage.com/results)' })
Conclusion
Implementing Lighthouse in your CI/CD pipeline transforms performance from an afterthought into a first-class requirement. While you'll encounter variability, timeouts, and baseline challenges, these are solvable with proper configuration and realistic thresholds.
The key is to start simple, automate early, and gradually increase strictness as your team's performance culture matures. By catching performance regressions before they reach production, you protect your users' experience and maintain the speed that drives engagement and conversions.
For more resources: