SiteLint vs Lighthouse comparison

    Establishing an excellent website or application is just the beginning and as technology evolves by the hour, it becomes difficult to keep the quality high to stay relevant in an ever-changing market. No matter what type of business you own, you will need software that is fully accessible, usable, fast, and without issues, as it has a high impact on the way customers reach out to you and buy your products or services.

    Companies and their teams begin to research tools they can use to perform audits and track their metrics to improve the quality and user experience of their websites and applications. Google Lighthouse becomes a free and outstanding tool on the market and that raises the question: We use Google Lighthouse for one-off snapshots of my site’s performance, accessibility, SEO, best practices. Why do we need other solutions?

    However, looking from the business perspective you may ask a better question: When is a free tool like Google Lighthouse adequate for my business needs? Does it cover areas when the business operates?

    If the business relies only on a free tool, like Lighthouse, it is inevitable that it will reveal some of the tool’s limitations. It is important to look at it more closely as not all perspectives are visible at the first look.

    Why Google Lighthouse?

    Lighthouse is an open-source, automated tool for improving the quality of web pages. You can also run Lighthouse against any page that’s public or which requires authentication. Note that Lighthouse doesn’t really go through authentication, but rather allows you to perform scanning because of WebDev or Chrome extension.

    Lighthouse allows you to audit a page with the following audits:

    • Performance
    • Progressive Web Apps
    • Best Practices
    • Accessibility
    • SEO

    After Lighthouse has run these audits against your chosen page it will then generate a report for you. The official Google description also states that Lighthouse can be run in Chrome DevTools, from a Chrome extension, from the PageSpeed Insights website, the command line, or as a NodeJS module. Lighthouse is built by Google and when your site is being audited with what Google considers to be best practice, then the results can help you understand how Google sees your site. Lighthouse does not know the purpose of your page. It does not know how important the page is or how it will be used. Because it does not know these aspects, Lighthouse must make assumptions about what constitutes good or bad. This can lead to scores that are not representative of a real user’s experience.

    If you only need to perform site audits occasionally, or only on some selected pages then a free tool like Lighthouse might be perfectly suitable for you. However, as the business grows, more aspects come to the game. What works in the laboratory always works differently on the user side. Users come to your site from around the world, and it’s essential to understand how it works and how it’s being used. That raises some questions:

    • How do we track quality based on real user usage?
    • How do we manage fixes when a free tool doesn’t provide real support and training for the engineering team?
    • How do we track the progress of fixes implementation over time?
    • Are results reflecting the user or lab environment?
    • Once the bug is found in the Audit itself, how long does it take to fix it when it’s reported?
    • Can we run audits on the user side? To measure quality on the user side.
    • Can we run on the site with sensitive data?
    • Do we have access to other tools that help in fixing issues? E.g. Color Contrast Finder or image optimization.

    Short summary

    Let’s have a look at the short summary:

    Short summary of SiteLint and Lighthouse features
    FeatureSiteLintLighthouse
    Chrome extensionNo1Yes
    NodeJS supportYesYes
    CLI supportNo1Yes
    Accessibility AuditYesYes2
    Performance AuditYesYes
    SEO AuditYesYes
    Progressive Web AppNo1Yes
    Best practices3YesYes
    Privacy AuditYesNo
    Quality Audit4YesNo

    Notes

    1. On the Roadmap.
    2. Only small subset (25) of rules are exposed from 3rd party tool.
    3. There are differences in what type of rules are executed.
    4. Quality Audit refers to the rules that identify problems that undermine the quality of your site, e.g. too big images compared to what’s rendered.

    What makes different in detail?

    Performing Audits: client-side vs server-side
    Audits are performed when visitors arrive on your site on any page.
    LighthouseAudits are performed on demand and on Google’s server side using only Chrome browser.
    Data Analytics over time
    All audits data are stored in selected period time so you can see how they were changing on the timeline as well as compare results for given days.
    LighthouseThere is no way to get the scan history. The data are delivered at the time a single page is scanned.
    Continues monitoring
    Audits are running 24/7/365 because they are being executed on the client side. No need to set up intervals or specific time to audit the site.
    LighthouseNot available.
    Single page vs multiple pages
    Audits are running every time your user visits the page. That means you always get results from the pages that are visited by real users.
    LighthouseOnly a single page can be scanned unless Lighthouse is part of automated testing. However, this still doesn’t reflect the real user environment.
    Gathering data on a device representative of your audience
    All results from audits are coming from a real user environment: browser, network and devices.
    LighthouseAudits are running with one of the predefined settings: for mobile or desktop and Simulated Slow 4G, 4x CPU Slowdown or Applied Slow 4G, 4x CPU Slowdown or No Throttling. This doesn’t reflect a real user environment.
    Single Chrome instance vs Cross browsers / OS / devices / networks
    Audits are performed on the client-side which reflects all possible combinations of browser type, version, settings, OS, devices, networks.
    LighthouseLighthouse uses the same Chrome version and settings. However, you can simulate a few other user scenarios and extrapolate results.
    Audit Rules
    • Our Audits don’t depend on any 3rd party tools.
    • SiteLint have > 120 rules and it keeps growing based on #1 results from real reports, #2 innovations, #3 users feedback.
    Lighthouse
    • Lighthouse uses 3rd party tools to perform some audits like Accessibility.
    • For Accessibility only a small subset of rules are used and it’s limited to the Level AA.
    • Estimated Savings aren’t accurate because those are only mathematical calculations.
    • There are only 14 rules for SEO.
    • Runtime issues aren’t tracked.
    Manual testing services
    Some Audits require manual testing, e.g. Accessibility due to technical limitations. SiteLint provides manual testing services so you can get a fully compliant site with required standard WCAG 2.1 Level AA or go beyond.
    LighthouseManual testing services aren’t available.
    Ability to see report with other pages affected by the same issue
    The same issue may appear on more than one page. SiteLint will give you a report which pages are also affected.
    LighthouseThe feature is not available.
    Optimized remediations process
    There is always a question of where we should start fixing issues. SiteLint gives a report where you can see pages and issues sorted by top errors and sessions. That way you can start fixing critical issues on the pages mostly visited by the users. The improvements will be significant and visible fast.
    LighthouseThe feature is not available.
    Single Page Applications support
    SiteLint works with Single Page Applications and monitors changes live.
    LighthouseIt doesn’t support Single Page Application. Although, Lighthouse can be injected on the page as is and does only a one-time audit. Any changes on the page later won’t be caught.
    Auditing pages with sensitive data
    SiteLint has an option to remove all sensitive data: texts, data from form controls, emails.
    LighthouseThere is no option to remove sensitive data while scanning the page.
    Measuring performance beyond initial page loading only
    SiteLint measures performance not only for initial page loading, but also while the app / website is running.
    LighthouseThere is no way to measure page performance over time. Only initial loading page.
    Measuring Core Web Vitals
    SiteLint measures Core Web Vitals for each page and whole site based on inputs from all users and calculates median.
    LighthouseCore Web Vitals are measured and calculated for a single page only as it can be used from DevTools, Chrome extension or NodeJS.
    Managing false positives
    SiteLint gives an ability to manage false positives directly in the code so you can tell which rules should be skipped for particular elements or components.
    LighthouseThe feature is not available.
    Partial content scanning
    SiteLint uses an Auditor (part of SiteLint platform) that gives an ability to scan any given fragment of the page. This feature will be exposed to the Platform in the incoming future.
    LighthouseThe feature is not available.

    The future

    As the browsers give more and more features SiteLint is going to measure and track more areas. For example:

    • Tracking Content Security Policy violation.
    • Feature detection like Payments API, Credentials API, Shape Detection API, so you can better understand what’s available and used.
    • Track the usage of the features for better understanding how users are using the site, e.g. filling the forms or handling errors.
    • Determine Native Behaviors: Local Notifications, Push Messages, User Idle Detection, Permissions, Task Scheduling.
    • Spell Checking.
    • Integrations: WordPress, JIRA (in a very unique way), and more.

    Summary

    Lighthouse provides many benefits. However, the best way to know which pages require improvements is to monitor your actual users’ usage. This is where SiteLint comes and gives the ability to monitor the site quality from multiple aspects of your live production users in real-time as well as in the development or pre-production environment. It tells you what’s actually happening for actual users.

    We value your comments and would be most grateful if you could let us know what you think. Please, feel free to suggest new features or correct possible errors. Contact us today.