In the last few months since we launched Torbit Insight, hundreds of top retailers and large media properties have adopted Real User Measurement on their sites. In fact, we’ve measured over 3 billion page views for retailers like Wayfair, CafePress and Build.com. As we’ve had the privilege of working with some of the largest sites around, we’ve noticed an ongoing trend. Our customers are starting to depend on the Real User Measurement (RUM) data we give them as their primary source for monitoring their website performance.
Performance measurement has traditionally been done using synthetic testing (sometimes also referred to as active monitoring). Synthetic testing is when you load a website on a regular interval from one or more locations around the world to see how fast it loads. This data is then used to generate reports or trigger alarms when there are performance issues with your site. While synthetic testing is certainly useful, hundreds of top sites are turning to Real User Measurement as a source of more accurate data.
While Synthetic testing is valuable for deep analysis and debugging, it has a few short comings. With synthetic testing you only get visibility into the specific pages that you test. This is typically a small fraction of the pages your customers actually visit, leaving you with large sections of your site without monitoring. Synthetic testing gets expensive, especially if you try and increase your coverage to more pages across your site. You’re also putting more stress on your servers, taking valuable capacity away from your actual visitors. Of course, the main problem with synthetic testing is that it makes so many assumptions about your visitors. There are dozens of factors that affect the speed at which someone is able to access your site. Where are they geographically located? What is their connection speed? Which browser are they using? Are they visiting for the first time, or are they a repeat visitor? All of these variables affect the loading experience for your visitors. If you want to know what your visitors are actually experiencing, you have to use Real User Measurement.
It’s impossible to test every variation of location, network connection speed, OS, browser & add-on. That’s not to say synthetic testing is bad. There’s a place for both.
There are a few key factors that are accelerating the adoption of Real User Measurement. We now have the web timing spec support in all of the major browsers. This allows us to collect highly accurate timing data from the browser itself, starting even before the page is loaded. This allows us to time things like DNS lookups and the time it takes to do the TCP handshake. One of the challenges of implementing RUM in the past has been simply the massive amount of data that it generates. With the explosion of “big data” tools, it’s now feasible to collect billions of samples and make sense of them. Thankfully, you don’t have to build it yourself, we offer a great Real User Measurement tool at Torbit and we even made it free for people to get started.
Every visitor matters. If your site is slow, chances are you are leaving visitors and revenue on the table. The first step in making your site faster is making sure you have an accurate way to measure your speed.