Real User Monitoring has been around since a long time but it is changing rapidly these days. The oldest form of real user monitoring is log data analysis. Simply going through your weblogs and combining data into unique visitor sessions and then processing these for subsequent reporting. This used to work pretty well in the good old days when traffic was still very limited but even then the big drawback was that it all relies on overnight processing of your logs and because of this reporting was always a day behind reality.
Modern day RUM solutions no longer work this way. Where internal and external monitoring both use automatic probes (active monitoring), robots that execute pings, visit URL's or execute multi page scripts, RUM is using passive monitoring and tracking the activities of real users. This means that you cannot choose agent locations or have to specify measurement frequency or specify what pages you would like include. You simply see whatever pages your clients visit, what browser they use and where they are based.
This is great for understanding customer behavior and something that is perfectly suited to be combined with the information that you get from Web Analytics. This can help you to get a better understanding of why users tend to leave on certain pages and to see if bad performance is having an impact on your conversion rates.
Just to complicate things a bit more, Real User Monitoring comes in two flavors. The first uses a hardware device to analyze all traffic between your site and its visitors. This technology is also called sniffing and it doesn't impact the load on your system. The disadvantage is that this technology really sees everything that goes on between your site and the client including passwords, credit card information, anything. Of course there are ways to block this but it is still a good reason why many companies (financial institutions for example) are extremely reluctant to deploy this. At the same time this technology allows you to completely replay every visitors complete session which comes in handy if you need to provide first line support.
The downside of this is that RUM is not very well suited for technical monitoring. First of all, RUM will only provide you with measurement data IF there are active users on your site. If there aren't you will not receive any information. This means that the performance of your site can be degrading without you knowing anything about it. The second problem with RUM is that your measurement data simply isn't consistent which makes it much harder to identify any issues. Last but not least, Real User Monitoring tends to produce tons of data and that requires a lot of storage and serious processing capacity (plus advanced analysis tools) to translate data into actionable information.
Recently some vendors have started to integrate RUM technology into solutions that originally focused on internal monitoring of your infrastructure and applications or even with load testing. This is a very interesting trend as it makes it much easier to analyze the data and it allows for integrated alaming and alerting.
So, the good news is that you can have a perfect monitoring system but the bad news is that you need to implement all three types (internal, external and RUM) to get the complete picture. Key Performance has a lot of experience in integrating measurement data from muliple sources and in providing you with automated data correlation and consolidated reporting.
For those of you who still haven't had enough and want to know much more about web monitoring I would like to recommend the book Complete Web Monitoring, watching your visitors, performance, communities and competitors by Alistair Crow and Sean Powell.