A Universal Challenge
Having worked with clients for more than five years in the field of digital analytics, I have noticed that one of the major roadblocks towards wider acceptance and use of analytics data is the lingering doubt about the data’s accuracy. In reality, managers would accept the reports we churn out, however, we know, based on the decisions they are making, that they have not wholeheartedly accepted them. Let us face it, after all, why should the decision makers trust the data we are presenting to them? In all likelihood, the Web Analytics System (the people, platforms, and processes) exist and work as a separate unit, rather than as part of the data user’s team.
So what would be an appropriate response if someone did ask you this question directly: Why should I trust your data? Sometimes the reason why we are unable to answer this question with conviction is because we ourselves do not fully trust the data. The more time we spend looking at the data, the higher the likelihood that we ourselves may come across a data conundrum that we cannot fully explain. Trust in analytics data, be it from us or from the end-users, cannot be developed in an environment where people are not familiar with the Web Analytics System that lies underneath!
The Big Question
The question “why should I trust your data?” is really an expression of curiosity about what exactly are you doing to protect the integrity and quality of our Web Analytics data. Our response should start with an emphatic “Aha! I am glad you asked me that question…”, rather than a verbalization of defensive play. I agree that we have plenty of routine tasks to process that we have very little time and resources available to put together evidence that point towards a dependable Web Analytics System. This being the case, we must realize that all our efforts will not result in tangible actions as long as the trust issue is not addressed first.
Demonstrating Dependability
Activities that will contribute towards building trust will fall in the important category rather than the urgent category. Therefore, developing trust in the Web Analytics System is not going to be easy, but it will be worth the effort. Based on my experience of working with organizations of different sizes and across industries, here are some of the things that you may want to start doing to address the deficit of trust:
- Emphasize quality control – This should really be a no-brainer, but the reason I have included this as the first point is because I have seen this critical step being skipped only to result in the erosion of trust that users have in the data. Let us face it, adding a step to the report creation and release process adds to the overheads. However, delivering reports with errors that could have been identified and fixed at the QC stage will not boost the credibility of the Web Analytics System. All reports that are released by the analytics team should have gone through a quality control process. Giving users accurate data in simple reports goes a long way in building trust for the whole system.
- Align tool with business – We have seen time and again that the data collected and processed in the analytics tool is not in-sync with what the end-users want. One way to address this is to evangelize the importance of scoping the reporting requirements before changes are made to the site. To address changes that are necessitated due to the evolution in the business you can schedule “KPI Discovery Sessions” at least once a quarter. These meetings should be facilitated so as to become an open forum where decision makers can voice their priorities in terms of the metrics that are tracked in the tools.
- Implement processes – The challenge in dealing with web analytics data is that often there are multiple points at which data inaccuracies may creep in. Changes in the business priorities, the addition of new features or content to the site, and the launch of new marketing programs throw up potential opportunities for inaccurate data collection and processing. All of us know that it is practically impossible to go back and fix data inaccuracies in the web analytics tool. Hence, it is important to put processes in place that will ensure that only accurate data gets pushed into the tool. Often, this requires a review and approval mechanism. In other words, the Web Analytics System should operate in-line with a set of Data Governance principles.
- Maintain an audit calendar – All digital assets evolve over time and this change is the combined result of the processes followed by the different business and technical teams. Given this constantly changing nature of websites, we strongly recommend that audits be conducted as part of the routine operations rather than as a one-time activity. The good news is that there are plenty of tools available in the market (such as ObservePoint, Tracking First, and Nabler’s proprietary crawlers) that you can use to automate the audit process. What is required is for one of your team members to be nominated as the person responsible for maintaining the audit calendar, conducting the routine audits and presenting the findings.
- Evangelize and promote self-service – Working on a report delivery model that promotes self-service will give you a double advantage. One, it breaks down the barrier between the analytics tool and end-users which will eventually lead to wider acceptance and consequentially greater trust. The bigger advantage is that the self-service model will free-up resources that can be used to execute other more important tasks. Activities that will contribute towards building trust will fall in the important category rather than the urgent category, right?
- D for Data & D for Documentation – Many organizations approach us for conducting an audit of their web analytics data, and one of our first questions is “Do you have the Business Requirements Document (BRD) and the Solution Design Reference (SDR) of the implementation?” In most cases these documents are unavailable, and in the case where we get one of these documents, it is usually outdated. If there is one point that you must put in practice from this blog post, then let it be a resolve that you will maintain a document that is a true reflection of the current state of the analytics implementation. If you do not have the documentation in place (or if it is outdated), then please calendar some time to complete this task. Then put in place a process by which the document is systematically updated to reflect changes in the tracking tool. Ideally, all changes should be documented first (which can be used as part of the approval workflow as per point 3 above) and then implemented in the tool.
- Collect feedback and change – Finally, no system is perfect and, therefore, we must go out and speak to everyone who uses our data about its usefulness. You can do this by scheduling a quarterly review of the analytics processes with all key stakeholders. You may want to set-up a simple online survey (use SurveyMonkey or Google Docs) and embed the link in the email sent with the reports. Whether you gather the feedback formally or informally, what is important is that you fine-tune your processes to serve your stakeholders to the best extent possible. It’s true that no Web Analytics System will ever be perfect, but why should it stop us from aiming for excellence?
Conclusion Every question is an opportunity! Questions from end-users that pertain to your Web Analytics System are an open invitation to explain what exactly you are doing to protect the integrity and quality of the Web Analytics data. I hope the pointers in this blog post have given you some starting points on what can be done within your organization so that you are better equipped to answer the question. Nabler Web Analytics Consultants have worked with multiple clients developing systems and process that make their web analytics data accurate and dependable. If you have any additional questions or a problem that needs to be addressed, then you can always reach out to one of Nabler’s consultants.