Deriving Insights From Feedback Pt 2

I’m pleased to share the second post in this series on deriving insights from feedback. This one was also co-written with Dan Heilbron, Head of Customer Experience Advisory Services, Asia-Pacific and Japan at Qualtrics.

In our last post, we took you through a great tool to determine if two variables are related: correlation analysis. In this post, we’re going to take things a step further by going through regression analysis.

When talking about a Voice of the Customer program, action is key. If you’re not doing something with the feedback customers give you, you’re wasting your customers’ time asking them for feedback in the first place. (NB: Simply reporting a Net Promoter Score on a dashboard does not count as taking action!)

So you’ve listened to customers and now you have a range of CX improvement initiatives devised. But which ones should you prioritise? Get it wrong and you’ve just wasted a bunch of time and money. It’s a high-risk decision!

It’s critical that you, as a CX leader within your organisation, focus on improving what matters most to customers (ie the key drivers of their experience with you), given the finite resources you have.

Enter regression analysis.

Without getting too tech-y, it’s a statistical measure that represents the proportion of the variance in a dependent variable that is explained by the independent variable(s) in a regression model. It is expressed through an R-squared (R²) value.

R² values range from 0 to 1, with higher values indicating a better fit of the model to the data. It is important to note that an R² value only measures the goodness-of-fit of a regression model and does not provide information about causality or the direction of the relationship between the variables.

So let’s illustrate how regression analysis can be used with an example that many CX practitioners will find familiar.

Let’s say you’re the CX Manager of a hotel, let’s call it The Underwood, and you want to understand the drivers of customer satisfaction within the check-in experience. You ask customers the following questions via a survey, following their check-in:

• Overall, on a scale of 1-5 with 1 being very bad and 5 being very good, how did you rate your check-in experience at The Underwood on [insert customer check-in date]?
• Overall, on a scale of 1-5 with 1 being very bad and 5 being very good, rate your experience with the following aspects of your check-in experience:
  1. 1. Check-in Speed
  2. 2. Ease of Finding Check-In desk
  3. 3. Employee Friendliness
  4. 4. Bellhop Support

You collect your data, run the analysis (there’s a range of tools that allow you to do this including Excel and Qualtrics’ StatsIQ tool) and get the following outputs:

So what does this mean?

Coefficients and p-values in regression analysis work together to tell you which relationships in your model are statistically significant and the nature of those relationships. The Intercept value of 0.544 indicates that ~54% of the dependant variable score can be explained by the independent variables assessed i.e. overall, the aspects of the check-in experience we’re asking about are important to customers when rating their overall check-in experience.

As for the independent variables (i.e. the key drivers), the employee friendliness variable has a positive coefficient of 0.625 with a small p-value (<0.05), indicating that it is a significant predictor of a customer’s overall rating for the hotel’s check-in experience. Specifically, every 1 percentage point increase in how much customers value employee friendliness will increase the overall check-in rating by 0.625 percentage points all else being equal.

The check-in speed, ease of finding check-in, and bellhop support variables do not appear to be significant predictors of overall rating, as their coefficients are close to zero and their p-values are not statistically significant. With p-values, to say with confidence that the result can be trusted, they need to be lower than 0.05.

Based on this, when trying to create a great check-in experience, the hotel CX manager will want to focus the training for front of house staff on friendliness, potentially de-prioritising those other aspects of the experience because they don’t mean as much to customers.

One “gotcha” to call out with this types of analysis is “multicollinearity”. This, in simple terms, means that when similar key driver questions are asked, the scores from these drivers are potentially eroding the coefficients of each other. In this case, check-in speed could itself be impacted by the ease of finding the check-in desk in the first place. If you sense multicollinearity may be impacting your results, consider removing some independent variables from your data set and then re-run your analysis. It could also be a sign your survey needs refreshing!

Regression analysis is a powerful tool to identify what matters most to customers. Using it enables CX Managers to prioritise their CX improvement initiatives to ensure that their CX programs are efficiently hitting the bullseye with customers every time. If you’ve got a statistical analysis tool you’d like us to cover in our next post, let us know in Comments and we’ll do our best to oblige.

About My Co-Author

Dan Heilbron, CCXP leads Qualtrics‘ APJ CX advisory team and is based in Melbourne, Australia. Dan spends his time helping clients across the APJ region to develop, advance and maximise the value of their XM program. Prior to joining Qualtrics, Dan worked with other large XM SaaS organisations in Australia, the UK and the USA and has provided XM advisory services including journey mapping, strategy, survey, dashboard and closed loop program design, and insights and ROI development and presentations for Fortune 100 and exchange listed clients, government and not-for-profit entities across 5 continents, in industries such as sport and entertainment, automotive, airlines, hospitality, retail, B2B, financial services and healthcare.