Impact Metrics and ROI

Often, we use the terms "Impact metric" and "ROI" (Return on Investment) interchangeably. They both show value, and they answer different questions. Impact metrics show what changed as a result of an initiative, while ROI shows the financial return relative to the cost. Does this difference matter? Well, it depends on who you are talking to. If you are talking to an executive leader, CFO, CRO, or an investor, it really matters. I know this firsthand, as I have made the mistake of saying "ROI" when I was talking about impact in a meeting with executive leaders, and it was immediately pointed out publicly that I didn’t know what I was talking about… Lesson Learned!!

Let’s start with impact metrics first. Impact metrics focus on the outcomes and tell you if the program is making a difference. We also use the term Outcome metrics to describe these.  Impact metrics also provide operational insight to improve programs. Why track impact metrics?

•  They show what’s actually working

•  They measure a specific customer/company problem

• They show leading indicators of business results

• They are easier to calculate for most teams

 There are 6 types of impact metrics that I track and show correlations with business metrics. Each of these helps show the impact of our programs.

•  Engagement metrics - which are users engaging. This helps us understand which customers are involved with our programs. Some questions we can answer with this type of metric are whether completion matters or if it is just engagement with certain content. How does this engagement affect other metrics? Do engaged customers drive higher revenue and/or decrease in support tickets? Which content is resonating or driving which behaviors?

• Penetration metrics - who is engaging. This helps us understand the breadth and the depth of customer engagement. We evaluate the total number of users, the total number of companies, and the number of users at each company. Some of the questions we can answer. Does the number of engaged users at a company affect other metrics? Which companies are not engaging, and what is this signaling?

• Adoption metrics - what are they using. This helps us understand if we are driving behaviors. We evaluate this at the user and company levels. Does the increase in certain usage patterns affect other metrics such as churn and revenue growth?

•  Efficiency metrics – how fast do users find value. This helps us understand if we are saving time for customers. Some of the questions that we can answer are: Do engaged customers implement faster? Do engaged customers submit fewer support tickets and/or resolve them faster? Do engaged customers find more value from our product?

•  Satisfaction metrics- what do they like. This helps us understand customer sentiment. We evaluate engagement CSAT, company CSAT, and NPS. Some questions we can answer are: Do companies with higher CSAT have a lower churn rate? Do customers with a higher CSAT buy more product?

• Revenue metrics- what do they pay for. This helps us understand our revenue-generating programs. We evaluate our contributions to the annual revenue. Some of the questions we can answer are how much revenue do we generate? How many do we give away as investment dollars? Do some programs sell better than others? Do customers who pay for programs buy more product or use more product?

We usually correlate impact metrics with business metrics such as ARR, churn, product usage, and support tickets, and evaluate by comparing pre- and post-engagement or engaged vs. unengaged. For revenue-generating programs, we show the revenue. We routinely show positive impact. Impact metrics are valuable on their own. They also play an important part in calculating the ROI.

ROI is the language investors and executives use as they fund initiatives.

Calculating the ROI sounds simple. What did you pay for it, and how much did you make from it? This sounds simple, but it is often much harder to do and requires cross-functional work. ROI is a simple formula
ROI = (Gain from program – Cost of program) / Cost of program) *100

The numbers that go into it are not as simple. We need to define what the gain is, and then we need to understand the real cost of the program.

Program Costs

Understanding program costs is where many CE leaders often make mistakes. The program costs include software and operational (OpEx) costs.

Software costs are the easy part. Add up the cost of all software specific to your team or program. Many programs have an LMS, authoring tools, a certification platform, proctoring, a badging platform, a lab platform, and delivery tools.

Most leaders cannot calculate the OpEx on their own. Usually, they will need the help of the finance team.. Each company I have seen has grouped these slightly differently but they are all included somewhere. The most common Opex groups:

•  Core Opex, which is the fully loaded salaries including base salaries, bonus, variable compensation, taxes, and benefits. A team of 5 with salaries ranging from 100k-180k often has a core Opex of around 900k

• Headcount overhead, which includes expenses such as facilities and office space, HR services, finance and accounting, legal, and IT. Many companies assign a per-employee cost per year. A rough estimate for a team of 5 would be 100K

•  Department operating budget, which includes travel and events, marketing, professional development, and awards. This varies greatly by company.

•  Shared systems, which includes CRM, analytics platform, communication tools, and productivity software. This varies greatly by company.

At some companies, the Finance team may not be willing to share this. You can either use rough estimates or work with the finance team and give them your numbers. Even with access to all of this, I prefer to work with my finance team to validate my data.

Example for a team of 5

Program Gains

The gain is the financial value created by the program. There are 5 types of gain that we can evaluate.

Direct revenue. If the program generates direct revenue, the calculation is simple. It is the generated revenue minus the program costs. Let’s assume a revenue of $1,500,000, and use the number above

ROI = ((revenue - program costs)/program costs) x 100

(1,500,000 -1,200,000) / 1,200,000 x 100= 25% ROI

Indirect revenue.  Evaluating the revenue of engaged versus non-engaged customers, you find that engaged customers grew revenue by $400,000 over non-engaged customers.

So, the indirect revenue is $400,000

Reduced churn: Evaluating churn shows that customers who engage with training have a 2% churn rate, while those who do not have a 9% churn rate. If the average annual revenue per customer is $50,000, and training helps retain 12 additional customers, the revenue protected by the program is:

14 customers × $50,000 = $700,000

So, the reduced churn gain is $700,000.

Reduced support tickets: Evaluating your support tickets shows that engaged customers submit 50% less tickets than those who are not engaged. This year, it amounted to 4,000 fewer support tickets. Assuming the cost of handling the ticket is $25 per ticket

4,000 × $25 = $100,000

So the support cost reduction gain is $100,000.

Faster time to value. Evaluating the time to implement shows that trained customers implemented 30 days faster. When customers reach value sooner, they are more likely to expand usage and renew.

This results in $500,000 in earlier expansion revenue

So the faster time-to-value gain is $500,000.

ROI = ((revenue - program costs)/program costs) x 100

(3,200,000-1,200,000) / 1,200,000 x 100= 167% ROI

Impact metrics and ROI are different things that work best together. Impact metrics help you understand what is changing because of your programs and where to improve. They provide the operational insight you need to run and optimize a Customer Education program. ROI translates that impact into financial terms that executives and investors understand.

When you combine the two, you move from simply showing activity to clearly demonstrating business value. Impact metrics explain why the program works, while ROI shows what that impact is worth to the business.

The goal should not be to replace impact metrics with ROI. The goal is to use impact metrics to build the evidence and correlations that ultimately support a credible ROI story. When done well, you can confidently and credibly articulate your program's measurable outcomes that matter to the business.