Product ROI, the return on investment calculation for work done in software engineering, is the subject of many coffee conversations I’ve had. One former CTO called it the “holy grail” of product / engineering management. Rich Mironov’s recent article on the allure of Product ROI highlights its lure and its pitfalls. Inspired by this, though, I started considering what it would take to make ROI calculations truly robust—and whether the investment is justified.

When I previously wrote about how product allocations should be done for financial planning, I alluded to the futility of making business cases for individual features. It doesn’t make practical sense in most of the cases I’ve seen. Of course, this is based on my experience. Putting my two decades of planning and investment experience aside, I want to consider what would it take to make a solid ROI calculation? What would the apparatus around it need to look like? That led me to this thought experiment:

Let’s start with the assumption that we are looking at ROI for planning purposes: ROI after the feature / product is built, is a much easier calculation. So, for this discussion, let’s assume everything is based on projections — a revenue projection (return) and a cost projection (investment). For this exercise, I’m assuming a B2B software product.

The Revenue Projection

The more nebulous part of the equation is always the benefit – the revenue portion. And, as such, it will require the most consideration. For robustness, let’s require looking at everything from pricing, to engagement, to the revenue outcomes.

The Perception of Value

Even for B2B, emotions and perception of value play a role in the revenue. In fact, I would start there — the perception (feeling) of value that the customer has. To get that, there would need to be consistent customer pricing research about usage, alternatives, considerations, and success criteria. This would roll into the more practical consideration of Economic Value Estimation to determine what the customer is getting and part-weight considerations of what is most valuable about the product — think conjoint analysis, pairwise or maxdiff comparisons. Maybe break out a Westendorp Price-Sensitivity study if you are feeling it.

The Analytics of Engagement

With the data in hand about what a customer _believes_ to be valuable, there would also need to be ongoing analysis about what the customer actually does on the platform. This would be satisfied by well functioning analytics that track usage, engagement, etc. We’d want to quickly analyze a segment about everything in the platform that is touched and by whom. It should also cover things like user pathing, feature discoverability, and the occasional in-product survey about the response of a feature or interaction.

The Value of the Account

With both the perception of value and the engagement fully understood, we’d want to add full account mapping for all the users–data matching the research, the analytics, and the CRM. Full account mapping lets us know the roles of each person at the company. With the previous analyses, we can understand what each role values and which role is doing what on a platform. Additionally, with contract amounts (because not every company is paying the same) layered on top, we can theorize the relative importance of everything, its value derived from the customers willingness and ability to pay for it. Finally, a proper functioning win / loss analysis should give us some indication whether all the former analyses are actually working correctly.

With all that working well, you have a foundation upon which to build revenue projections — specifically interrogating things to a feature level. 

The Cost Projection

The other side of this equation can also be tricky. For each new effort, what would it take to get it done? And, of course, the requirements will be less than what’s necessary to estimate, but estimates will still be necessary for the ROI calculation. To get a more robust estimate for this exercise, I suggest a comparison of three activities: the team that will perform the task, an expert architect who knows the code and the complexity, and an operations analyst doing historical comparisons. 

Team Estimation: Standard Practice

The standard practice is to have the team do the estimation – think planning poker. And, even when doing this at small, agile-friendly increments, these estimates can go wildly off. And, of course, the revenue projection will demand completeness, so timelines will be longer; estimates will be bigger and more complex; and, therefore, more inaccurate.

An Architect: Expert Opinion

With the team already doing their estimate, let’s assume there is a full-time architect who can do the estimates for them as well. This expert would have to know how the systems work and how the implementation might need to happen. They would review technical designs, consider the underlying code, and with their knowledge of what it would take, come up with their own estimate.

An Analyst: Historic Trends

In addition to the expert estimate, there could also be an operations analyst looking at the team’s historic performance — what has their velocity been in the past when dealing with similar projects? How has the team changed? And based on actual performance and a set of variables, provide another estimate.

With all three of these estimates, there would likely be a low estimate (perhaps the architect who thinks things are easy) and a high estimate (the operations analyst who has the benefit of data). The team’s estimate could adjust the high estimate up or the low estimate down, but it would not adjust both. And so, we now have a ranged estimate that can be used on the cost side of our equation.

The Cost of ROI

The challenge is that you would need to be able to pull together all this analysis quickly and efficiently as ideas arise – in days, not weeks or months. To make this happen on an ongoing basis — as planning is ongoing and new ideas are considered — this entire apparatus would have to be stood up as a function unto itself. Meaning, there would be a team (let’s call it the Central Planning Committee, for fun), and they would take any spec, feature, product idea, and run the analysis on it. The labor involved in this is specialized, and what’s more, should be focused on accuracy. Product managers are too often judged on whether they can get agreement on estimates vs accuracy of those estimates.

Based on the processes outlined above, I would estimate the following:

• Two researchers running the pricing / value analyses, ongoing
• An analytics engineer would pull together the relevant analytics
• The team would put forward their estimate, but so would the architect
• The estimate would be reviewed by the operations analyst as well.
• A financial analyst would put together the assumptions and projections on both the revenue and the cost side to get a range of possible return scenarios.

That sounds great, until you realize that I just added ~6 people and somewhere between $1m – $2m of operating costs to your payroll. (Let’s assume the team is already doing estimates, so no added cost there.)

And, here’s the real question — Is that worth it? Is defensible ROI worth $1m – $2m a year? Is accuracy that important? And, is it even achievable? Think about how often an M&A projection goes awry.

In the future, I think this process could be streamlined with better analytics and AI agents that can do some of the analyses on existing data sets, but there is no way to satisfy the demands of “defensible ROI” without having some dedicated team to understand how to shape the analysis. It might not need to be this big, but there would need to be some specialization and dedicated people for the analysis. And, while this is likely overkill, I think there’s a “good enough” product ROI discussion somewhere underneath this. But, that would be a different post.

Note: Thanks to Mike Sands for discussing some of the early thinking on this and Todd Vaccaro for giving me some ideas on the post.