Krishna Kumar
@krishnablue.com
The clusters have gotten more baroque as their scope and influence has grown, but the reports should largely be treated as vendor marketing at this point imo.
Whatever pretensions it has for scientific rigor is long gone.
Whatever pretensions it has for scientific rigor is long gone.
October 26, 2025 at 12:35 PM
The clusters have gotten more baroque as their scope and influence has grown, but the reports should largely be treated as vendor marketing at this point imo.
Whatever pretensions it has for scientific rigor is long gone.
Whatever pretensions it has for scientific rigor is long gone.
The DORA reports have more or less always worked backwards from the story they are trying to tell. In the beginning they were trying to “validate” the benefits of continuous delivery, and they introduced the four terribly named “performance” clusters to make their case.
October 26, 2025 at 12:35 PM
The DORA reports have more or less always worked backwards from the story they are trying to tell. In the beginning they were trying to “validate” the benefits of continuous delivery, and they introduced the four terribly named “performance” clusters to make their case.
Means you have to much more modest about the scope of what you claim you are measuring.
Which is fine, if improving is the goal.
Which is fine, if improving is the goal.
October 19, 2025 at 11:23 AM
Means you have to much more modest about the scope of what you claim you are measuring.
Which is fine, if improving is the goal.
Which is fine, if improving is the goal.
The good about building measurement systems that have provable mathematical properties is that you are no longer constrained by the imaginations of thought-leaders.
Anyone should be able to apply the same principle consistently and get predictable results.
Anyone should be able to apply the same principle consistently and get predictable results.
October 19, 2025 at 11:23 AM
The good about building measurement systems that have provable mathematical properties is that you are no longer constrained by the imaginations of thought-leaders.
Anyone should be able to apply the same principle consistently and get predictable results.
Anyone should be able to apply the same principle consistently and get predictable results.
The mathematically grounded techniques I am developing at that blog aim to solve these issues.
I’ll be releasing MIT licensed libraries that anyone can use to validate the concepts on their own data in the next month.
I’ll be releasing MIT licensed libraries that anyone can use to validate the concepts on their own data in the next month.
October 19, 2025 at 11:23 AM
The mathematically grounded techniques I am developing at that blog aim to solve these issues.
I’ll be releasing MIT licensed libraries that anyone can use to validate the concepts on their own data in the next month.
I’ll be releasing MIT licensed libraries that anyone can use to validate the concepts on their own data in the next month.
The honest answer to many questions we are asked is “we don’t know.” But that’s not acceptable. Until we get better about that we’ll never find better solutions.
October 18, 2025 at 9:44 PM
The honest answer to many questions we are asked is “we don’t know.” But that’s not acceptable. Until we get better about that we’ll never find better solutions.
There is also the tendency to ignore correctness and rigor in favor of ease of selling an idea to a wide audience.
Without rigor this stuff quickly turns into empty slogans.
Without rigor this stuff quickly turns into empty slogans.
October 18, 2025 at 9:41 PM
There is also the tendency to ignore correctness and rigor in favor of ease of selling an idea to a wide audience.
Without rigor this stuff quickly turns into empty slogans.
Without rigor this stuff quickly turns into empty slogans.
The list of technical problems with the way we do “metrics” is very large..
October 18, 2025 at 9:39 PM
The list of technical problems with the way we do “metrics” is very large..
Then there is whole question of precise definitions of metrics. Etc
October 18, 2025 at 9:39 PM
Then there is whole question of precise definitions of metrics. Etc
rather than the way we do it right now. Pull “industry standard” metrics off the shelf without understanding the context.
October 18, 2025 at 9:39 PM
rather than the way we do it right now. Pull “industry standard” metrics off the shelf without understanding the context.
Yup. Lots of work to do on the fundamentals of measurement and systems analysis. The relevant metrics are always highly context specific and fall out of the analysis of a specific improvement we are trying to make.
October 18, 2025 at 9:39 PM
Yup. Lots of work to do on the fundamentals of measurement and systems analysis. The relevant metrics are always highly context specific and fall out of the analysis of a specific improvement we are trying to make.
If you want to get an idea of what that kind of approach looks like this is an example:
open.substack.com/pub/thepolar...
open.substack.com/pub/thepolar...
How Flows Stabilize
A Tale of Flows - Part 1
open.substack.com
October 18, 2025 at 6:18 PM
If you want to get an idea of what that kind of approach looks like this is an example:
open.substack.com/pub/thepolar...
open.substack.com/pub/thepolar...
But I’m definitely saying “don’t do that” when it comes to a lot of very mainstream ideas in how we measure things and talk about what those measurements mean.
October 18, 2025 at 6:18 PM
But I’m definitely saying “don’t do that” when it comes to a lot of very mainstream ideas in how we measure things and talk about what those measurements mean.
I probably should write it up more coherently on my blog. My approach to measurement is so far from the mainstream right now and I’m definitely not saying “do this not that” except in very circumscribed contexts.
October 18, 2025 at 6:18 PM
I probably should write it up more coherently on my blog. My approach to measurement is so far from the mainstream right now and I’m definitely not saying “do this not that” except in very circumscribed contexts.
The lack of discipline and rigor in this area keeps us from looking for better solutions and instead focus on marketing what is a fundamentally broken approach to measurement.
October 18, 2025 at 6:03 PM
The lack of discipline and rigor in this area keeps us from looking for better solutions and instead focus on marketing what is a fundamentally broken approach to measurement.
So yes, until I can see a more convincing argument about how your are establishing cause and effect in your claims about software delivery performance and things like market share and market cap, I’ll probably just keep saying “hand waving”
October 18, 2025 at 6:03 PM
So yes, until I can see a more convincing argument about how your are establishing cause and effect in your claims about software delivery performance and things like market share and market cap, I’ll probably just keep saying “hand waving”
We’ll only make progress in our industry once we get past this habit of making poorly substantiated claims with weak allusive arguments about being “data driven”.
October 18, 2025 at 6:03 PM
We’ll only make progress in our industry once we get past this habit of making poorly substantiated claims with weak allusive arguments about being “data driven”.
Steve I am arguing for more modesty in what we claim we can do with the techniques we have and the how we measure them and what the measurements say about impact etc.
October 18, 2025 at 6:03 PM
Steve I am arguing for more modesty in what we claim we can do with the techniques we have and the how we measure them and what the measurements say about impact etc.
“Connection” - again hand waving.
None of it is systematic or rigorous.
It’s not that I don’t believe the techniques are sound.
I do believe our techniques to reason about it with it with data are mostly BS.
It’s the measurement philosophy that is broken.
None of it is systematic or rigorous.
It’s not that I don’t believe the techniques are sound.
I do believe our techniques to reason about it with it with data are mostly BS.
It’s the measurement philosophy that is broken.
October 18, 2025 at 5:15 PM
“Connection” - again hand waving.
None of it is systematic or rigorous.
It’s not that I don’t believe the techniques are sound.
I do believe our techniques to reason about it with it with data are mostly BS.
It’s the measurement philosophy that is broken.
None of it is systematic or rigorous.
It’s not that I don’t believe the techniques are sound.
I do believe our techniques to reason about it with it with data are mostly BS.
It’s the measurement philosophy that is broken.
I’m arguing that people criticizing claims of AI impact from measuring acceptance rate of suggestions are conveniently ignoring that this the same template we use for the “impact” of DORA metrics.
There are metrics and hand waving about impact.
There are metrics and hand waving about impact.
October 18, 2025 at 4:43 PM
I’m arguing that people criticizing claims of AI impact from measuring acceptance rate of suggestions are conveniently ignoring that this the same template we use for the “impact” of DORA metrics.
There are metrics and hand waving about impact.
There are metrics and hand waving about impact.
My position is we need more rigor and less handwaving when we make these connections.
October 18, 2025 at 4:40 PM
My position is we need more rigor and less handwaving when we make these connections.
Not at all. But the metrics measure what they measure. They don’t measure business value. There is too much handwaving that implies otherwise.
October 18, 2025 at 4:39 PM
Not at all. But the metrics measure what they measure. They don’t measure business value. There is too much handwaving that implies otherwise.
“Kinda measure your batch size and bureaucracy” is the problem.
Much more honest to say they measure adoption of DevOps practices.
Just as Ai suggestion acceptance rate measures AI adoption.
Neither directly can make a plausible cause-effect claim on “impact”
Much more honest to say they measure adoption of DevOps practices.
Just as Ai suggestion acceptance rate measures AI adoption.
Neither directly can make a plausible cause-effect claim on “impact”
October 18, 2025 at 3:38 PM
“Kinda measure your batch size and bureaucracy” is the problem.
Much more honest to say they measure adoption of DevOps practices.
Just as Ai suggestion acceptance rate measures AI adoption.
Neither directly can make a plausible cause-effect claim on “impact”
Much more honest to say they measure adoption of DevOps practices.
Just as Ai suggestion acceptance rate measures AI adoption.
Neither directly can make a plausible cause-effect claim on “impact”
One could argue the “proven metrics” do the same thing
DORA metrics measure “DevOps” impact and so on 😁
This is just the next in a long line of metrics that fundamentally get measurement in software wrong.
DORA metrics measure “DevOps” impact and so on 😁
This is just the next in a long line of metrics that fundamentally get measurement in software wrong.
October 18, 2025 at 12:38 PM
One could argue the “proven metrics” do the same thing
DORA metrics measure “DevOps” impact and so on 😁
This is just the next in a long line of metrics that fundamentally get measurement in software wrong.
DORA metrics measure “DevOps” impact and so on 😁
This is just the next in a long line of metrics that fundamentally get measurement in software wrong.