EA value…. where are thou? [Part 2 – KPIs]
There’s hardly any EA web or discussion where measuring EA value would not be a hot topic. I’ve read tons of opinions on this matter. And I am still not fully satisfied with any of those approaches or lines of argumentation.
I see three major directions in which EA justification is done:
1.) Ex-post Justification
Basically, after all the work is done, we look back and estimate how much the company earned or saved. One recent example I saw here from Troux Technologies.
2.) Measuring the EA Work
EA sets up processes, creates artefacts and then measures these. This is quite common example of KPIs used to communicate how well EA is doing:
– Number of annually reviewed artefacts
– Percentage of AS-IS architecture completed
– Number/Percentage of projects reviewed
– Number of projects rejected
There’s plenty of these which one can invent quite easily.
3.) Measuring Bright Future
It is kind of magic (as with most business cases:). One can “play cards” of how dreadful the world would be without EA and look how awesome we helped to make it.
As probably many of you, I have a mental problem with all of these. Here comes why:
Ad 1.) There’s one success, but many parties to claim the benefits afterwards. First of all, there are the decision makers who – very likely – decided for running or stopping a project that brought the benefits. Second of all, there’s the project and all its members who implemented the change. And if done successfully, they have full right to claim the benefits as well. Third of all, there’s EA folks claiming it’s all because of them. And fourth of all, there’s the rest of the world (all directly or indirectly involved people, departments, consultants, software & service providers) wanting to be part of that success. And not without any reason. Procurement could help to get better conditions, consultants bring best practices, it all adds to the final package.
Additionally, there’s the whole environment that influences the result. There might be other projects supporting the very same benefits from a completely different angle not related to each other in the project portfolio, external market changes etc.
Now, there’s one package of benefits and many parties to harvest the single fruit and no formula for splitting it up. How much of the credits should then EA get???
And at the end one personal remark. In my past experience I have two big success stories I am really proud of in my EA career. But in none of those cases I got the credit at all. Why? Because at the end of the day many people believed it was their idea, identified themselves with the idea, committed to it and made it happen. The hard work at the beginning, hell lot of thinking, communicating, convincing, bringing people together and getting them on the same page, passing the idea etc. was perhaps not even perceived as a “real” work.
Ad 2.) Most of these show (in the better case) how well we are running our EA function and processes. It is a kind of “efficiency” measure. It however does not show at all if what we do makes sense at all – if it is the right thing to do (effectiveness measure).
With many of these KPIs it is even very difficult to judge what is a good and what is a bad result. Is high number of rejected projects a good or a bad sign? Sure, If I want to, I could market this as savings made by EA. But if this number is low, or even zero, should I interpret it as a fact that I have no power to stop a project? Or that the quality of projects is so good and they are so well prepared and aligned with the strategy and all that stuff that there is no need to reject a project? And is this high quality natural, or is it a prevention effect of having processes, review boards and other things in place?
Ad 3.) This one I dislike probably the most because my inner myself struggles with a lot. Security and compliance are cases where this happens a lot as it is difficult to estimate potential impacts if “Lion escapes the zoo”. The same is valid quite often with EA, trying to estimate what would happen if EA was/wasn’t there. It’s so far from reality and so hard to believe.
Now getting back to my first post on what I believe EA (at least in my current position) is mostly doing – providing relevant information for decision makers.
Take this very simple example: Maintaining candidates for decommissioning.
It is clearly something that can lower costs at the end. We can gather information, identify potential candidates, verify costs and benefits, propose changes, scenarios, create business cases. We can establish, optimize and run processes so that a decision is made at the right place at the right time and by the right people. We can consult the project, align it with other activities in the company or do QA on the project. It’s a hell lot of work. Very useful work though. But I would not dare to state, that the benefits for actually decommissioning things in the company go to EA.
So here comes the question: How much is “knowing” worth? How much is worth helping others succeed?
An obvious answer might be to charge the customers for EA services. But isn’t that already given in our salaries, in the sheer existence of a team of (more-less) trusted and experienced people? The value should be then perceived by our customers (decision makers) and their trust in the relevancy and quality of information (our services) we provide. If it’s not, we might not be doing a good job. This trust and need for our services is then worth more than any artificial number we market as our benefits for the company.
Not to be only on the critical side, there is for me one lesson learned as well. It seems to me that concrete case studies work better than any KPI. Real cases from the company of the work that was done. What problem was identified, what we have done on the way to resolving it, how it was resolved and what are the benefits. Without judging, blaming, wearing borrowed plums, as unbiased information as possible presented in an understandable way, pointing out what would not (most likely) happen without involvement of EA.
Now I am ready to get some “slap in the face” with your feedbacks:)