Fallacies of RAG Reports

3 thoughts on “Fallacies of RAG Reports”

1. Pingback: B-17E Bomber – Tight coupling of Color Coding with Numerical Data | Key Steps To Success

2. My summary of about a dozen of posts to the original of this article at https://www.projectmanagement.com/contentPages/article.cfm?ID=315648&thisPageURL=/articles/315648/The-Fallacy-of-Red–Amber–Green-Reporting&msg=Your%20comment%20has%20been%20posted%20below.&CFID=153339298&CFTOKEN=a10cb784fecf1eef-55E66EAD-B6B5-8CE7-F40A50E8DC53CF0E

   =========================
   By scanning all the comments/questions, provided so far, I was able to group them as follows:
   • Requiring some examples on “how”, examples of practical implementation. I selected some quotes, that made me feel that such grouping would be logical:
   o “What are suitable replacements for RAGs?”
   o “What are suggested replacements?”
   o any good metrics they have successfully implemented to make process more scientific?
   o “…attach a numerical value to the color to the item being rated. With a provided legend …”
   o “It would be good if you included links to specific agile metrics that you believe serve to solve this problem. ”
   o “What are the “various scaling techniques and a variety of dynamic agile tools”

   • Requiring more light to be shined on some deep/systemic organizational dysfunctions that exist today in many organizations. Here are some examples:
   o “but provides an assurance that the PM is in control!!!”
   o “…The PM should be providing recommendations on decisions to be taken. …”
   o “The PMO should be responsible for making sure that management reporting is consistent and that management understands what it means.”
   o “There is a responsibility on management to reward and encourage the correct behaviour”
   o “I have seen management change when they understand the control they regain by receiving honest project status reports. ”
   o “Upper management does not have the time to stay up to date on incrementally changing charts and metrics. They EXPECT me to escalate issues as needed and RAG is a means to do this”
   o “…the most useful aspect of status reporting using RAG, changing in status is picked up before it becomes a serious issue”

   I wonder if dust has settled or if there are more questions/comments to expect that may need to be placed in grouping, other than the two above.
   While waiting, I would like everyone to take a quick look at the following:
   • An entertaining piece that did not make it to this post, although it was meant to be a part of it: https://www.keystepstosuccess.com/2016/01/b-17e-bomber-tight-coupling-of-color-coding-with-numerical-data/
   • Larman’s Laws of Organizational Behavior: https://www.craiglarman.com/wiki/index.php?title=Larman%27s_Laws_of_Organizational_Behavior
   • For this audience, specifically, I would like to recommend the top shelf: https://www.keystepstosuccess.com/recommended-books/

   I would expect to have more questions that can be placed in the second grouping. But still testing my assumption.

   Reply

3. 
   When asked to make a post of this kind on project management site, I did hesitate, as I was not sure this subject was appropriate in this forum. So, I have to take on the responsibility of my decision.
   
   But, I would want to spend very little time discussing “substitutions” to RAGs and a bit more time addressing more fundamental (systemic problems) that underline RAGs. Substitutions? Well, they are right in the original post. Any properly calibrated chart (burn up, burn down, CFD, iteration report, release report) can producea real-time snapshot state of affairs for a single team, multiple teams, across a program or portfolio (hopefully, not ‘fake’ portfolios, of ‘fake’ projects). You just have to learn how to read it. The benefit you are getting is that it is instant and free. Free – meaning you don’t have to have a a person or a few persons sitting and manually collect numbers (worse yet, colors) to produce a roll-up number (worse yet, color). Explaining ‘how’ this can be done would be out of scope of this discussion, as this would take us town the path of discussing tooling. I would expect that anyone, who is familiar with incremental product development, has seen or experimented with such tools…. Any ‘element’ of product/program/portfolio development can be ‘plugged in’ such tooling (even non-development efforts) and it would produce numeric trends and forecasts that make RAGs simply useless. We may actually free up some people, so that they can go back and to some other work (perhaps code, test, data mine)?
   But it is really color coding that makes RAGs so harmful today? No. It is rather what goes in them and who is behind. So, back to systemic issues. Please, allow some quotes from below:
   -“…The PM should be providing recommendations on decisions to be taken. …”. – Why? Why should be non-doer (someone who does not code or test) make decisions on behalf of skilled, smart, self-organized, cross-trained team members? Where is autonomy? Where is mastery and purpose? Why skilled workers need to be treated based on Tyrolian principles of carrots and sticks? In transformation workshops, we play a game “who stole my cheese” by laying down a RACI chart and asking students use colored dots, to align roles to responsibilities… Oddly, about 85% of responsibilities is shared by a real customer, team-producer and team facilitaror/enabler (please, note, I am avoiding to use scrum terms).
   -“…but provides an assurance that the PM is in control!!!” – Really? Why? In control of what? Skilled workers? Work that they do? Workers have their own deliverables and if they are properly self-organized and motivated why do they need a shepherd, exhibiting “command and control” behavior above their heads? Why not rely on a ‘servant leader’ instead, who is not a ‘command and controller’ but rather and enabler and impediments remover for team? Why not step back so that a team can perform (as a team) to their best abilities?
   -“The PMO should be responsible for making sure that management reporting is consistent and that management understands what it means.” – Why should another organizational layer that sits between producers and consumers be responsible for translation of data? Why not increase transparency and visibility? I want to be very clear that I decouple senior management FROM mid-level and first line management. Working very closely with many senior managers (CTO, CIO level) I can go on record and state that majority of them don’t see much value in RAGs as they exist today. They are all well aware of how much gaming and ‘books cooking’ takes place. They put up with today’s RAGs for the lack of better substitution. Sadly, for some of them, it is a ‘sugar pill’ as well – to see green boxes on ppt (how assuring). For some of them, it is a way to delegate responsibility down beneath themselves. If things go south, they can delegate accountability to a layer beneath. But the majority of C-level execs would rather get raw metrics directly from teams, without massaging, manipulating and collating data, instead of having data ‘roll upwards’ through multiple organizational layers. Raw numerical, meaningful values, coming from down below, even without color coding, are much more powerful than subjective and ambiguous RAGs produced by individuals that sit so far from action. Smart execs do gemba – constantly. They get out of their office chairs and come to the floor where action takes place. Very smart execs redesign their org structure altogether. They remove silos and unnecessary org layers (organizational MUDA). They transform orgs from siloed-functional orgs to cross-functional orgs that are aligned to product lines and true customers.
   -“I have seen management change when they understand the control they regain by receiving honest project status reports. ” – I highly doubt that senior management is feared of losing control. Execs that who have seen the power of real-time information flow directly from teams (steady velocity, percentage of Epic completion, Release burnup, etc) never want to go back to meaningless RAG reporting produced by non-doers. Execs love transparency and reliability. But in order to achieve it, organizational redesign is required and not all senior leaders are ready to do it (yet). Rocking the boat takes a lot of bravery.
   -“Upper management does not have the time to stay up to date on incrementally changing charts and metrics. They EXPECT me to escalate issues as needed and RAG is a means to do this” – charts aside, I think that your upper management might be missing out on real action. I would recommend the management to do what Taiichi Ohno has taught us decades ago, pioneering ‘gemba’ with TPS. I am not suggesting that what you deliver to them is not accurate (I have no right to make such judgement) but not coming down to trenches and not talking to teams directly is a mistake.
   -“…the most useful aspect of status reporting using RAG, changing in status is picked up before it becomes a serious issue” – in majority of cases, arbitrary and subjective color assignment lags behind a real status change. Green->green->green->green->red has been seen way too many times equate RAGs of a waterfall projects to a colored pressure gauge of a diver’s tank or B-17’s bomber gas tank (there color coding is meaningful). I would not how much gas I need to put to my tank to make an arrow go from yellow to green but I would be lying to my sr. manager if I tried to suggest him what percentage of BRD completion would take us “back on track” (BRDs – another example of huge organizational waste but not going there)
   -“There is a responsibility on management to reward and encourage the correct behavior” —…..maybe. But of very senior management, not of first-level line management delivering meaningless end-year performance appraisals to their employees. And rewards should be given to teams not individuals, if at all. And NOT based on individual performance but rather on collective performance (at most). I shared some highly recommended books in my previous post for a reason. Skinnerian behaviorism is very toxic and harmful. Removing money from the table for skilled workers – this is what needs to be done. Please, youtube Daniel Pink RSA Animate.
   RAGs, as we know them today, are a by-product of organizational design., of multi-layered organizational structure where internal contracts, blame shifting and organizational MUDA is a norm. Reducing the latter, will make the former (RAGs) become less important.
   g

   Reply