BTW: German version here/Deutsche Version hier.
Again and again and in different contexts W. Ross Ashby’s „Law of Necessary Diversity“ is quoted and I have never heard a contradiction. I read Ashby’s book „Introduction to Cybernetics“ a long time ago and learned a lot from it. The law so much quoted (chapter 11/6), of course, impressed and convinced me very much at that time. But soon I had my problems with the conclusions drawn from it and they became bigger and bigger.
Why is this important and, above all, what does it have to do with our concrete challenges in managing projects? I think that the conclusions are decidedly misleading and that it puts the success of projects at risk. So it’s not just my own personal problem of understanding, there are serious practical consequences at stake.
What did Ashby actually say?
Let’s start at the source. Ashby refers to game theory, and he describes the situation of two players, R and D. When D makes a play, R must decide which play to counter with. The number of possible outcomes of the game now depends on how large the number of possible moves of D is and how far R manages to reduce this variety so that ideally only the outcomes in which he is the winner occur. If there is always only one possible outcome, no matter what D does, where would the variety of outcomes be reduced to its absolute minimum, namely 1. If R has two different moves available, then at best he can reduce the variety of outcomes to half the possibilities. The law reads, „Thus, if the variety in outcomes is to be reduced to a fixed number or to a fixed fraction of D’s variety, then R’s variety must increase at least to the necessary minimum. Only diversity in R’s moves can lower diversity in outcomes.“ And a few paragraphs further on the sentence that everyone quotes: „Only diversity can destroy diversity“.
That the last sentence cannot be true in this way should be immediately obvious. Let’s take a fly as an example, which flies a complicated course. All it takes is a well-aimed stroke with the fly’s flap (an action with very little complexity) to destroy the fly’s diversity of movement once and for all. Some will classify this example as harebrained, but it brings us to the crucial point. If we wanted to reduce the diversity of the fly’s positions relative to us, we would have to be just as mobile as the fly and follow on its heels. Thus, if one acts within the same set of rules as the opponent (to abstract from the unfortunate fly), then Ashby’s Law applies. But if a player switches paradigms, it loses its validity.
The law of requisite diversity applies when everyone is operating within the same set of goals and rules. The goal of the game is also important. Let’s take a real game like tennis or soccer. In tennis there is always a winner and a loser, in soccer there can be a draw. If one accepts maximizing the number of wins and minimizing the number of losses as the goal, the variety of outcomes is drastically reduced. Success in soccer is possible with a complex short passing game (Tiki-Taka, we remember the successes of FC Barcelona) as well as with a defensive strategy (Catenaccio, with which Inter Milan was once very successful). Tiki-Taka is more likely to be associated with the attribute „diversity“, only this diversity can be „destroyed“ with a much less diverse style of play, as has been empirically observed for some time.
Enough with analogies
But now enough of analogies, let’s turn to our field of action. If one were to interpret Ashby’s law as all business authors known to me do, then a complex project characterized by diversity of possible outcomes would have to be managed by a correspondingly complex project organization. Let’s stay briefly in Ashby’s terminology: A complex project is characterized by a variety of influencing factors and results (including all relevant intermediate results). This is countered by a complex project organization (matrix is always a good idea there), a large number of precisely defined work packages, detailed schedules with milestones, dependency matrices, risk catalogs, controlling reports, etc., etc. If deviations from the plan are detected, they are responded to with even more complex structures and processes, because: Complexity can only be mastered with complexity (if we replace the misleading word destroy).
But: This is exactly the trap that Paul Watzlawick so aptly described in his „Guide to Unhappiness“: „More of the same“. He sees this as „one of the most successful and effective catastrophe recipes that has developed on our planet over millions of years and has led to the extinction of entire species (p. 28)“.
You can and must reduce complexity
I have experienced it so many times: If deadlines are not met, scheduling is refined and controlling is expanded. Once I had to step in at short notice as a crisis manager and report twice a week on the adherence to about 50 detailed deadlines, leaving correspondingly less time to work. Only after a few weeks did I manage to reduce this to once a week, and after some interim successes we were able to switch to two-week iterations, the results of which were reported. That’s also how we got to success. So we responded to the diversity of project challenges (many deadlines did not hold or could not be guaranteed) by reducing complexity. Luhmann has argued at length that it is precisely the reduction of complexity that is the crucial cultural achievement. Given the overwhelming volume of his publications, I would like to refer here to a key work where he presents trust as a mechanism for reducing social complexity. For our context, for example, trust in the competence of the project team allows a significant reduction of control mechanisms and vice versa.
Agile project management is simple yet effective
If we look at agile process models, it is noticeable that they are organized in a significantly less complex way than classically managed projects. The fixed timing of iterations, the breaking down of the entire workload to a backlog list of user stories, the abandonment of success controls by announcing degrees of completion (instead, the delivery of functioning software at the end of each iteration serves as the only relevant measure of progress) are examples of radical simplifications. However, agile process models were created to handle projects that are characterized by high complexity (moving target). Their success proves them right. The success rate of large software projects is significantly higher when using agile methods than with „classic“ process models, the more so the larger (and thus probably also more complex) the project. This is the clear finding of the Standish Group.
Summarized: You can react to the complexity of the task by increasing the complexity of the project organization. However, this is not a good idea. It is better to meet this complexity with simple but effective structures. This can be Scrum as a particularly uncomplex, because strictly predefined execution method. It can be XP, with a particularly simple intervention: let users and technicians work together directly. It can also be Lean, Kanban or DSDM. I gave an overview of these in another blog post, with a list of links.
What does this mean for practice
So is Ashby’s law wrong? No! Ashby’s law is just like the laws of physics. For example, the law of falling states that a feather and a lead ball fall to the ground at the same rate. In everyday life we cannot observe this phenomenon, because the law is valid in such a way only in the absolute vacuum. Ashby’s law is valid only within a closed control system with fixed target variables. This is overlooked in the common conclusions drawn for practice. This in turn leads to negative consequences. Keep it simple is also too undifferentiated a recommendation; the important thing is to find a suitable simplification. Agile process models are an example of this.