Category Archives: Decision Theory for Planners
We don’t need the mealy mouth stuff we often get for goals. We need goals that our community understands, rallies around and works toward. If a goal does not make people want to act, then that goal is useless.
The word “encourage” should never appear in a plan’s goals or objectives. If I encourage my son to study for a test, “encouraging” him is not the goal. “Pass the test” is the goal. “Encourage” is a cop out, which is why officials sometimes like it for goals they don’t want to support. If you cannot get support beyond “encourage,” then define what the parties involved can support, or cut the goal out. You’ll do more good by leaving it out than by giving it lip service.
The reason this blog was called Decisions, Decisions, Decisions was because it was supposed to be about geeky stuff to do with decision theory and its relationship with arguments in planning theory.
No such look the NPPF came along and took over.
But it helps to look at the key argument over the NPPF, over the ‘presumption’ from this perspective as it helps to tidy up certain confusion. I dont think we will be able to move on a ‘clarify’ the NPPF until this confusion is dispelled.
There are four logically distinct ways people have used the term in the debate.
A predisposition of the decision maker to a result, an expectation of a decision outcome by a user of the system, a statement of the burden of proof in evidential terms and as a decision tree of heuristics. Let me explain
1. A predisposition of the decision maker to an outcome
When people say that the planning system should be more ‘positive’ this is what they mean. Now that outcome could be yes to everything as the term ‘default answer is yes’ phrase implies or only to certain kinds of development in certain places.
At the heart of this is the argument about whether planning should just be a neutral ringmaster waiting for things to come or should be actively seek to built the right things in the right places. I should state that im firmly in the latter camp. My hero are those like Patrick Geddes and Raymond Unwin who got good planning done even before modern planning legal systems existing. But a presumption for the right things in the right places also logically implies a presumption against the wrong things in the wrong places.
Part of the confusion lies in the fear that a predisposition implies planning should become predisposed to cheerleading bad development rather than assessing it.
Lets say you have a case of the right thing in the wrong place? Or the wrong thing in the right place? This illustrates that planning is not just about assessing the sustainability of the development but ensuring that the use of land is the most sustainable it can be.
Give an illustration.
Lets say a site is zoned for a sustainable urban extension.
Along comes an application for a riding school on the same site.
Ok lets test if the development per-se is sustainable – yes – ok do you approve it? As a decision maker you are predisposed to positive outcomes, if an application comes along which is sustainable ill approve it? But the application may also be preventing a more sustainable option for the same site. If your predisposition is to positive outcomes where an application is sustainable and it is the most sustainable option likely to come forward for the site this might lead you to a completely different answer.
What this illustrates is that a positive disposition is never enough to make a decision – what you also need are clear decisions rules – what in decision theory are called heuristics.
Simplerly a decision rule by itself tells you little about what the likely outcome is going to be. If the decision maker makes every opportunity to pick holes and find holes the outcome will be no whatever the decision rule. Predisposition is about the culture of planning.
2. An expectation of decision outcome by a user of the system
Today I got up as an innocent many and hopefully will go to bed as one. This may sound strange but this labelling is important in English Common law. You are presumed innocent until proven guilty. This sense of the word, to avoid labelling and prevent abuse of the powers of the state is rather more difficult to apply to regulatory decisions. Is my planning application ‘presumed’ to be approved before I have supplied any evidence or even submitted it?
What you presume will rather depend on what you political position is on private property. If you hold a more Jeffersonian position you might feel you have every right to do what you want with it unless it harms the personal safety of others. If you hold a more Hayeckian view you might feel that others have no right to do what they want with their property if it harms, without just compensation, the value of your property. Others of course hold a more ‘commonweal’ approach to this issue.
Such an expectation will tell you nothing about the eventual mediation and decision outcome amongst these competing views, even amongst those with exactly the same political philosophy about property rights. My thesis is that concepts of property rights have by themselves no bearing on decision outcomes - unless you have an extreme ‘do what you like where you like’ view, one that only applies where you have no planning laws or the decision is made not to apply them. Lay those arguments aside and move on. The real issue is the public policy decision rule for resolving property rights disputes.
So this concept of a ‘label’ taken from criminal law helps us very little with issues of resolution of competing civil expectations.
3. A statement of the onus of proof in evidential terms
Whose job is it to show that a scheme is sustainable? Can an applicant submit a scheme with no evidence and then could a decision maker then only refuse it if they have paid for evidence sufficient to knock it down?
There has been endless confusion over this issue. Can you refuse an application for example for not providing a tree survey? The DCLGs position on this has changed I believe over the course of the consultation, which is at least a sign of constructive engagement. For example the mythbusters document refers to applicants having to show that a scheme is sustainable.
The onus of proof is different from the burden. For example the fact that I have to supply evidence by itself doesnt not tell you if the burden is ‘on the balance of probability or beyond reasonable doubt.
4. A decision tree of heuristics
This is really the nub of it the policy on how the decision maker makes planning decisions. The NPPF as currently drafted is as clear as mud on this point.
The decision maker will in every case apply a series of heuristics to the decision.
Decision rule 1 in most public policy decisions is typically – do I have enough evidence to decide if all of the subsequent decisions rules have been met – if no go back to the applicant, if yes proceed to decision rule 2 – and so on.
Under out current planning law decision rule 2 would be – does the scheme comply with the development plan considered as a whole. If yes …etc.
Then the decision trees branches concerning ‘other material considerations’ - weighing benefits and harm – it is unclear but this component of the presumption would appear not to have changed the decision tree but to have changed the burden of proof of this heuristic, no longer would you be able to refuse harmful development, the barrier is raised to ‘significantly outweigh.’ One can see days at public inquiries spent with arguments about just how ’significant’ and ‘clear; harm is rather than looking at the evidence of the harms and benefits which is what the inspector will want to look at.
Though I would rather the word ‘presumption’ was banished from planning law to end such confusions I am relaxed about its use if these separate components are teased out and made clear. This is what we have tried to do in the alternative draft. Try to make sense of the ‘presumption’.
Both PAS and the DCLG seem obsessed with activity based costing as a means of estimating the real full cost of processing planning applications.
But its bunkum.
In both the public and private sector its largely discredited.
The 2008 Review of Policing for the Home Office by Sir Ronnie Flanagan concluded that that manually driven ABC was an inefficient use of resources: it was expensive and difficult to implement for small gains, and a poor value, and that alternative methods should be used. Barely 4 years after its use had been mandated by the home office.
I know some private sector consultancies where staff spend up to 10 minutes every hour time recording. Are they happy then raising their variable cost base by 17% in this way!
Many would claim, apart from the time costs, it just doesn’t work as an accurate means of accounting. For example the field of throughput accounting many examples have been found of where cost accounting gives false results and if private sector bodies followed it they lose profits and if public sector bodies did they reduce productivity (Corbett, T. 1998. Throughput Accounting. North River Press.)
Lean accounting and throughput accounting have given us more accurate metrics so why does the planning profession use a method that is falling out of use elsewhere and is just plain flawed.
Lean accounting lesson – All you need to measures costs is to measure throughput and inventory once you have made an accurate assessment of labour and on costs for each person in a process. Now lets say you have 100 planning applications – for sake of argument all the same complexity – generating Y income. Divide your staff (again for simplicity assume all cost the same) costs x by Y you have your throughput over the period (lets say six weeks). If staff have 4 weeks target to process then if there intray stacks up you are loosing money/productivity, if it goes down if you gaining it. That’s all you need to measure the inputs and outputs. The same applies to staff commenting. It only gets complex with applications of different types. But if you set staff time processing targets for different categories of application – and you can benchmark these – you can apply the same input-output metrics. What is more such metrics will tell you if you are eating into or building up a backlog and whether you are gaining or losing productivity.
Activity based costing is a bit like trying to measure the GDP of a country by measuring the cost of every single transaction – an impossible task. All you need to measure is the changes to balances.
Their website Getting Lost is dedicated to the science of why people get lost and how we find our way. One of their interests is a richer understanding of Developmental Topographical Disorientation, the disorder of those that get lost very easily, including in extreme cases people getting lost in their own home. I once knew a planner with this condition who feared going on site and dreaded a proposed office move. When I took him on site once he was unable to describe the view in front of him.
Though orientation and navigation is undoubtedly complex, most people are able to reach their destination with little effort. This ability is produced by two primary behavioural mechanisms.
The first behavioural mechanism consists of making use of a variety of information (landmarks, body turns, distances, etc) in order to become familiar with the environment. This may either consist of learning a defined (fixed) route in the surrounding, or acquiring a more general knowledge of the environment including its layout and the spatial locations of landmarks available within it. Time spent in the environment as well as the information people decide to focus while navigating play a critical role on the ability that individuals will develop in orienting in that environment. The type of orientation that people adopt while learning and experiencing the environment mainly focuses on environmental landmarks and it is referred to as spatial memory.
The second behavioural mechanism happens unconsciously and is used while following habitual routes that individuals became very familiar with such as going from room to room in the house or workplace, or even driving from home to work along the same route for a long time. Navigation is guided by automatic motor sequences and can occur in the absence of landmarks. No explicit knowledge of turn sequences or landmark associations are required. This type of orientation is supported by procedural memory, the same memory system that produces learned motor activity such as riding a bike or playing a piano.
They don’t seem to give enough attention to the social interaction important in wayfinding. For example we are the only primate with white eyes, we can see where our colleagues are looking and even babies have this interactive ability. It seems to have evolved to enable early humans to undertake complex foraging rather than simply following game in a group, and point to our colleagues where the best places for food are.
A recent book by the American Philosopher Robert Kirkman has an original take on this issue – looking at planning decisions as an ethical issue.
Kirkman’s key issue is whether a place either constrains us or enables us to seek the good life. The second consideration is how the identified good is distributed among people. Is it fair? The third consideration is how the identified good is distributed through time. Will it last? Finally, there’s the question of process, is it legiminate?
Kirkman recognises that planning issues raise difficult issues of justice, one person benefitting from a good place, might be excluding others. A person with liberty to build where they like may be harming issues of well being valued by the people already living in the locality. This become even more difficult when it comes to the issue of what scale a decision should be made.
“To the extent all of the different ranges of government pull against one another, each asserting its own rights and prerogatives, there is less likely to be an effective response to problems in the built environment. Perhaps most important, there is often a mismatch between the scale of problems and the scale of government authority with the power to address them.”
Kirkman conderiders the issue of housing allocations. Almost everyone values a sufficient amount of housing affordable to residents with the range of incomes somewhere in their wider region, but the same people can exhbit Nimby tendencies about putting it in their own neighborhood, and few homeowners want to see their own home become more affordable through increasing supply.
This of course is a classic case of the fallacy of composition, that the aggregation of individual and locality decisions about housing locally does not reflect the community will about housing nationally.
Approaching this problem with too small a scale, and you get inefficient fragmentation; too large a scale, and you’re apt to be insulated from what citizens actually want for their own community.
Kirkman therefore promotes decision making on housing allocations based on regions which arn’t too large for individuals and localities to participate in.
Planners would recognise this as planning on a sub-regional scale.
It is interesting then to see a coherent argument that decisions on metropolitan growth taken at too smal a scale can be unethical.
A properly structured series of goals, objectives and targets can and should achieve five things, it is important to clarify these as confusion over these concepts in rife in planning;
1) Above all act as a framework for managing change and responding to the challenges that a place/organization faces – there is much to learn then from some simple concepts of management theory;
2) Act as a focus in forward planning on whether actions take you towards or goal or away from it; and solely focusing activity on removing constraints on achieving goals – this is a key insight from operations research and time management;
3) Act as a framework for making rational decisions against multiple criteria on a common basis, and prioritizing goals and actions – this is an insight from decision theory;
4) Act as a means as identifying spatial conflicts between land uses that meet different goals and how to rationally resolve spatial conflicts between land uses – this is a relatively recent concept from GIS thinking and its application to planning policy;
5) Act as a metric for the evaluation of different policy options against some preset criteria – such as sustainability appraisal/integrated appraisal etc.
Each of these approaches are closely interrelated and have greatly influenced each other. Taken together they can be used to create powerful, insightful, effective planning documents and processes.
Goals as a Management Tool
Goals as a planning tool will not be effective unless placed within a management context through which the ‘plan’ will be delivered.
A goal in this sense is a description of an end state that an organization wishes to achieve. The management literature tends to use goals and objectives as interchangeable terms, but the full benefits of a structured approach towards goal setting is only achieved if a hierarchical approach is taken distinguishing between goals, objectives and policies (this is mandated in some jurisdictions).
Goals in this sense are the high level and short statement setting out where a place/organization wants to be after a period of time.
Objectives are the specific statements setting out measures to achieve a goal.
A target is a metricated objective, or a metric for an objective, designed as a tool of performance management.
Example: Goal: Diversify the economy reducing reliance on oil and gas exports; Objectives in support of that goal: Provide sufficient land for light industries/logistics for which Bahrain has a competitive exports advantage, Allow for the planned expansion of existing heavy industries where there is a sustainability case for these to be located in Bahrain, etc. etc.
Notes: These are designed to illustrate the point rather than being suggested goals or objectives.
Goals and objectives should begin with a verb. This improves clarity of intent, implies action, and establishes a rhythm of ideas that enhances understanding. (Carr and Zwick).
The setting of goals and objectives is an iterative process, in particular at the early stages of a project up to and beyond the point at which tentative options are formulated.
From my experience 15-20 objectives is about the right number. Beyond this paperwork and monitoring becomes onerous.
The importance of Management by Objectives
The concepts that management should be about achieving the objectives of an organisation (Druker) and that objectives should be SMART (Locke). What is less well known is the origin in thinking about how people in an organization respond to objectives, and how organizations need to set goals to respond to changes in their environment.
Five Principles of Goal Setting (Locke and Latham)
To motivate, goals must have:
- Clarity – Clear goals are measurable and unambiguous
- Challenge – People are often motivated by achievement, and they’ll judge a goal based on the significance of the anticipated accomplishment
- Commitment – Stakeholders are more likely to “buy into” a goal if they feel they were part of creating that goal
- Feedback – Feedback from stakeholders provides opportunities to clarify expectations, adjust goal difficulty, and gain recognition
- Appropriate for Task Complexity – For complex goals give the person sufficient time to meet the goal or improve performance and provide enough time for the person to practice or learn what is expected and required for success.
Or backward design, is used quite often in education and project management. The idea is to start with your ultimate objective, your end goal, and then work backward from there to develop your plan. By starting at the end and looking back, you can mentally prepare yourself for success, map out the specific milestones you need to reach, and identify where in your plan you have to be particularly energetic or creative to achieve the desired results.
This is important where actions must follow a sequence, e.g. construct a road to open up a landlocked development area, or where (as is often the case in planning) achievement of a goal requires delivery of counter-intuitive prior actions.
Competitiveness Strategy & The Challenge-Response Framework (Michael E. Porter and Toynbee)
As set out by the highly influential work of Porter Competitiveness Strategy involves:
- Performing a situation analysis, self-evaluation and competitor analysis: both internal and external; both micro and macro.
- Concurrent with this assessment, objectives are set. These objectives should be parallel to a time-line; some are in the short-term and others on the long-term. This involves crafting vision statements (long term view of a possible future), mission statements (the role that the organization gives itself in society), overall corporate objectives (both financial and strategic) etc.
- These objectives should, in the light of the situation analysis, suggest a strategic plan. The plan provides the details of how to achieve these objectives.
This work has primarily concerned corporations, although it has been influential in framing ‘business growth strategies’ that have been used by many municipalities internationally in determining priorities for growth of their private sectors.
This framework, primarily concerned with cost and competitive advantage, requires adjustment when adapted to the public sector management of place, when factors more than primarily cost must come into play (although Porter does recommend a balanced score card approach) and the importance of civic responsibility and community leadership must be recognised. I have found very effective, when working with political leadership, the ‘challenge-response’ framework originally formulated by the Historian Albert Toynbee, to be an effective substitute:
- Cities and civilizations arise by the response of creative individuals to challenges presented by situations of special difficulty created by the external environment.
- Progress consists in meeting difficulties by responding in creative ways;
- The breakdown of society occurs when creative individuals fail to lead through the exercise of creative power, resulting in withdrawal of the allegiance of the majority and a subsequent loss of social unity.
Using this framework one can analyse the special challenges facing a place, with goals being those creative responses to these challenges.
As an example here is the most extreme example I could think of:
Maldives – Challenge – sea level rise will drown the country within 100 years. Potential responses, persuade other countries to adopt carbon reduction targets of over 80%, or create new areas of reclaimed land and move population, or move population to an area purchased from another country (the ‘uganda’ solution), or become a virtual country of refugees (like Palestine). Not all challenges might have an achievable or desirable response!
 Specific, Measurable, Achievable, Realistic, Time Bound.
 “A Theory of Goal Setting and Task Performance.” Prentice-Hall (19 Dec 1989)
 Competitive Strategy: Techniques for Analyzing Industries and Competitors 1998.
 Named after the early british suggestion of resolving the Palestine question by creating a jewish homeland in Uganda.
Of the spectrum of all possible policy options the frame of all reasonable options is called the ‘Overton Window‘. It is named after its originator, Joseph P. Overton, former vice president of the Mackinac Center for Public Policy
By reasonable its means those considered reasonable in terms of the current political discourse, rather than the additional test of being reasonable in terms of practicality etc.
It is a means of visualizing which ideas define that range of acceptance. Proponents of policies outside the window seek to persuade or educate the public so that the window either “moves” or expands to encompass them. Opponents of current policies, or similar ones currently within the window, likewise seek to convince people that these should be considered unacceptable.
After Overton’s death the concept caused a lot of chatter amongst thinks tanks, expanding the idea so that new ideas entering the public discourse they can shift the window of what is considered reasonable. add the concept of moving the window, such as deliberately promoting ideas even less acceptable than the previous “outer fringe” ideas, with the intention of making the current fringe ideas acceptable by comparison.
Lets take an example. A political party proposes education vouchers, this is condemned, but following public arguments a new ‘compromise plan is put forward which may be considered reasonable, whereas if this watered down plan had been put forward originally it may have at first been considered extreme.
The following diagram sums it up.
The technique has been used by many previously fringe think tanks to make their ideas seem mainstream. A classic example is the very anti-planning discourse used by the Policy Exchange (see numerous posts here exposing there methods). The term has even been taken up by that way-out king of conspiracy theories Glenn Beck as the Title of a dreaful novel.
It is a form of door in your face negotiation. ‘Would you donate £5,000 to our organisation? No ok how about £5′
The onslaught of essays, breakfast briefings and speeches by those, including those in No10, proposing an abolsute minimalist and anti-planning approach in the new NPPF is a classic example. At the end of the day the planning system might not be watered down as such extreme thinkers want but it might have been watered down considerably.
The problem is that even giving credence to offbeam ideas can give them a false credence. This is known as ”Okrent’s Law” during his tenure asa news paper editor Okrents commented about his job ”The pursuit of balance can create imbalance because sometimes something is true,” referring to the phenomenon of the press providing legitimacy to fringe or viewpoints in an effort to appear even-handed. Classic examples is undue press given to climate change deniers and opponents to evolution.
“Some would say that hydrogen cyanide is a delicious and necessary part of the human diet, but others claim it is a toxic and dangerous substance. The truth must therefore be somewhere in between
So if you see someone trying to create an Overton Window there can be only one rational response, don’t compromise.
When it is not necessary to make a decision, it is necessary not to make a decision.” ~ Lord Falkland
“A good plan, executed now, is better than a perfect plan next week.”
- General George S. Patton, Jr.
The above two quotes show two sides of the delay dilemma. For Falkland the risks of deciding now outweighs the benefits. Better information, or changed circumstances make deciding now risky. For Patton the risks of waiting outweighs the benefits, we might have good understanding of the situation now, but not in the future, by when circumstances might not be in our favour. Patton of course was one for putting the other side on the back foot, with his nostrom of ‘attack is the best means of defence’.
We then seem to have two factors that weigh on the issue of the meta-decision, that is how, when and why we make a formal decision. How fast circumstances are changing and the quality of our information and the perception of how this may change in the future.
You might think then that a simple approach of rational certainty. I.E a two factor optimisation model would suffice of the form of: If the benefits of making one of two or more options now outweigh the uncertainties and dis-benefits of making the decision in the future then make the decision.
Such a theory though does not stand up to serious thought.
When we make a meta-decision to decide we are making two decisions, to decide and the choice we decide. when we make a meta-decision not to decide we make only one.
The rational certainty approach does not stack up because the decision to decide and the choice we make are always and indivisibly one decision. We decide because our choice and its risks have advantages to the staus-quo ante when projected forward, even where projected forward one infinitesimally small division in time.
Where we cannot decide it is either because the uncertainties of delay outweigh the advantages of delay or we are struck by geniune indecision between two or more choices and are seeking resolution of those choices.
Meta-decisions therefore have a fundamental antisymmetry in time compared to other decisions. Indeed the uncertainties associated with meta-decisions seem to dominate the public policy process.
What is more because the future is fundamentally unknowable rational ignorance is essential for making a decision at all. As you can always seek out more information it might seem more rational to delay, but that delay has an opportunity cost in terms of future information seeking yourself or from others.
A decision to decide is effectively saying to yourself I cant predict the future but im going to make an assumption that the future will not be significantly divergent from the future I have mentally projected. That is the future will not be surprising.
If though that future is surprising to someone else we may have an advantage over them, particularly in terms of economic advantage. Seen in this way it supports GLS Shackles view that economic decisions are made in terms of assessment of competing degrees of potential surprise, and that decision making is a creative act in terms of how we shape the future.
As well as not being able to disentangle a meta-decision of yes from a decision we cannot disentangle such decisions from other actors.
“In some negotiations, the tactic of one side might be to delay negotiations indefinitely. For example, environmentalists can often discourage a developer through protracted litigations.” – Howard Raiffa, in The Art and Science of Negotiation, 1982
A decision space is not just defined by the actor making the decision but those who supply information and can react to a decision seeking to overturn it or by supplying information in the hope that the decision is delayed. Such examples are legion in planning.
Strategic delays are often used by disputants who do not have the power to win directly in court or through procedure. Rather than waste their resources on a fight they cannot win, some use delaying tactics to frustrate their opponent hoping that circumstances turn in their favour, such as a change in national policy. If a group is part of the decision-making process but does not want change, the slower the process, the better. If they do not have a say in the design of the process, then they can deliberately delay it by stalling on their involvement.
Most recently the crackdown on corruption in India has become so great that civil servants are trying to prevaricate on all decisions by giving excessive briefing notes to ministers because they know that all decisions will subject them to scrutiny whether innocent or not.
Also complex decisions are interconnected, often part of a social mess, so we are often left in a position of a possibly infinite regress of not being able to decide one thing without deciding everything. This might seem rational looking at individual widenings of the decision space but overall the dis-benefits, of a decision being delayed exponentially, force a cut of in terms of the scope of the expected interconnected impacts of our decisions.
The final problem with rational certainty is that the vast majority of decision made are not made on the basis of assessment of more than one option.
Gary Kein’s research, set out in his book ‘Intuition at Work: Why Developing Your Gut Instincts Will Make You Better at What You Do (2002) derived from his work in the US miltary on how commanders make decisions.
He found that 95% of decsions are made without options, without concious consideration of evidence, they were made on instinct. This led him to wonder, with only one option waht was the decision compared to?
He believes that a given situation, the decision maker will pick up cues and indicators that let them recognise patterns. Based on these patterns and the decision they have to make, the person chooses a single course of action, an ’ action script‘, that they consider will achieve the outcome – they ran a mental simulation.
The mental simulation was based on mental models that the decision maker had developed through experience. In other words, the decision maker has an idea how things work based on the knowledge that has been gained from experience. S/he compares the option against what is known to work.
This is intuitively correct. Inexperienced planners will often be paralysed by the need to seek more evidence and the choice of options, experienced planners will often leap to a conclusion and then either seek evidence to support it or disprove it. As people become more expert in their chosen field, as they have more and more experience, their ability to recognise patterns is enhanced. This gives them more options to choose from. Which means that, more often than not, the first option they choose will work.
If the decision maker considers the action script will achieve the outcome, they go ahead.
If they consider that it might not work because of a potential problem, they may try and alter the action script in some way. If mentally they don’t think it will work, they discard it completely, and choose a second action script.
This is then mentally rehearsed and so on until they find an action script that they think will work. This is then utilized. Note that in this recognition primed decision making model there is no comparison of alternatives. If this pattern is used often then the decsion can be routinized. Routinized decisions can also be instituationalised as policy, indeed this is the source of policy that works, based on expereince of achieving successful outcomes.
This theory is very similar to Gerd Gigerenzers ideas of gut instincts we met in the first part of this series.
One of the key reasons The Nazis lost the battle of Britain was the inability of their fighters to protect bombers for more than about 20 minutes because of small fuel tanks. Field-marshal and later Air Inspector General Erhard Milch had recommended months before the battle of Britain that cheap drop tanks should be developed in preparation. The programme never went ahead, why? It was questioning the conventional wisdom of the organisation and of his superiors, who tended to discount the information of those lower down the hierarchy. It is what is known as hierarchical incompetence .
The theory is in any organisation that someone will get promoted in an organisation if they are competent and not get promoted if they are not. The consequence of course is the Peter Principle, people will get promoted to the level of their own incompetence. The employee’s incompetence is not necessarily a result of the higher-ranking position being more difficult — simply, that job may be crucially different from the job in which the employee previously excelled, and thus requires different work skills, which the employee may not possess, skills such as management rather than technical competency.
Hitlers skill was as a party leader and rabble rouser, his incompetence was as a commander in chief, never wishing to make a tactical retreat or listen to the ideas of others. The film Das Bunker should be read as an allegory on how not to make decisions in any large organisation.
The relevance paradox occurs because people seek only information that they see is relevant to them. There may be information. That is not perceived as relevant because the person does not already have it and becomes relevant only he has it. Lets give an example. Google etc. may seem like a completely open way of seeking information, we are in command. No website operators have worked out that people that hold to a particular view are only likely to seek out information confirming that view, so they optimise their websites to capture those people in search results. Its called search engine optimisation or SEO for short. Its a multi-billion dollar business. Lets say you wished to set up a baby clothes website. You need to capture those people with babies and who might be searching for things on the net about babies. Lets say there was a health scare. You set up a magazine with lots of articles on the baby health scare but at the side put links and adverts for baby clothes. Lets say you were a climate change denier and did a search for ‘global warming disproof’ you might convince yourself that all of the evidence disproved it. But youd be looking in the wrong place. Eli Pariser has called this the Filter Bubble, its another illustration of confirmation bias.
Poor information is at the route of why decisions go wrong. So how may we improve on information flows and gathering.
One tactic is to stress test the foundation of the belief as to why something is correct. For example town planners have lots of foundational belief they believe to be true. They are unaware that many many people disagree with these because they have never sought them out. They are shocked when in positions of power they implement anti-panning beliefs based on concepts they have never heard of. Deliberately seeking out such contrary information, through role playing, can inform you if your foundational arguments need correction.
Organisations possess considerable tacit knowledge, knowledge that people are not often aware of they possess or how it can be valuable to others. Communicating tacit knowledge generally requires extensive personal contact and trust. In overly hierarchical organisations restrict the flow of such knowledge.
Dave Andrews of the Open University has developed many ideas of how to improve the flow of information across organisations. In the late 1970s he was looking at energy he found that energy and raw material could be saved, and pollution avoided by interlinking industrial processes.
Most professionals and policy makers were not interested in these opportunities because they were unaware of them, for example the waste heat from power stations, which is an example of hierarchical incompetence and the relevance paradox. Andrews called this the need to find interlock . As no individual or group can ever understand anything fully on their own their is a need to interlock with the knowledge and ideas of others.
Writing in 1984 he urged urgent development of “lateral media” – a system where a PC in every home would be linked by modems and the telephone network and be equipped with software to enable messages, news and inquiries to be forwarded selectively to create a cloud of lateral communications hopping from computer to computer (similar to the way a flock of birds or shoal of fish communicated in order to stay in a coherent whole). He had unknowingly invented social media.
The problem of course with social media is when we lock ourselves in the filter bubble and restrict the potential benefits of interlock knowledge.
An interlock diagram is a way of bringing together expert knowledge to pierace teh filter bubble
For example in energy policy a woprking group might consist of people expert is several spheres of energy.
If then you map the limits of the potential of each sector you expose the tacit knowledge and throw up opportunities for interconnection. Its a way of putting expert knowledge at the fingertips of the decision maker.
Andrews, David; The IRG Solution – Hierarchical Incompetence and how to overcome it. Souvenir Press, London, 1984 – available online here.
Geoffrey West the noted English physicist has studied scale for many years, moving from looking at physics, to biology to now cities and firms.
“I’ve always wanted to find the rules that govern everything,”
“It’s amazing that such rules exist. It’s even more amazing that we can find them.”
“We spend all this time thinking about cities in terms of their local details, their restaurants and museums and weather,”
“I had this hunch that there was something more, that every city was also shaped by a set of hidden laws.”
West took an idea from biology, Kleiber’s scaling law, and tried to explain it. This looks at how the metabolism of a creature changes with size. So for example while an elephant is 10,000 times the size of a guinea pig, it needs only 1,000 times as much energy. The metabolic rate of a creature is equal to its mass taken to the three-fourths power. The number 4 seemed to pop up in all kinds of ways in similar laws of scale.
West sought to explain this by looking for laws which can be described as describing the infrastructure of space, the energy needed to take things around a network. 4 was no accident, it a network it describes the three degrees of freedom of space plus one which can be explained as the ‘fractal’ quality of networks, like the crinkliness of the UK coast which gets longer the closer you look at it.
This seemed to describe a certain increasing return to scale about physical phenomenon.
West looked at whether this applied to cities, gathering massive amounts of information with his co-worker Luis Bettencourt. These ideas have been gathering increasing attention in recent months as they supply clues to describing a number of long understood but previously disparate phenomenon.
They found massive economies of scale in settlements, just as in big animals.
They concluded that cities are a lot like elephants. In city after city, the indicators of urban “metabolism,” like the number of gas stations or the total surface area of roads, showed that when a city doubles in size, it requires an increase in resources of only 85 percent.
Economists had long studied the agglomeration economies of scale of cities. Most famously Alfred Marshall
When an industry has chosen a locality for itself, it is likely to stay there for long; so great are the advantages which people following the same skilled trade get from near neighborhood to one another. The mysteries of the trade become no mysteries; but are as if were in the air, and children learn many of them unconsciously. Good work is appreciated; inventions and improvements in machinery, in processes and the general organization of the business have their merits promptly discussed; if one man starts a new idea, it is taken up by others and combined with suggestions of their own; and thus it becomes the source of new ideas.
Marshall described three advantages 1) shared inputs, 2) labor market pooling and skill matches, and 3) knowledge spillovers, but all three of these are network effects explainable by the mathematics of West’s law, the ability of a good, member of a labour market or information to diffuse along a network.
Jane Jacobs argued that spillovers between industries in a diverse city was more important that spillover within a specialised industrial district, but in a more fundamental sense both effects have a deeper cause, how advantages scale.
In recent years Noble Prize Winner Paul Krugman and others in the ‘new economic geography’ have been grappling with a problem. Without increasing returns to scale it is not possible to explain the growth and formation of cities at all, but acknowledging increasing returns undermines many of the assumptions of neo-classical economics.
But West found that not just the size of the economy increased by 15% per capita with city size, so did other things such as the number of crimes, traffic, even disease. It was the increased volume of such interactions in cities that Jane Jacobs promoted of course and West sees his ideas as offering a mathematical explanation of them. West describes the purpose of urban planning as finding a way to minimize our distress in cities while maximizing our interactions.
West and Bettencourt refer to this phenomenon as “superlinear scaling,” The slope leads to city growth from the positive feedback loop of urban life — a growing city makes everyone in that city more productive, which encourages growth which encourages more people to move to the city.
“When we started living in cities, we did something that had never happened before in the history of life…We broke away from the equations of biology, all of which are sublinear. Every other creature gets slower as it gets bigger [ because of the energy costs]. That’s why the elephant plods along. But in cities, the opposite happens. As cities get bigger, everything starts accelerating. There is no equivalent for this in nature. It would be like finding an elephant that’s proportionally faster than a mouse.”
So whilst large cities are more energy efficient per capita than smaller ones they lead to growth which consumes more energy overall. Man can break the constraints of biology by generating his own energy.
“The only thing that stops the superlinear equations is when we run out of something we need,…and so the growth slows down. If nothing else changes, the system will eventually start to collapse.”
For West only innovation can get us out of this resource bind
“It’s like we’re on the edge of a cliff, about to run out of something, and then we find a new way of creating wealth. That means we can start to climb again.”
This is an old idea in economics, the classical ‘steady state’ providing limits to growth of Ricardo, JS Mill and Marx, inspiring John Raes and Schumpeters concept of swarming innovations creating new opportunities for growth. Of Jevons concept that increasing cost from declining energy sources (in his day coal but today oil) will spur innovation in new energy technologies.
Bettencourt and West have also begun to look at the issue of the size of corporations, however the relationship is the opposite of cities, as the number of employees grows, the amount of profit per employee shrinks.
For West the the decline in profit per employee makes large companies increasingly vulnerable to market volatility. Since the company now has to support an expensive staff — overhead costs increase with size — even a small disturbance can lead to significant losses. As West puts it, “Companies are killed by their need to keep on getting bigger.” He may not realise it but that is pretty similar to Marx’s theory of the declining rate of profit. West so far does not explain the cause of the decline in profit per employee. Marx’s equations had the hidden assumption that capital intensity would increase at a faster rate than the profit generated/unit of labour, if it doesn’t you get an increasing rate of profit, and no capitalists rationally introduces a new technology that will reduce their profits to negative.
An explanation could be in the rigid way large firms are organised, limiting communication and innovation between staff that naturally occurs in small firms and between multiple individuals in cities. Large firms that promote such innovation tend to be the most successful. But combining the concept of increasing returns to scale of industrial processes with decreasing returns to scale at the level of the firm (the number of employees) seems to solve a number of puzzles in economics – such as why successful firms don’t expand infinitely (Sraffa 1926), and why small firms tend to create most new jobs.
It may also explain some puzzles in urban design. Andreas Duany is famous for his theory of scale invariance, that there are principles which apply at all scales of planning, from the region to the neighbourhood, such as access to open space.
West and Bettencourt’s ideas are new and sweeping. At first site they appear to be anomalies. For many large African cities there is no increasing efficiencies of scale, I would suggest though this is from a model of urbanisation which has seen investment in infrastructure fail to keep pace with city growth, leading to huge dis-economies.
A similar problem, which has led some urban researchers to be dismissive, is the decline of many large american cities and the growth and job creation rates of many suburban areas. I believe though there are explanations for this. American cities had grown beyond their bounds leading the centre taking on high city fixed costs and high tax earners living outside the city limits. This can create a downwards fiscal spiral. Also when industrial cities specialise in the same type of firm all growing together they can all decline together. Finally when cities forget what they are about, creating economies of scale in labour and information, through encouraging loss of central urban jobs and low cost labour, cities can easily hollow out. Where the disadvantages and costs of living in a settlement exceed the benefits people will settle elsewhere, spawning new cities or sprawl. The problem though with sprawl is the increasing costs with scale and lack of increasing advantages. Sprawl can drag an economy down in the same way an economy reliant on old industries can.