Why Does Organisational Change Fail?

“We have forty million reasons for failure, but not a single excuse.” Rudyard Kipling, ‘The Lesson’

Organisations in the 21st century face growing challenges arising from globalisation, increased competition, technological change and a dispersed and increasingly mobile workforce. The external environment is arguably becoming more volatile, unstable, chaotic and ambiguous, and the pace of change required for organisations simply to survive has ostensibly increased as a result. Heraclitus’ dictum that “everything changes and nothing abides” is becoming more appropriate when considering the context in which modern organisations operate (Tsoukas and Chia 2002). Management of change has thus become a core requirement of modern-day leaders, who want their change programmes to be successful (Hughes 2010) in order for their organisations to thrive and prosper.  This has given rise to a plethora of academic studies and practitioner texts providing leaders and change agents with advice on managing successful organisational change.
Yet, against this backdrop, the odds of change management programmes actually succeeding are seemingly not good.  Numerous authors report that organisational change efforts fail more often than they succeed, summarised by Sorge and Van Witteloostuijn’s (2004 p.1212) assertion that “…organizational change has an undeniable tendency to produce failure.”
This essay will consider the question of why, given the various approaches available, organisational change still fails. It will start by acknowledging the many differing definitions of ‘organisational change’ and also consider what is meant by ‘failure’ in relation to change initiatives. It will explore the validity of the widely espoused failure rates and the extent to which they might be accepted as accurate. It will then argue that the answer to this question will always be grounded in epistemological and ontological perspective and examine the perceived reasons for failure from both a realist/objectivist and a nominalist/subjectivist position.

Get Help With Your Essay
If you need assistance with writing your essay, our professional essay writing service is here to help!
Essay Writing Service

Organisational change is a familiar characteristic of organisational life, often seen as a tool which senior managers employ when seeking to improve processes or increase efficiencies to achieve strategic goals (Thomas et al 2016). However, despite the pervasive nature of the phenomenon, there is little consensus in the academic or practitioner literature as to a definition. Van de Ven and Poole (1995 p.512) state that organisational change “is an empirical observation of difference in form, quality, or state over time in an organizational entity. The entity may be an individual’s job, a work group, an organizational strategy, a program, a product, or the overall organization.”  This gives the impression of change being a difference in states that can be observed ‘empirically’ and therefore in some way is tangible and measurable. It also acknowledges that many elements of the organisation may be subject to change.
However, Jones (2010 cited in Hughes 2010 p.13) defines organisational change as “…the process by which organisations move from their present state to some desired future state to increase their effectiveness.”  Thus, in this interpretation, organisational change is a process that controls movement from one state to another and the entire organisation is in the process of moving.
Helms Mills et al (2009) consider organisational change to be the “alteration of a core aspect of an organization’s operation”. This suggests that organisational change only relates to a central element of the organisation’s business.
As Frahm (2007 cited in Hughes 2010 p.13) notes, the organisational change literature covers ‘a vast and diverse body of work that encompasses micro and macro views of the firm, as well as varying foci (scale of change, type of change, interventions in change and people in change”.  Van de Ven and Poole (1995 p.533) present a typology of four main change theories: life cycle, teleology, dialectics and evolution but, as the authors note themselves, ‘change and development processes are often more complex than any one of these theories suggests’. The fact that organisational change includes such a wide variety of initiatives and programmes leads to a key criticism in the academic literature, which is the difficulty of building theory from research that does not differentiate between a myriad of different types of change (By 2005).
One element of organisational change that does seem to have gathered some consensus however, both in the practitioner and academic literature, is the idea that the majority of all organisational change projects fail (Thomas et al 2016). The failure rates quoted are often depressingly high – 70% is frequently cited (Kotter 2008) and some estimates even go as high as 93% (Decker et al 2012).  The suggestion is that change initiatives generally do not deliver the outcomes expected by those who invest in or evaluate the change. This leads therefore to consideration of whether organisational change does fail to such a degree and if so, why that might be.
The first point to consider is whether change programmes actually do fail to the suggested extent.  In this respect, it is necessary to first attempt to define what is meant by ‘failure’ in relation to a change initiative. According to Decker et al (2012 p.31) change failure is a failure to implement a change programme, which they define as “either a new project or strategy that was formulated and not implemented, or one that was implemented but with poor results (e.g., missed schedules, expected value not reached…)”. Based on a review of the literature, they offer a taxonomy of over 60 critical failure factors: things that must go wrong in order for change implementation process to be perceived as having failed. These failure factors include the individuals involved and the organisation’s culture, processes and communication. In a further meta-analysis, Mosadeghrad and Ansarian (2014) identified 55 barriers to successful change implementation. The failure of change programmes was variously attributed to a lack of senior management support, inadequate education and training, issues with organisational culture, apathetic employees, ineffective leadership, lack of communication or financial support and the absences of clear plans, underlining that the potential reasons for the perceived failure of change programmes are numerous.
However, Hughes (2011) argues that the rhetoric surrounding the failure of change initiatives may be misleading and the purported failure statistics do not seem to be grounded in solid empirical data. His analysis of five well-respected, published papers which all cite a 70 percent organizational change failure rate concludes that it is impossible to identify the source of this statistic and there is little valid or reliable supporting empirical evidence to substantiate it. Indeed, Kotter’s (2008) use of the 70% failure statistic seems to be purely an estimate from his own personal experience. There are relatively few studies that have based high failure rates on actual research conducted by the authors themselves. Burnes (2011) cites a small number of studies (Whyte and Witcher 1992; Rogers et al. 2006; Senturia et al. 2008 and Tarokh et al. 2008) but advises caution over the validity of their findings. Thus, without reliable data it is difficult to know the true extent to which organisational change programmes fail (Burnes 2011).  If there is little empirical evidence of change initiatives failing, the next step is to consider what might give rise to this common perception of change failure and this will benefit from contemplating differing epistemological and ontological perspectives.
The question of what constitutes change failure may be dependent upon ontology. Ontology is defined as the study of being and the ontological issue for consideration here is whether organisational change is a cognitive construct that is tangible and measurable and whether it exists independently of, and externally to, the individuals involved. For the purposes of this discussion, a realist/objectivist position will be contrasted with a nominalist/subjectivist position.
Adopting a scientific and rational viewpoint, change is approached from a positivist/realist perspective of being a quantifiable, measurable and objective reality that exists independently of the individuals experiencing it. Positivist literature suggests that the key to change is proper planning and efficient delivery of the plan. Leadership is seen as being central to this process and it is accepted that the success or failure of change programmes will lie with leaders’ skill at managing change. This has given rise to the expectation that leaders will be ‘transformational’, with the ability to create and communicate a compelling vision for the future and lead their companies through what can sometimes be quite traumatic changes (Lui 2010).
Authors such as Kotter, Kanter and Pugh prescribe logical, linear ‘n-step’ roadmaps for successfully managing change (Burnes 2011), which have proved popular with leaders who are expected to be able to set the objectives of change programmes, steer the organisation through them and judge whether they have been completed or are successful. This school of thought sees change strategies as being intentional and rational, requiring analysis, planning and logical implementation (Helms Mills et al 2009). Organisational change in this respect has become reified (Tsoukas and Chia 2002) and treated as an exceptional situation, with a defined beginning and end, rather than a continuous natural state. 
Viewed from this perspective, change failure is seen as a failure to implement the change programme in accordance with the prescribed steps and therefore not achieving the objective identified at the outset. However, this may lead to a very narrow view of success and failure. Thomas et al’s (2016) review of 62 organisations participating in change activity found that change initiatives are regarded as a failure when the changes do not deliver what were originally identified as the ‘valued outcomes’ – the desired goals or objectives of the change. However, many of the participants in the change programmes under study were still able to provide evidence of other significant benefits that had been derived as a result of the implementation.  Another study (Thomas et al 2012 cited in Thomas et al 2016) found that organisations that failed to achieve the original objective often instigated further change efforts to try to achieve the intended aim. 
In order for organisations to be persuaded to invest in change, leaders and change agents must create a solid and compelling argument for why the change is needed from the outset (Thomas et al 2016). Most leaders start with a clear idea of what they believe needs to change for the organisation to thrive (Smith and Graetz 2011). This might include claims of potential quantifiable improvements and opportunities. A programme is only considered to have been a success if the desired goals are achieved. If that is not the case, then the programme is likely to be considered a failure, regardless of whether the actual outcomes are still of value. 
It therefore follows that success and failure are relative states that can only be judged in relation to a predetermined measure or marker, which has to be established by a stakeholder or stakeholders in the change process (either internally or externally to the organisation).  This highlights a key issue with the planned and rational approach to change management, in that the desired outcomes set out from the start of the change programme may create unrealistic expectations of what might be achieved. Original plans are seldom realised exactly as envisaged (Smith and Graetz 2011), so change agents following prescriptive change advice may be setting themselves up for failure, even if the change brings positive results.
It also places a strong focus on planned change, which an organisation decides to undertake, rather than unplanned change that it did not initiate. This downplays, therefore, the complexities of change and the impact of external influences or unplanned events which can divert or disrupt even the most well-defined of plans.  The complexity and speed of unplanned change make it challenging for leaders and change agents to anticipate and manage. Thus, these linear models of change are challenged by Herrero who suggests that change is viral, seen as a series of waves, and that orderly models are out of step with today’s “chaotic, unpredictable, fast-moving, multi-directional and multi-dependent world” (Herrero 2008 cited in Harrison 2015 p.60).
It is also argued that the preoccupation in the literature with planned change fails to recognise the perpetually changing nature of organisations.  Tsoukas and Chia (2002) define organizational change “as an ongoing process, a stream of interactions, and a flow of situated initiatives, as opposed to a set of episodic events” (p. 568). Burnes (2011) sees change as continuous experimentation and adaptation to the needs of a dynamic environment. In contrast to the view that change is episodic and orchestrated from the top-down, this perspective sees change as being ‘grounded in the ongoing practices of organizational actors” (Orlikowski 1996 cited in Tsoukas & Chia 2002).  Weick and Quinn (1999 cited in Tsoukas & Chia 2002) suggest that organisational change may be better described as ‘changing’ in order to convey the dynamic nature of organisational life.  If change is a continuous process, it could be questioned whether it can really be evaluated at any point as having failed. 
However, the concern with constant change is that it may make individuals less responsive to the demands of change. This leads to the possibility of ‘repetitive change syndrome’ (Hughes 2010) leading to overload, change-related chaos, employee burnout and the further potential for failure of the change programme.
An alternative ontological perspective, therefore, is that which views change as a cognitive construct that can only exist in the discourses and perceptions of those involved in it. This constructionist position would argue that as change is co-created by individuals, it can only be viewed subjectively. Individuals and groups will therefore have their own view of what constitutes success or failure (Herold et al 2008). The move therefore is away from the idea of having a single, objective ‘truth’, towards a search for multiple ‘truths’ which are created through individual subjective perspective.
This approach sees a shift from problem-solving to sense-making (Palmer and Dudford 1996) and acknowledges that many realities exist within organisations and these may not be aligned with the visions proposed by senior management (Sinclair 1994). It is not possible to create fully structured change and the outcome of the changes that are implemented cannot be controlled by management (Sinclair 1994).  Thomas (2016) cites research by Stensaker and Falkenber (2007) which found that different sections within the same organisation reacted to the same change initiative in a variety of different ways. If one accepts that there are ‘multiple realities’ then it follows that each individual, both managers and followers, will be seeing the events in ways that may have little in common. If there can be no single objective truth to be discovered, the issue then becomes how agreement is reached about the reality of a situation between contending points of view (Cooper and Burrell 1988 cited by Marshak and Grant 2008). Individual receptiveness to change and differing interpretations of the change may impact upon the extent to which the change is seen as a success or a failure (Thomas et al 2016). This viewpoint highlights the centrality of language, social construction and critical perspectives to the study of organizational change. Adopting a discursive approach can be a useful way of considering why organisational change programmes might fail (Marshak and Grant 2008).
With an emphasis on discourse and power relations, it is argued that change initiatives fail because talk of change frequently suppresses the possibility of conflicting organizational meanings (McLellan 2011). In a qualitative study of organisational change at an art college, McLellan (2011) shows how adopting a communication focus can reveal the extent to which discourse alignment and conflict suppression during conversations can lead to the failure of change programmes. He concludes that the reason a sustainability change programme failed at ArtCollege was not through lack of opportunity to talk about the change, but because conflicts between the college’s existing understandings and the new discourse of the sustainability project were suppressed, rather than being articulated and explored. Communication therefore ceases to be simply an activity for enabling change, a ‘persuasive practice, or process, of information sharing’ (McLellan 2011 p.477) and begins to take on the mantle of a political process; a practice that can also cause change to fail because the ways that it is talked about within the organisation do not enable new meanings to emerge.
Given the complexity of the literature, it is acknowledged that there are innumerable perspectives on the reasons for organisational change failure.  Weick (1998 in Tsoukas and Chia 2002) has observed that the main barriers to rethinking change are the ontological and epistemological commitments that have underpinned research into the subject and therefore it is suggested that the definition of, and what constitutes, change failure is also grounded in the epistemological and ontological perspective of the researcher.  When considering why organisational change programmes fail, despite all the research and tools available, the answer is best provided by Zammuto (2001), who notes, “I don’t think it is likely that there ever will be clear agreement about what constitutes effective change management for the simple reason that the answer will always depend on who you ask and what they value.”
References

Burnes, B. (2005) ‘Complexity theories and organizational change.’ International Journal of Management Reviews, Vol 7 No. 2 pp. 73–90.
Burnes, B. (2011) ‘Introduction: Why Does Change Fail, and What Can We Do About It?’ Journal of Change Management, Vol 11 No. 4 pp. 445–450.
By, R.T. (2005) ‘Organisational Change Management: A Critical Review.’ Journal of Change Management Vol 5 No. 4 pp. 369–380.
Decker, P., Durand, R., Mayfield, C.O., McCormack, C., Skinner, D. and Perdue, G. (2012) ‘Predicting implementation failure in organization change.’ Journal of Organizational Culture, Communications and Conflict, Vol 16 No. 2 pp. 29-49.
Harrison, P. (2015) ‘Communicating Change’ in Ruck, K. (ed.) Exploring Internal Communication Farnham: Gower pp. 57-64.
Helms Mills, J., Dye, K., and Mills A. (2009) ‘Understanding organizational change.’ Abingdon: Routledge.
Herold, D.M., Fedor, D.B., Caldwell, S. and Liu, Y. (2008) ‘The Effects of Transformational and Change Leadership on Employees’ Commitment to a Change: A Multilevel Study.’ Journal of Applied Psychology Vol 93 No. 2 pp. 346–357.
Hughes, M. (2010) ‘Managing Change: A Critical Perspective.’ (2nd Ed) London: CIPD
Hughes, M. (2011) ‘Do 70 Per Cent of All Organizational Change Initiatives Really Fail?’ Journal of Change Management, Vol 11 No. 4 pp. 451-464.
Kotter, J.P. (2008) ‘A Sense of Urgency Boston’, MA: Harvard Business School Press.
Lui, H. (2010) ‘When Leaders Fail: A Typology of Failures and Framing Strategies.’ Management Communication Quarterly, Vol 24 No. 2 pp. 232–259.
Marshak, R. and Grant, D. (2008) ‘Organizational discourse and new OD practices.’ British Journal of Management, Vol 19 Special Edition 1 pp. 7–19.
McLellan, J.G. (2011) ‘Reconsidering Communication and the Discursive Politics of Organizational Change.’ Journal of Change Management Vol 11 No. 4 pp. 465–480.
Morgan, G. (1986) ‘Images of Organization’. London: Sage.
Mosadeghrad, A.M. and Ansarian, M. (2014) ‘Why do organisational change programmes fail?’ International Journal of Strategic Change Management Vol 5 No. 3 pp. 189-216.
Palmer, I. and Dudford, R. (1996) ‘Conflicting uses of metaphors: reconceptualizing their use in the field of organizational change.’ Academy of Management Review Vol 21 No.3 pp. 691-717.
Sinclair J. (1994) ‘Reacting to What?’ Journal of Organizational Change Management Vol 7 No. 5 pp. 32-40.
Smith, A.C.T, and Graetz, F.M. (2011) ‘Philosophies of Organizational Change’ Cheltenham: Edward Elgar.
Sorge, A. and Van Witteloostuijn, A. (2004) ‘The (Non) sense of organizational change: an essay about universal management hypes, sick consultancy metaphors and healthy organization theories.’ Organization Studies Vol 25 No. 7 pp.1205-1231.
Thomas, J., George, S. and Rose, T. (2016) ‘Deciphering value discourse’s role in explaining the persistent perception of change failure’. Journal of Change Management Vol 16 No. 4 pp. 271-296.
Tsoukas, H. and Chia, R. (2002) ‘On Organizational Becoming: Rethinking Organizational Change.’ Organization Science Vol 13 No. 5 pp. 567-58.
Van de Ven, A.H. and Poole, M.S. (1995) ‘Explaining development and change in organizations.’ Academy of Management Review, Vol 20 No. 3 pp. 510-540.
Zammuto, R.F. (2001) ‘Book review of breaking the code of change.’ Administrative Science Quarterly, Vol 46 No. 4 pp. 796–798.