Over the last several years I've been fortunate to work with a lot of companies involved in agile adoptions. I hesitate to use the word transformation because I have seen far few actual transformations. There are several reasons for this, certainly more than five as indicated in the title. However, I see patterns at virtually every client organization over the past several years that are predictably similar. The five reasons outlined below are just a few of the reasons you're probably struggling.
Why are you even bothering with an agile transformation? Is it because of the state of your business? Are market conditions driving the need? Is the underlying cause the perception that it's cutting edge and you fear your company will be left behind if you don't transform? Yes, a lot of organizations are adopting agile and calling themselves agile. But please, don't do it unless you understand the business challenge you're trying to address. If you can't clearly articulate the reason you're transforming, you will like invest a lot of time, money, and effort and realize minimal results.
There are a lot of arguments about whether agile is a "mindset" or a "process" and I've even been involved in some of those arguments. But after doing this for a while I've realized that it's ultimately about addressing business challenges like being able to change direction quickly based on market conditions and/or customer need; or, better yet, operating in a manner that causes the market to react to you.
Does your company believe agile is something the software people do? It's very common to hear that one of the reasons agile is being adopted is to deliver faster. That is awesome; however, there are many questions that raises. For example, deliver what faster, and to whom? How much faster? What is the expectation of quality? Delivering the same ratio of defects to functionality may be worse than delivering as you are right now. All of these questions have to be taken into consideration from a systems perspective.
Organizations can be thought of as systems of systems. Any improvement effort can be approached with systems thinking. Adopting a methodology like Scrum, LeSS, SAFe, etc. may be needed, but it must be within the context of an overall organizational ecosystem. Think of them as what they are—a tool, not a silver bullet to slay your business pain. While frameworks like Scrum, Kanban, and even scaling frameworks like SAFe can be enablers; the bigger picture is that agile is better thought of as a management technology rather than as a development methodology.
Heavyweight process, centralized (hierarchical) decision-making, and functional specialization are all traits of a traditional organizational structure. Likewise, tight coupling, legacy code, and years of accumulated technical debt are typical in many enterprise-level software architectures. All of these forces work in tandem as inhibitors to enterprise agility. Adopting an agile methodology, Scrum for example, can help expose areas that need to be addressed. It's up to the organization to recognize and address the issues. Simply doing daily Scrums and other ceremonies is not enough. When you chose to adopt agile nobody told you you'd have to be willing to rethink your organization and its systems. However, it's necessary and critical.
Consider your current business context. If an industry disruption occurred, what is your level of confidence that you'd be able to react quickly enough to counter the disruption? If a competitor, or small startup, were to change the rules of the game could you incorporate changes into your systems quickly enough to ensure viability and long-term success? Unfortunately, many companies are not structured in a way that will allow them to adapt to changing market conditions. Low risk tolerance has led to centralized decision-making and bureaucratic process, unwittingly leading to sloth-like movement. Organizations tend to get more serious about change when demise is at the doorstep. This is analogous to quitting smoking only after a cancer diagnosis has been delivered.
While I do agree that individuals need to be trained, a more important aspect is to help people see the bigger picture in their context. It's been invigorating to me, as an instructor, to see individuals get excited about the possibility of adopting a new way of working. Too often, these individuals are thrust back into an organizational system that is ill-equipped to support that new way of working (see reason 3). Agility is not accomplished with training alone; it takes patience, thoughtful empathetic leadership, and the intestinal fortitude to address difficult sometimes harsh business realities.
Undoubtedly, most organizations will introduce the new role of ScrumMaster or agile coach when they first move to agile. People are sent to Certified ScrumMaster courses, given shiny new resume letters and turned loose to improve their teams and organizations - well, in theory that's what's supposed to happen. The sad reality is that most of these individuals lack the influence and/or positional authority to impact any true and lasting change. The result is that those individuals try to impact change at the level they can - the team level.
As a ScrumMaster this becomes a major source of frustration. You can see issues and impediments that need to be addressed, yet even if you raise them to management, they may be unwilling to address the issues because doing so would cause short-term disruption and they're getting paid to minimize disruption. Also, when you optimize at the team level you can simultaneously, and unknowingly, sub-optimize at a system level. It's not uncommon to see teams delivering with increased efficiency to downstream organizations or teams without the capability to consume what they're delivering.
How do you address these issues and move forward on a path of continuous improvement? The consultant in me wants to say "it depends" because every situation is contextual. But, in general, you can start with items considered to be the nucleus of agility, including team formation. A small-scale pilot approach can be used to establish and demonstrate early success as well as identifying trouble areas as you scale within your context. More to come on that in a subsequent post.