Agile Governance

The Digital and Agile Transformations of the enterprise both pose a challenge to classical IT governance frameworks. This chapter reveals the limits of existing IT governance practices and illustrates the features of new governance models.

The typical organization chart for an enterprise defines hierarchical and functional reporting relationships, together with the scope of responsibilities for organizational entities. It does little to help understand how decision-making will happen. This is the scope of governance, which defines who has power and accountability, and who makes decisions. Governance is defined as: “a company’s allocation of decision rights and accountabilities: who has the authority to make – and is held accountable for – key decisions” [MIT]. Good governance aligns decision-making authority with accountability. “Authority” is defined as the power to act and “accountability” is defined as being accountable or liable of the outcome of decisions or activities.

A lack of coherence between the allocation of responsibility, authority, and accountability results in disfunctional organizations.

The two established governance forms are:

  • Corporate governance, which defines the processes and structures through which the board of directors oversees what the enterprise executives do

  • IT governance, which defines the decision rights and accountability framework for encouraging desirable behaviors in the use of IT [Weill 2005]

The way enterprises define IT governance is not uniform. Some IT governance models are centralized while others are de-centralized. Ad hoc governance bodies are created to oversee large transformation initiatives.

New terminology and practices emerge as illustrated in Governance Forms, which is not an organizational chart and only positions governance models:

  • Digital governance, which covers a scope comparable to the one described in Agile Governance

  • The TOGAF Architecture Governance model

The TOGAF Standard, Version 9.2 [TOGAF Standard 2018], defines Architecture Governance as the practice of monitoring and directing architecture-related work. The goal is to deliver desired outcomes and adhere to relevant principles, guardrails, and roadmaps. Therefore, one of the objectives of architecture governance is to ensure that business value is delivered in alignment with strategy, and follows the business and technical compliance criteria.

Architecture governance has two dimensions per the TOGAF definition: to monitor and direct; refer to “Part 2, Guidance on Architecture Governance” of The Open Group White Paper, World-Class EA: Governors’ Approach to Developing and Exercising an Enterprise Architecture Governance Capability (see World-Class EA 2017) that explains in detail and provides guidance on how to govern in the following dimensions: “Strategic Architecture”, “Governing the Implementation”, “Governing the Architecture Practice”, and “Governing Value Realization”.

Architecture governance should also be applied at difference levels within the enterprise, as explained in the TOGAF Standard; see Chapter 44, Section 44.1.1: Levels of Governance within the Enterprise. The architecture organization, as illustrated in Figure 44-2 of the TOGAF Standard, highlights the major structural elements required for an architecture governance initiative. For more details, see the TOGAF Standard, Chapter 44: Architecture Governance.

The enterprise which decides to deploy a specific governance model determines the relative positioning of IT versus architecture governance. IT governance must cohabit with digital/Agile governance or be replaced by it.

fig-governance-forms
Figure 1. Governance Forms

Classical IT Governance

Most classical IT governance models:

  • Appoint a decision-making body, for which names and composition vary; for example, an IT executive committee or a large project committee, which may include the CEOs of operational entities

  • Set rules to determine which projects or programs will be reviewed, and what happens when approval is denied

  • Organize architecture reviews to inform IT governance committee decisions

  • Set up architecture offices to conduct reviews, define standards, and drive compliance

  • Set up focus groups to identify best practices in key areas, such as Application Architecture, testing, infrastructure, or security

  • Consolidate IT budgets and manage an enterprise-wide IT investment portfolio

  • Track the utilization of IT resources to verify that once projects are approved, they follow plans and comply with budget guidelines

Classical IT governance does not prevent large projects or programs from failing. Large enterprises have experienced project or program failures resulting in several hundred million dollar write-offs. For example, a large financial services firm had to “kill” a new securities handling system that was supposed to be deployed in several European countries, or a bank that failed to consolidate two core banking systems derailing the Mergers & Acquisitions (M&A) business case.

Among the root causes of these failures, governance issues represent a fair share; for example:

  • Governance committee members who break their commitments without facing consequences

  • Decisions tainted by power struggles or territory battles

  • Ad hoc governance which weakens established IT governance

  • Too little cross-silo cooperation before IT governance bodies make decisions

  • Committee members who influence decisions despite not being accountable for outcomes

A governance model alone cannot fix problems that are rooted in the social system or culture of the enterprise; for example, when:

  • Committee members do not respect their commitments

  • Enterprise silos use governance committees as a political battleground rather than cooperating upstream

  • Leaders find “good reason” to avoid respecting IT governance decisions

Governance deployment will not be effective if it does not address social system or organizational culture issues. The adoption of Agile ways of working at scale tends to challenge classical IT governance even further.

Governance in the Face of Agile

Enterprises which deploy Agile ways of working report that the first visible impact on IT governance is a reduction in the number of projects that get reviewed by IT governance committees. Because the shift to Agile tends to split large projects into smaller pieces no longer managed as a whole, fewer initiatives reach the minimum size to meet the threshold of IT governance. These smaller pieces are no longer projects, but products delivered by Agile teams; they move under the radar of classical IT governance.

Agile organizations are composed of teams of teams that operate in rapid learning and decision-making cycles. The Agile way of working is not compatible with static and siloed organizations that allocate decision rights along reporting lines. Classical IT governance body members are not close enough to the field. They often base their decisions on “second hand” information that may not reflect current reality. In contrast, Agile organizations “instill a common purpose and use new data to give decision rights to the teams closest to the information” [Brosseau 2019].

Agile organizations shift from project to product, which is about replacing temporary teams with stable teams led by product owners. Most Agile teams are cross-functional and stream-aligned. Product owners are responsible and accountable for delivering product features or sub-journeys, which include software components. Because Agile teams are cross-functional and led by product owners who have business and technology acumen, the project manager role is becoming less relevant. Classical IT governance is structured around projects and programs that are reviewed by committees. This is incongruent with new Agile ways of working, that are not project-centric.

In Agile organizations there are fewer standards and processes. Those that remain are defined differently. Instead of being specified as a set of tasks or activities to accomplish, standards are now used as guardrails, which express policy intent. For example, instead of mandating software engineers to select software products from a catalog of approved technologies, a policy could specify a set of operational and security requirements expressed as guardrails. In this example, the purpose is to develop software systems that can be operated reliably and securely. Candidate software products are assessed against guardrails by Agile teams in a bottom-up manner versus being defined and imposed top-down by an architecture office.

Agile Governance

Agile governance should not be conceived in isolation. It is part of a larger whole which includes the enterprise social system and its organization. Agile governance is not compatible with command and control thinking.

When authority flows from top to bottom, management is at risk of making decisions that are not anchored in reality. Insights into real problems and opportunities become obscured by the simplification and abstraction of reporting information.

When bottom-up communication is reduced to project status reports discussed during governance committees, it reduces the number of interactions creating even more distance between reality and the few “in command”. Command and control thinking is not an effective way of aligning autonomous teams, because top-down flawed decisions are likely to clash with autonomous teams. Therefore, Agile governance shall not replace a true shift of the enterprise toward Agile, as illustrated in Towards the Agile Organization.

fig-agile-organization
Figure 2. Towards the Agile Organization

Agile organizations promote both autonomy and alignment [Ross 2019] by:

  • Providing distinct goals and objectives to autonomous teams, and aligning them without introducing layers of hierarchy

  • Setting formal sharing mechanisms that synchronize activities as the number of teams grows

  • Defining architectural guardrails that facilitate autonomy by ensuring that individual components are compatible

Agile governance uses the four levers below to align autonomous teams:

  • Shared purpose or “true north” [LEI], defined as “an organization’s strategic and philosophical vision that may include “hard” business goals such as revenue and profits as well as “broadbrush” visionary objectives that appeal to the heart”

  • Shared consciousness, described by General Stanley McChrystal [McChrystal 2015] as “sharing a fundamental, holistic understanding of the operating environment and of our own organization while also preserving each team’s distinct skill sets”

  • Forcing functions (or guardrails), described by John Rossman [Rossman 2019] as a set of “guidelines, restrictions, requirements, or commitments that “force”, or direct, a desirable outcome without having to manage all the details of making it happen”

  • Feedback loops that help to adjust the behavior of autonomous teams

As Agile teams become responsible for delivering and deploying software that impacts the quality of customer experience, the scope and size of the central IT organization decreases. What remains central mainly covers:

  • Corporate functions, such as IT strategy or the architecture office

  • Infrastructure and IT operations, which deliver services to Agile teams

  • Cross-cutting services, ranging from federated identity management to collaboration services

  • Systems with high compliance or investor obligations

In addition to infrastructure platforms, the digital enterprise develops and operates business platforms. Platforms need governance as a set of rules concerning who gets to participate, how to divide cost and value, and how to resolve conflicts. For example, a large financial services organization has created a service marketplace governed by a Design and Operations Authority (DOA).

The missions of the DOA are to:

  • Support the shift toward an “as-a-service” model, by ensuring services are built and used consistently across the enterprise

  • Define and maintain service development and exposition guardrails to bring together service consumers, providers, and operators

  • Define forcing functions to ensure that every service meets minimum quality requirements before being published in the marketplace

  • Ensure the timely publication of services in the Market Place Catalog

As more and more IT capabilities are moving under the responsibility of cross-functional Agile teams, business and IT governance are merged into an integrated model; as illustrated in Integrated Business/IT Governance and described in points (1) to (4) in the text below.

fig-integrated-business-it-governance
Figure 3. Integrated Business/IT Governance
1 Quarterly Business Reviews verify that tribes fulfill their commitments
2 Tribes are monitored by their sponsors, who progressively release resources to match market growth needs
3 Tribes evolve autonomously, paced by Agile ceremonies
4 Postmortems are conducted during business reviews; resource reallocation decisions take team performance and business situations into account

The integrated governance model does not isolate IT investment decisions anymore. IT investments are a subset of the investments made by the business to develop new products or journeys. Investment decisions are evaluated against business outcome metrics.

The Agile governance features described in this chapter need to be adapted to fit the context, culture, and maturity level of each enterprise.