Why Architecture?

Description

The word “architecture” is usually associated with physical construction: buildings, landscapes, civil infrastructure, and so forth. It was appropriated by systems engineers at IBM around 1960 to describe the problems of designing complex information processing hardware and software. This leads to some confusion, and occasional questions from “real” architects as to why IT people are calling themselves “architects”. Perhaps a different choice of word would have been advisable.

In our journey to date, we have covered:

We have also covered the practices by which ideas and intentions are established and translated by investment into actions, including:

As we have progressed in our journey and scaled our company up, all these areas have continued to evolve. Specialization emerges. You have people with deep experience in cloud architectures and individuals with deep experience in e-records management and compliance. You do not have too many who are deep in both.

Your product portfolio (internal and external) is now in the hundreds or thousands. Some were built with the latest technology, and others run on older technologies now perceived to be dead ends. However, investing in rewriting or re-platforming them would not provide as much value as other uses of the funds, so you have to manage the risk of the older technology.

Investment decisions become harder. You are far beyond the days when you had a one-product focus. You have multiple interacting products and multiple interacting teams, and the relationship between the teams and outputs is not one-to-one. Understanding the business case for the investment gets harder; when you have a thousand services over multiple business units, how do you know if someone is proposing a redundant new one?

Moreover, there are the big headaches. A major commercial product version is going off support, and it is the perfect time to think about a rewrite or re-platform (say into the cloud). However, the moving pieces and interdependencies are formidably complex, and if you get the analysis wrong, the business impact will be severe. You acquire another firm, with a lot of overlapping activities, and start to see the need for “Business Architecture” to clarify your understanding of business processes, organizations, and capabilities. Alternatively, a major outage hits the business hard, and all of a sudden the organizational priority (from the Board on down) is “fix this, so it never happens again”. Everything else is to go “on hold”. Except, of course, it cannot.

In response to these and a thousand other complexities of digital management as organizations scale up, a general-purpose coordination capability emerges, sometimes called Enterprise Architecture. In this Competency Category, we will discuss its definition, organizational dynamics, and value proposition.

Defining Enterprise Architecture

The fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution. [IEEE 2011]
— ISO/IEC/IEEE 42010
The Enterprise Architecture is the organizing logic for business processes and IT infrastructure, reflecting the integration and standardization requirements of the company’s operating model. The Enterprise Architecture provides a long-term view of a company’s processes, systems, and technologies so that individual projects can build capabilities – not just fulfill immediate needs. [Ross et al 2006]
— Ross et al.
Enterprise Architecture as Strategy: Creating a Foundation for Business Execution
Enterprise Architecture is the representation of the structure and behavior of an enterprise’s IT landscape in relation to its business environment. It reflects the current and future use of IT in the enterprise and provides a roadmap to reach a future state. [Bente et al. 2012]
— Bente et al.
Collaborative Enterprise Architecture
The purpose of Enterprise Architecture is to optimize across the enterprise the often fragmented legacy of processes (both manual and automated) into an integrated environment that is responsive to change and supportive of the delivery of the business strategy. [C220]
— The TOGAF Standard

“Architecture” as a term by itself is something you have encountered since your earliest days as a startup. Perhaps you used it to describe the choice of technologies you used for your products. Or the most important components in your application. Or the common services (e.g., authentication) you developed to support multiple products. The architecture concept is therefore not something new or foreign. But what does it mean to say we have an “Enterprise” Architecture? Enterprise Architecture is the unification of this document’s topics into a common, formalized, scalable framework for understanding. It means we are “doing architecture” comprehensively, considering the enterprise itself as a system to be architected. It also may mean we have a program for sustaining the work of those doing architecture in the technical, application, solution, data, process, or business domains.

In terms of our emergence model, Enterprise Architecture assumes multi-product, “team of teams” problems. As an overall domain of practice, Enterprise Architecture encompasses a variety of specialist domains (some of which we have already encountered) as we will discuss in the next Competency Category. Some of those domains do make sense at smaller, single-product contexts (e.g., software architecture).

There are numerous definitions of Enterprise Architecture; see, for example, IEEE 2011, Ross et al. 2006, and Bente et al. 2012. It can be defined as:

  • An organizational unit

  • An organizational capability

  • A formalized program

  • A professional discipline or set of practices

  • A process or process group; an ongoing activity or activities

  • A large-scale artifact (i.e., an integrated model consisting of catalogs, diagrams, and matrices) maintained on an ongoing basis for communication and planning

  • An integrated and standardized language for reasoning about complexity

In general, definitions of Enterprise Architecture characterize it as a coordination and problem-solving discipline, suited to large-scale problems at the intersection of digital technology and human organization. An important function of architecture is supporting a shared mental model of the complex organization. We were first introduced to the importance of shared mental models in Work Management and this need has only increased as the organization became more complex. Enterprise Architecture provides the tools and techniques for sustaining shared mental models of complexity at scale.

It may be useful to compare architecture to the activity of map-making. In map-making we have:

  • The actual terrain

  • The capability of map making: surveying, drawing, etc.

  • The process of surveying and mapping it

  • The resulting map as a document (i.e., artifact)

And once the map is made, you might use it for a wide variety of purposes, and you also might find that once you start to use the map, you wish it had more information. Similarly, in Enterprise Architecture, it is important to remember that there are different concerns:

  • Operational reality

  • The capability of representing it for planning and analysis (“being” an architect or an architecture organization; having the skills and tools)

  • The process of representing and analyzing the operational reality (“doing” architecture)

  • The actual representation (the “architecture” as a “thing” – a model, an artifact, etc.; recall our previous discussion of information representation)

And, like a map, once you have the architecture, you can use it for a wide variety of purposes, but also you may find it incomplete in various ways.

Architecture Organization

There are three major themes we will discuss in terms of the overall organizational positioning of Enterprise Architecture:

  • The line versus staff concept and its origins

  • Contrasting the concepts of “business model” versus “operating model”

  • The other major organizational units of interest to Enterprise Architecture

Architecture as Staff Function

We saw in Governance how governance emerges, as a response to scale and growth, and the concerns for risk and assurance in the face of increasing pressures of the external environment. One important response has been the emergence of the “line versus staff” distinction. As Christian Millotat (and many others) have noted: “Many elements that have become integral parts of managerial economics and organizing sciences can be traced back” to military staff systems [Millotat 1992]. These include:

  • Collecting and combining knowledge so that decisions are as well informed as possible

  • Supporting specialized roles and functions (e.g., legal experts, engineers)

  • Operating supply chains and other services that function best when shared

Staff functions in the enterprise include planning, coordination, and operations; broadly speaking, and with key differences depending on the industry, the following are considered “staff”:

  • Financial management

  • Human resources management

  • Legal services

  • Purchasing and vendor management (varies within companies and between industries; for example, in retail “merchandising” is a line function)

  • IT (however, with Digital Transformation this is increasingly overlapping with R&D and driven directly by line management)

  • Facilities management

  • Strategic planning and forecasting

While the following are considered “lines” (analogous to the warfighting units in the military):

  • Sales

  • Marketing

  • Operations

  • R&D (varies w/company and industry)

Enterprise Architecture has as a key part of its mission the task of collecting and combining knowledge to support decision-making. Therefore, an Enterprise Architecture organization can be seen as a form of staff organization. Most often, it is seen as a specialized staff function within the larger staff function of IT management and, with the increased role of digital technology, there are corresponding pressures to “move Enterprise Architecture out of IT”, as we will discuss below.

The classic line/staff division is a powerful concept, pervasive throughout organizational theory. But it has important limitations:

  • Staff organizations can “lose touch”, become insular and self-serving, and indeed accumulate power in dangerous and unaccountable ways; for this reason, officers are rotated between line and staff positions in the US military

  • Staff “expertise” may matter less and less in complex and chaotic environments requiring experimental and adaptive approaches

  • If a feedback loop involves both line and staff organizations, it risks being delayed; the delay waiting for “headquarters approvals” has been a common theme in line/staff organizations

In the history of line versus staff relations, we see tensions similar to those between Enterprise Architecture and advocates of Agile methods. The challenges, debates, and conflicts have only changed in their content, but not their essential form.

But in many cases, centralizing staff expertise and the definition of acceptable practices for a domain is still essential; see the discussion on matrix organizations and even feature versus component teams.

Enterprise Architecture and the Operating Model

In terms of overall positioning, Enterprise Architecture is often portrayed as mediating between strategy and portfolio management; see Enterprise Architecture Context, Based on Ross, derived from Ross et al. 2006, Figure 1-2.

EA context
Figure 1. Enterprise Architecture Context, Based on Ross

Notice the distinction between Enterprise Architecture as a capability and as an artifact. The practice of Enterprise Architecture is not the same as the actual Enterprise Architecture. For the purposes of this document, we define Enterprise Architecture’s concerns as essentially the enterprise operating model: process, data, organizational capabilities, and systems.

One of the most frequently used visualizations of Enterprise Architecture’s concerns is the Zachman Framework; see A Variation on the Zachman Framework, loosely based on Zachman 1987, and succeeding work.

Zachman Framework
Figure 2. A Variation on the Zachman Framework

We were exposed to the data modeling progression from conceptual to logical to the physical data model in Information Management. The Zachman Framework generalizes this progression to various views of importance to organizations, as shown in the columns:

  • What

  • How

  • Where

  • Who

  • When

  • Why

Overall, the Zachman Framework represents the range of organizational operating model concerns well. Certainly, sustaining a large and complex organization requires attention to all its concerns. But what good does it do to simply document the contents of each cell? Such activity needs to have relevance for organizational planning and strategy; otherwise, it is just waste.

Peer Organizations

It is reasonable to associate the Business Model Canvas with strategy, and the Zachman Framework with the Enterprise Architecture as an artifact; see Business Model versus Operating Model.

This distinction helps us position the Enterprise Architecture group with respect to key partner organizations:

  • Organizational strategy

  • Portfolio and investment management

  • I&O

Organizational Strategy

The operating model needs to support the business model, and so, therefore, Enterprise Architecture needs a close and ongoing relationship with organizational strategists, whether they are themselves line or staff. Defining digital strategies is a challenging topic; we have touched on it in Digital Context, Product Management, and Investment and Portfolio. Further discussion at the enterprise level will be deferred to a future edition of this document.

Portfolio and Investment Management

Architecture needs to be tied to the organization’s investment management process. This may be easier said than done, given the silos that exist. As Scott Bernard notes: “Enterprise Architecture tends to be viewed as a hostile takeover by program managers and executives who have previously had a lot of independence in developing solutions for their own requirements” [Bernard 2012].

Many organizations have a long legacy of project-driven development, in which the operational consequences of the project were too often given short shrift. The resulting technical debt can be crippling. Now that there is more of a move towards “you build it, you run it” the operability aspects of systems are (perhaps) improving. However, ongoing scrutiny and management are still needed at the investment front end, if the enterprise is to manage important objectives like vendor leverage, minimizing technical debt, reducing investment redundancy, controlling the security perimeter, and keeping skills acquisition cost efficient (more on this in The Value of Enterprise Architecture).

Portfolio management is discussed in depth in a subsequent Competency Category.

Business model _versus_ operating model
Figure 3. Business Model versus Operating Model

Infrastructure and Operations (I&O)

Finally, the Enterprise Architecture group often has a close relationship with I&O groups. This is because in organizations where operations is a shared service, the risks, and inefficiencies of technical fragmentation are often most apparent to the operations team. In organizations where operations is increasingly distributed to the application teams (“you build it, you run it”) the above may be less true.

Other staff organizations that may develop close relationships with Enterprise Architecture include vendor management and sourcing, risk management, compliance, and security. Notice that some of these have strong Governance[governance] connections (although we do not consider “governance” itself to be an organization, which is why it was not included in the discussion above).

The Value of Enterprise Architecture

Enterprise Architecture is well suited to demonstrate clear, quantifiable value to the organization. Architects are usually among the most experienced staff in the organization, and their function is necessary. Yet, demonstrating value takes some thought and effort. Statements like “promoting enterprise-wide thinking” might provoke skepticism. However, there are several outcomes provided by Enterprise Architecture and enterprise-wide thinking:

  • Shortening planning and decision-making (e.g., through curating information)

  • Curating a shared enterprise language and mental model

  • Increased speed of delivering new functionality

  • Reduced and simplified portfolios

  • Reducing duplication and rework

  • Reducing headcount (e.g., in processes)

Illustrated in Architecture Impacts on Enterprise Value is a high-level impact mapping representation.

Architecture value
Figure 4. Architecture Impacts on Enterprise Value

The diagram suggests a number of specific, measurable outcomes from typical Enterprise Architecture goals. Without exploring every line of value:

  • A reduced technology portfolio can and should result in improved sourcing, improved support, improved security, reduced IT staffing spend, and potentially reduced development time; for example, vendors may offer more favorable terms when their products are preferred standards throughout an organization – a smaller product portfolio is easier to secure

  • Better understanding of the current estate should reduce investigation times and outages, and reduce the risk of regulatory violations; for example, if regulators require evidence that employee medical records have not been removed from the country, architecture’s curating of that information will expedite the compliance response

  • Ensuring systems are adaptable (e.g., they have service interfaces) and resilient (they are designed for operability) should improve both time-to-market over the product’s lifecycle, and, ultimately, effectiveness in customer acquisition and retention

These are not intangible suggestions. We have previously studied the work and influence of Don Reinertsen, who emphasizes the critical importance of an economic model.

Some might say that architecture’s value is “intangible”. If you are tempted to say this, you should read Douglas Hubbard’s How to Measure Anything: Finding the Value of “Intangibles” in Business [Hubbard 2014].

We close this section by discussing two current value concepts and how architecture contributes (or detracts) from them:

  • Cost of delay

  • Technical debt

Reducing Cost of Delay

We have covered the concept of cost of delay previously, at the team and product level. However, this powerful concept can also be applied at higher levels. Cost of delay is not a concept familiar to most architects. It poses two important challenges:

  • How can architecture help reduce the cost of delay within the product portfolio?

  • How can architecture not, itself, introduce un-economical cost of delay?

As we have discussed previously, the definition of cost of delay is intuitive. It is the opportunity cost of not having a given product or service available for use: the foregone revenues, the cost of the workarounds and inefficiencies. If the architecture process becomes the critical path for a product or service’s release (a common experience), then the architecture process is responsible for that product’s or service’s cost of delay.

The cost of delay can take various forms, some of them significant. For example, suppose there is a need to demonstrate a product to key clients at a trade show. This could be the company’s best opportunity to develop business; the sales team estimates $12 million in funnel opportunities based on previous experience that should result in at least $2 million in sales in the year, with projections of another $1 million in maintenance and renewals. However, if the product is not ready, these benefits will not materialize. If everything else is ready, but the architecture process is delaying product readiness, then the architecture process is incurring $2 million in the cost of delay. This is bad. The architecture process is clearly impacting significant business objectives and revenue.

But the question still needs to be asked: “what benefits do we receive from having an architecture process?”. We discussed such benefits above. Are these benefits adding up to $2 million a year? No? Then your architecture process does not make good economic sense. On the other hand, what if the architects were kept out of the picture, and the product team chooses an untested technology, instead of re-using a well-known and reliable approach already proven for that company? What if that decision were the cause of missing the trade show? What if it can be shown that re-usable components identified as architecture standards were increasing the speed of delivery, and reducing the cost of delay, because they are reducing the need for product teams to perform risky (and yet redundant) R&D activities?

Quantifying these benefits across a portfolio is difficult, but should be attempted. Cost of delay can and should be calculated at the portfolio level, and this can provide “enterprise-level decision rules” that can help an organization understand the cost and value of operating model changes [Reinertsen 1997] including instituting processes (such as change management or Technology Lifecycle Management), or even the establishment of Enterprise Architecture itself.

Technical Debt Revisited

We touched on technical debt in Application Delivery's discussion of refactoring. Technical debt is a metaphor first introduced by Ward Cunningham [Cunningham 1992] in the context of software, and widely discussed in the industry. It can be applied more broadly at the portfolio level, and in that sense, is sometimes discussed in Enterprise Architecture. Debt exists in the form of obsolete products and technologies; redundant capabilities and systems; interfaces tightly-coupled where they should be loose and open, and many other forms.

Technical debt, like the cost of delay, can and should be quantified. We will discuss approaches to that in the Competency Category on portfolio management.

Scaling the Enterprise Mental Model

We have often referred to the concept of common ground through this document. The architecture supports common ground understanding at scale, by curating a shared mental model for the organization. In doing so, it enables the “right emergent behaviors” (as Adrian Cockcroft suggests). It also enables communication across diversity and may improve staffing flexibility and mobility among teams.

Evidence of Notability

Architecture has been a metaphor for digital systems strategy and design since the 1960s. It has given rise to multiple professional organizations, and frameworks, and much literature.

Limitations

Architecture (all practices) is encountering resistance from digital-first organizations and its messaging and value proposition needs to evolve.

Related Topics