Supporting Collaborating Agile Teams

Dealing with large, complex systems requirement is difficult in any methodology, and is perhaps the biggest challenge in adopting Agile ways of working. By their very nature, these large systems are not very Agile because the large number of dependencies they exhibit slows down change.

Some Agile methodologists would say you simply have to break down those systems into chunks that are each manageable by a single team. Microservices are an example of such an approach but, as anyone who has seen them in action knows, the result is often that the complexity that used to be hidden within the single large system now shows up in the connections and communication patterns between these microservices.

As Conway’s Law [7] states, organizations design systems that mirror their communication structures. This is quite obvious since if your system architecture cuts across these communication structures, the extra effort of communicating between these teams often leads them to redesign the system structure so that each team has their own area of responsibility in the system architecture (e.g., a microservice), or to restructure teams to align with the architecture components.

From an agility point of view, such component teams are not always optimal though. If you have, for example, a three-layer architecture with a web front end, business logic layer, and database, you may be tempted to have three corresponding teams. Many features, however, then require all three teams to collaborate, since features often need something in the interface, some business logic, and some storage. This will lead to teams having to wait for each other, lowering your speed of change and agility.

Agile methods therefore tend to favor cross-component, cross-disciplinary feature teams consisting of experts on each of the parts of the architecture that are potentially touched by a feature, and often also including, for example, experts on User Experience (UX), testing, and DevOps. Such feature teams can tackle a feature end-to-end, at the expense of some efficiency. Moreover, if some specialized expertise (on security, database technology, etc.) is scarce and needed by multiple teams, you run into limitations.

Since both approaches have their pros and cons, most organizations tend to have a combination of feature and component teams based on the specifics of their solutions. Deciding what works best strongly depends on your architecture.

With ArchiMate models, you can serve both approaches. In fact, the subdivision between (active) structure and behavior in ArchiMate models is quite similar to that between components and features. In the end, you need both, but you can choose to start with one or the other in organizing the work.

For a newly to-be-developed solution we recommend starting your models from a feature perspective, expressed in ArchiMate behavior concepts and using, for example, services, functions, and processes. Which application components (or business roles and actors in case of a non-IT-based part of the solution) perform that behavior, and via which interfaces you can get these services, are next steps. On the contrary, when creating models of an existing solution, it is often easier to start with the components and their interfaces, since those are often more "visible" and concrete than services and functions, which are somewhat more abstract and may require some deeper analysis of these components.

However you structure your teams and architecture, in all cases there is a need for communication that increases with organization (and system) size. On the one hand, this is "vertical", to convey the enterprise vision so that teams align in the same direction, and to provide feedback and contribute innovative ideas up the food chain from development to management. A clear line of sight helps ensure that goals are clear to everyone involved and priorities are aligned with these goals. Although organizations may use all kinds of tools to keep track of their development initiatives, architecture modeling is the most efficient way to illustrate the connections between strategy, capabilities, and development portfolio items.

And vice versa, feedback on the feasibility and desirability of those goals comes from the "boots on the ground" in Agile teams, who are in direct contact with users, know about the possibilities and limitations of technology, and provide improvements, innovations, and refinements to adapt to local circumstances. Architecture models provide such an unambiguous connection between motivation and realization, and allow you to trace dependencies and effects of changes in priorities, planning, etc.

Equally important is the lateral communication between teams that work on parts of the solution that need to communicate, provide an integral user experience, share resources, and in general depend on each other. A common technique to draw these dependencies is using red woolen strings on a Kanban or Scrum board, but that will not scale up across larger teams and teams of teams (although spanning those threads across your work floor from board to board would be interesting!). Moreover, you cannot easily use this in any remote setting. Again, architecture models are of help here, clearly showing how different components are interconnected, how business processes rely on IT systems, which users are working with which applications, where the data is stored, etc.

Most Agile methodologies include well-defined ceremonies. Some of them, like Program Increment (PI) planning, are a good moment to share and build common understanding. Architecture models can be used to prepare such ceremonies and keep track of key information and decisions. Of course, such architecture views should be designed for and with key team members such as the product owner to ensure that the visualizations produced will be adopted and used inside the teams. An example is shown in Example Input for PI Planning, providing teams with high-level scoping information for a PI, focused on a specific feature, an application service of a risk assessment application that is to be updated to support new and updated business functions.

Next to supporting the communication between different groups to achieve a common understanding, you can also analyze these models at a deeper level. Dependency analysis is one example; for instance, how does a change in the sprint planning of one team (for example, pushing a feature out a few sprints) impact other teams, the Agile Release Train, and possibly up to important business goals and stakeholders? Cost or resource analysis might be another; for example, are we putting our efforts where the highest strategic or customer value is to be captured?

The vertical communication in an Agile organization is considered on the one hand, aligning the work being done with the overall vision of the enterprise, and on the other with providing a feedback loop from practice back into that vision.

This vision is important to define priorities and planning. What are the areas in which the organization wants to invest? What are the main epics to evolve?

As discussed in Modeling the Intent: Themes, Epics, Features, and Stories as ArchiMate Motivation Elements, the architectural intent can be expressed in the ArchiMate motivation concepts. In addition, strategy concepts such as capability and resource can be used to express a high-level solution structure in terms of what the enterprise wants to be able to do, and the means it needs to do it.

This way, you can define a high-level solution architecture that gives a general direction to the Agile teams without constraining them too much in the specifics of their implementation. Drilling down from capabilities and resources gets us to the level of the ArchiMate core elements, such as business process and application component. These can be used to provide more concrete direction and models at this level are typically the purview of roles like domain and system architects. Using these models, they can provide a more concrete and detailed shared architectural vision to the teams, and ensure coherence within an Agile Release Train.

These models can also be used by teams to validate their own designs and, if they deviate, to discuss why this is. Perhaps the system architecture is not feasible; some parts will take more time than expected so the sprint and release planning need to be revised, or there is a new technology available that provides important benefits. This feedback, in turn, may lead to refinements or changes to the system architecture and/or backlog planning, and could potentially even influence higher-level goals and priorities.

To support the lateral communication between teams, architecture models can be helpful in a number of ways.

First of all, an important consideration is the division of work between teams. Some of this is about clustering features into sensible groups that can be tackled coherently. Having a good overview of the various parts of the system on which the teams are working is an immense help in structuring the work.

For both component and feature teams, knowing who is working on which parts of a solution is essential to avoid conflicts, gaps, overlaps, and rework. Typically, each team is fully responsible for the inner workings of what they build; i.e., its software architecture and design. They can use whatever they choose to design that, ranging from sketches on whiteboards to some simple ArchiMate models, and different teams may work differently. How these relate to each other is where things should be standardized in some way, so that all teams have the same understanding. This is where, for example, a set of ArchiMate views of the communication between application components in your landscape can be a central design artifact. An example of this is shown in Target Application Landscape with Flow Relationships.

In such an overview, you could also use colors to show which teams are responsible for which parts of the landscape, and to identify gaps or overlaps that need to be addressed.

Zooming in brings us at the level where individual teams are responsible. They may want to express, for instance, more detail about the expected internal behavior of the components for which they are responsible. In Data Warehousing Functionality we see an example of that, with the data processing performed by the "Data Warehousing Solution" in support of the epic of "Detailed Insights in Customer Behavior" and the "Data Driven Insurance Capability" mentioned in ArchiSurance Architecture Intent.

Zooming in on a specific application component or technology element, showing the services it provides, and how it communicates with its environment can help teams to define connections to such a component. This can potentially be used to define interface agreements between providing and receiving teams, so that both sides know exactly how their work is going to connect.

In Data Acquisition API, we see an example that illustrates how the "Data Acquisition" technology function from Data Warehousing Functionality is implemented, the "Communication" services this function provides to the "Internet of Things (IoT) Device" that provides this data, and the "Representational State Transfer (REST) Application Programming Interface (API)" that is used to offer these services. These devices collect data on customer behavior that is used to fine-tune the insurance premiums of ArchiSurance. An illustrative example is a black box in your car that monitors your driving behavior, so that if you drive smoothly and do not speed, you may get a discount on your auto insurance.

Data Acquisition API starts from a component perspective of the world, so this would perhaps be most relevant if you have component teams. In the case of feature teams, focusing first on the application services that provide these features to users may make more sense. Drilling down from the application services leads you to the application functions necessary for providing them, and then you can identify which application components are involved in performing these functions.

Since in Agile development communication is more important than detailed documentation, the use of models should be focused on expressing the ideas behind an architecture rather than the details of a design. This is a great fit with the principles behind the ArchiMate language. From the start, the ArchiMate modeling notation was designed as a lean-and-mean language with minimal overhead, aimed at expressing the essence rather than the details of an enterprise. Its focus on communication can even be seen in the construction of the language, which intentionally mimics the structure of human language with its subjects, verbs, and objects, reflected in the ArchiMate active structure, behavior, and passive structure concepts.

Some may object that ArchiMate symbols look too technical for non-IT audiences. The language was designed to resemble other modeling languages with boxes, arrows, and icons to make it easy to learn for anyone with some modeling experience, and this makes it look a bit "techy". But the definition of the language also separates the content from the visualization of a model and stresses the importance of stakeholder-oriented viewpoints. The standard describes a viewpoint mechanism to support the creation of such viewpoints. However, given the breadth of stakeholders out there, it would not be feasible to standardize these viewpoints themselves. Different audiences need different depictions of architecture content, from simple lists to detailed diagrams, and from simple, colorful pictures to formally precise specifications. This is where the expertise of the architect comes in, creating the appealing visualizations of underlying models that best serve their audience.