Process Management

Description

Dictionary.com defines process as “a systematic series of actions directed to some end …​ a continuous action, operation, or series of changes taking place in a definite manner”. We saw the concept of “process” start to emerge in Work Management, as work becomes more specialized and repeatable and our card walls got more complicated.

We have discussed work management, which is an important precursor of process management. Work management is less formalized; a wide variety of activities are handled through flexible Kanban-style boards or “card walls” based on the simplest “process” of:

  • To do

  • Doing

  • Done

However, the simple card wall/Kanban board can easily become much more complex, with the addition of swim lanes and additional columns, including holding areas for blocked work. As we discussed in Work Management, when tasks become more routine, repeatable, and specialized, formal process management starts to emerge. Process management starts with the fundamental capability for coordinated work management, and refines it much further.

Process, in terms of “business process”, has been a topic of study and field for professional development for many years. Pioneering BPM authors such as Michael Hammer [Hammer & Champy 2001] and Geary Rummler [Rummler & Brache 1995] have had an immense impact on business operations, with concepts such as Hammer’s Business Process Re-engineering (BPR). BPR initiatives are intended to identify waste and streamline processes, eliminating steps that no longer add value. BPR efforts often require new or enhanced digital systems.

In the Lean world, value stream mapping represents a way of understanding the end-to-end flow of value, typically in a manufacturing or supply chain operational context [Rother & Shook 2003].

The Lean Enterprise Institute defines “value stream” as: “All of the actions, both value-creating and non-value-creating, required to bring a product from concept to launch (also known as the development value stream) and from order to delivery (also known as the operational value stream). These include actions to process information from the customer and actions to transform the product on its way to the customer”.

Making value streams visible helps understand current conditions and identify issues and problems. A Value Stream Map (VSM) is a simple diagram of every step involved in the material and information flows needed to deliver value. The Lean Enterprise Institute indicates that: “Value Stream Maps can be drawn for different points in time as a way to raise consciousness of opportunities for improvement. A current state map follows a product’s path from order to delivery to determine the current conditions. A future state map deploys the opportunities for improvement identified in the current state map to achieve a higher level of performance at some future point”.

Toyota considers a clear process vision, or “target condition”, to be the most fundamental objective for improving operations; see Rother 2010, Chapters 5 and 6. Designing processes, improving them, and using them to improve overall performance is an ongoing activity in most, if not all organizations. VSM is a powerful tool that can help define a clear process vision.

In your company, work has been specializing. A simple card-based Kanban approach is no longer sufficient. You are finding that some work is repetitive, and you need to remember to do certain things in certain orders. For example, a new human resources manager was hired and decided that a sticky note of “hire someone new for us” was not sufficient. As she pointed out to the team, hiring employees was a regulated activity, with legal liability, requiring confidentiality, that needed to follow a defined sequence of tasks:

  • Establishing the need and purpose for the position

  • Soliciting candidates

  • Filtering the candidates

  • Selecting a final choice

  • Registering that new employee in the payroll system

  • Getting the new employee set up with benefits providers (insurance, retirement, etc.)

  • Getting the new employee working space, equipment, and access to systems

  • Training the new employee in organizational policies, as well as any position-specific orientation

The sales, marketing, and finance teams have similarly been developing ordered lists of tasks that are consistently executed as end-to-end sequences. And even in the core digital development and operations teams, they are finding that some tasks are repetitive and need documentation so they are performed consistently.

Your entire digital product pipeline may be called a “process”. From initial feature idea through production, you seek a consistent means for identifying and implementing valuable functionality. Sometimes this work requires intensive, iterative collaboration and is unpredictable (e.g., developing a user interface); sometimes, the work is more repeatable (e.g., packaging and releasing tested functionality).

You are hiring more specialized people with specialized backgrounds. Many of them enter your organization and immediately ask process questions:

  • What is your security process?

  • What is your architecture process?

  • What is your portfolio process?

You have not had these conversations before. What do they mean by these different “processes”? They seem to have some expectation based on their previous employment, and if you say “we do not have one” they tend to be confused. You are becoming concerned that your company may be running some risk, although you also are wary that “process” might mean slowing things down, and you cannot afford that.

However, some team members are cautious of the word “process”. The term “process police” arises in an unhappy way.

“Are we going to have auditors tracking whether we filled out all our forms correctly?”, one team member asks.

“We used to get these “process consultants” at my old company, and they would leave piles of three-ring binders on the shelf that no-one ever looked at,” another person says.

“I cannot write code according to someone’s recipe!” says a third with some passion, and to general agreement from the developers in the room.

The irony is that digital products are based on process automation. The idea that certain tasks can be done repeatably and at scale through digitization is fundamental to all use of computers. The digital service is fundamentally an automated process, one that can be intricate and complicated. That is what computers do. But, process management also spans human activities, and that is where things get more complex.

Processes are how we ensure consistency, repeatability, and quality. You get expected treatment at banks, coffee shops, and dentists because they follow processes. IT systems often enable processes – a mortgage origination system is based on IT software that enforces a consistent process. IT management itself follows certain processes, for all the same reasons as above.

However, processes can cause problems. Like project management, the practice of process management is under scrutiny in the new Lean and Agile-influenced digital world. Processes imply queues, and in digital and other product development-based organizations, this means invisible work-in-process. For every employee you hire who expects you to have processes, another will have bad process experiences at previous employers. Nevertheless, process remains an important tool in your toolkit for organization design.

Process is a broad concept used throughout business operations. The coverage here is primarily about process as applied to the digital organization. There is a bit of a recursive paradox here; in part, we are talking about the process by which business processes are analyzed and sometimes automated. By definition, this overall “process” (you could call it a meta-process) cannot be made too prescriptive or predictable.

The concept of “process” is important and will persist through Digital Transformation. We need a robust set of tools to discuss it. This Competency Area will break the problem into a lifecycle of:

  • Process conception

  • Process content

  • Process execution

  • Process improvement

Although we do not preface these topics with “Agile” or “Lean”, bringing these together with related perspectives is the intent of this Competency Category.

Process Conception

Processes can provoke reactions when their value is not understood, or has decayed. Such reactions are commonplace in social media (and even well-regarded professional books), but we need a more objective and rational approach to understand the pros and cons of processes. We have seen a number of neutral concepts towards this end from authors such as Don Reinertsen and Diane Strode:

A process is a technique, a tool, and no technique should be implemented without a thorough understanding of the organizational context. Nor should any technique be implemented without rigorous, disciplined follow-up as to its real effects, both direct and indirect. Many of the issues with process come from a cultural failure to seek an understanding of the organization needs in objective terms such as these. We will think about this cultural failure more in the discussion of Toyota Kata.

A skeptical and self-critical, “go and see” approach is, therefore, essential. Too often, processes are instituted in reaction to the last crisis, imposed top-down, and rarely evaluated for effectiveness. Allowing affected parties to lead a process re-design is a core Lean principle (kaizen). On the other hand, uncoordinated local control of processes can also have destructive effects as discussed below.

Process Execution

Since our initial discussions in Work Management on work management, we find ourselves returning full circle. Despite the various ways in which work is conceived, funded, and formulated, in the end “it’s all just work”. The digital organization must retain a concern for the “human resources” (that is, people) who find themselves at the mercy of:

The Lean movement manages through minimizing waste and over-processing. This means both removing unnecessary steps from processes, and eliminating unnecessary processes completely when required. Correspondingly, the processes that remain should have high levels of visibility. They should be taken with the utmost seriousness, and their status should be central to most people’s awareness. This is the purpose of Andon.

From workflow tools to collaboration and digital exhaust. One reason process tends to generate friction and be unpopular is the poor usability of workflow tools. Older tools tend to present myriads of data fields to the user and expect a high degree of training. Each state change in the process is supposed to be logged and tracked by having someone sign in to the tool and update status manually.

By contrast, modern workflow approaches take full advantage of mobile platforms and integration with technology like chatrooms and ChatOps. Mobile development imposes higher standards for User Experience (UX) design, which makes tracking workflow somewhat easier. Integrated software pipelines that integrate Application Lifecycle Management (ALM) and/or project management with source control and build management are increasingly gaining favor. For example:

  • A user logs a new feature request in the ALM tool

  • When the request is assigned to a developer, the tool automatically creates a feature branch in the source control system for the developer to work on

  • The developer writes tests and associated code and merges changes back to the central repository once tests are passed successfully

  • The system automatically runs build tests

  • The ALM tool is automatically updated accordingly with completion if all tests pass

See also the previous discussion of ChatOps, which similarly combines communication and execution in a low-friction manner, while providing rich digital exhaust as an audit trail.

In general, the idea is that we can understand digital processes not through painful manual status updates, but rather through their digital exhaust – the data byproducts of people performing the value-add day-to-day work, at speed, and with the flow instead of constant delays for approvals and status updates.

Measuring Process

One of the most important reasons for repeatable processes is so that they can be measured and understood. Repeatable processes are measured in terms of:

  • Speed

  • Effort

  • Quality

  • Variation

  • Outcomes

at the most general level and, of course, all of those measurements must be defined much more specifically depending on the process. Operations (often in the form of business processes) generate data, and data can be aggregated and reported on. Such reporting serves as a form of feedback for management and even governance. Examples of metrics might include:

  • Quarterly sales as a dollar amount

  • Percentage of time a service or system is available

  • Number of successful releases or pushes of code (new functionality)

Measurement is an essential aspect of process management but must be carefully designed. Measuring processes can have unforeseen results. Process participants will behave according to how the process is measured. If a help desk operator is measured and rated on how many calls they process an hour, the quality of those interactions may suffer. It is critical that any process Key Performance Indicators (KPIs) be understood in terms of the highest possible business objectives. Is the objective truly to process as many calls as possible? Or is it to satisfy the customer so they need not turn to other channels to get their answers?

A variety of terms and practices exist in process metrics and measurement, such as:

  • The Balanced Scorecard

  • The concept of a metrics hierarchy

  • Leading versus lagging indicators

The Balanced Scorecard is a commonly seen approach for measuring and managing organizations. First proposed by Robert S. Kaplan and David P. Norton [Kaplan & Norton 1992] in the Harvard Business Review, the Balanced Scorecard groups metrics into the following subject areas:

  • Financial

  • Customer

  • Internal business processes

  • Learning and growth

Metrics can be seen as “lower” versus “higher” level. For example, the metrics from a particular product might be aggregated into a hierarchy with the metrics from all products, to provide an overall metric of product success. Some metrics are perceived to be of particular importance for business processes, and thus may be termed KPIs. Metrics can indicate past performance (lagging), or predict future performance (leading).

The Disadvantages of Process

Netflix CTO Reed Hastings, in an influential public presentation Netflix Culture: Freedom and Responsibility, presents a skeptical attitude towards process. In his view, process emerges as a result of an organization’s talent pool becoming diluted with growth, while at the same time its operations become more complex.

Hastings observes that companies that become overly process-focused can reap great rewards as long as their market stays stable. However, when markets change, they also can be fatally slow to adapt. The Netflix strategy is to focus on hiring the highest-performance employees and keeping processes minimal. They admit that their industry (minimally regulated, creative, non-life-critical) is well suited to this approach [Hastings 2009].

The Pitfall of Process “Silos”

One organization enthusiastically embraced process improvement, with good reason: customers, suppliers, and employees found the company’s processes slow, inconsistent, and error-prone. Unfortunately, they were so enthusiastic that each team defined the work of their small group or department as a complete process. Of course, each of these was, in fact, the contribution of a specialized functional group to some larger, but unidentified, processes. Each of these “processes” was “improved” independently, and you can guess what happened. Within the boundaries of each process, improvements were implemented that made work more efficient from the perspective of the performer. However, these mini-processes were efficient largely because they had front-end constraints that made work easier for the performer but imposed a burden on the customer or the preceding process. The attendant delay and effort meant that the true business processes behaved even more poorly than they had before. This is a common outcome when processes are defined too “small”. Moral: Don’t confuse sub-processes or activities with business processes. [Sharp & Patrick 2008]
— Alec Sharp and Patrick McDermott
Workflow Modeling: Tools for Process Improvement and Applications Development

The above quote well illustrates the dangers of combining local optimization and process management. Many current authors speak highly of self-organizing teams, but self-organizing teams may seek to optimize locally. Process management was originally intended to overcome this problem, but modeling techniques can be applied at various levels, including within specific departments. This is where enterprise Business Architecture can assist, by identifying these longer, end-to-end flows of value and highlighting the hand-off areas, so that the process benefits the larger objective. More information about Business Architecture can be found in the Open Business Architecture (O-BA) Part II: Business Architecture Capabilities, Value Stages, and Activities, a Standard of The Open Group [P171], Section 3.2, “Way of Thinking”.

Process Proliferation

Another pitfall we cover here is that of process proliferation. Process is a powerful tool. Ultimately it is how value is delivered. However, too many processes can have negative results on an organization. One thing often overlooked in process management and process frameworks is any attention to the resource impacts of the process. This is a primary difference between project and process management; in process management (both theory and frameworks), resource availability is, in general, assumed.

More advanced forms of process modeling and simulation such as “discrete event simulation” [Sweetser 1999] can provide insight into the resource demands for processes. However, such techniques require specialized tooling and are not part of the typical BPM practitioner’s skillset.

Many enterprise environments have multiple cross-functional processes such as:

  • Service requests

  • Compliance certifications

  • Asset validations

  • Provisioning requests

  • Capacity assessments

  • Change approvals

  • Training obligations

  • Performance assessments

  • Audit responses

  • Expense reporting

  • Travel approvals

These processes sometimes seem to be implemented on the assumption that enterprises can always accommodate another process. The result can be a dramatic overburden for digital staff in complex environments. A frequently-discussed responsibility of Scrum masters and product owners is to “run interference” and keep such enterprise processes from eroding team cohesion and effectiveness. It is, therefore, advisable to at least keep an inventory of processes that may impose demand on staff, and understand both the aggregate demand as well as the degree of multi-tasking and context-switching that may result (as discussed in Work Management). Thorough automation of all processes to the maximum extent possible can also drive value, to reduce latency, load, and multi-tasking.

Rather than a simplistic view of “process bad” or “process good”, it is better to view process as simply a coordination approach. It is a powerful one with important disadvantages. It should be understood in terms of coordination contexts such as time and space shifting and predictability.

It is also important to consider lighter-weight variations on process, such as case management, checklists, and the submittal schedule.

Evidence of Notability

Process management has a long history across business and organizational management in general. Notable works include Michael Hammer’s Re-engineering the Corporation [Hammer & Champy 2001] and Geary Rummler and Alan Brache’s Improving Performance [Rummler & Brache 1995].

Limitations

Process management, like its earlier precursor (in this document) workflow management, is not well suited for higher-variability, higher-touch tasks.

Related Topics