Work Management and Lean
Product development drives a wide variety of work activities. As your product matures, you encounter both routine and non-routine work. Some of the work depends on other work getting done. Sometimes you do not realize this immediately. All of this work needs to be tracked.
Work management may start with verbal requests, emails, even postal mail. If you ask your colleague to do one thing, and they do not have anything else to do, it is likely that the two of you will remember. If you ask them to do four things over a few days, you might both still remember. But if you are asking for new things every day, it is likely that some things will get missed. You each might start keeping your own “to do” list, and this mechanism can handle a dozen or two dozen tasks. Consider an example of three people, each with their own to do list; see Work Flowing Across Three To-Do Lists.
In this situation, each person has their own “mental model” of what needs to be done, and their own tracking mechanism. We do not know how the work is being transmitted: emails, phone calls, hallway conversations. (“Say, Joe, there is an issue with Customer X’s bill, can you please look into it?”)
But what happens when there are three of you? Mary asks Aparna to do something, and in order to complete it, she needs something from Joe, whom Mary is also asking to complete other tasks. As an organization scales, this can easily lead to confusion and mistakes being made.
At some point, you need to formalize your model of the work, how it is described, and how it flows. This is such a fundamental problem in human society that many different systems, tools, and processes have been developed over the centuries to address it.
Probably the most important is the shared task reference point. What does this mean? The “task” is made “real” by associating it with a common, agreed artifact.
For example, a “ticket” may be created, or a "work order”. Or a “story”, written down on a sticky note. At our current level of understanding, there is little difference between these concepts. The important thing they have in common is an independent existence. That is, Mary, Aparna, and Joe might all change jobs, but the artifact persists independently of them. Notice also that the artifact – the ticket, the sticky note – is not the actual task, which is an intangible, consensus concept. It is a representation of this intangible “intent to perform”. We will discuss these issues of representation further in Information Management.
A complex IT-based system is not needed if you are all in the same room! (Nor for that matter a complex process framework, such as ITIL or COBIT®. There is a risk in using such frameworks at this stage of evolution – they add too much overhead for your level of growth.) It is also still too early for formal project management. The “project team” would be most or all of the organization, so what would be the point? A shared white board in a public location might be all that is needed; see Common List. This gives the team a “shared mental model” of who is doing what.
The design of the task board in Common List has some issues, however. After the team gets tired of erasing and rewriting the tasks and their current assignments, they might adopt something more like Simple Task Board.
The board itself might be a white board or a cork bulletin board with push pins. The notes could be sticky, or index cards. There are automated solutions as well. The tool does not really matter. The important thing is that, at a glance, the entire team can see its flow of work and who is doing what.
This is sometimes called a “Kanban board”, although David Anderson (originator of the Kanban software method [Anderson 2010] himself terms the basic technique a “card wall”. It also has been called a “Scrum Board”. The board at its most basic is not specific to either methodology. The term “Kanban” itself derives from Lean manufacturing principles; we will cover this in-depth in the next section. The basic board is widely used because it is a powerful artifact. Behind its deceptive simplicity are considerable industrial experience and relevant theory from operations management and human factors. However, it has scalability limitations. What if the team is not all in the same room? We will cover this and related issues in Context III.
The card wall or Kanban board is the first channel we have for “demand management”. Demand management is a term meaning “understanding and planning for required or anticipated services or work”. Managing day-to-day incoming work is a form of demand management. Capturing and assessing ideas for next year’s project portfolio (if you use projects) is also demand management at a larger scale.
To understand Kanban we should start with Lean. Lean is a term invented by American researchers who investigated Japanese industrial practices and their success in the 20th century. After the end of World War II, no-one expected the Japanese economy to recover the way it did. The recovery is credited to practices developed and promoted by Taiichi Ohno and Shigeo Shingo at Toyota [Ohno 1988]. These practices included:
Respect for people
Small batch sizes (driving towards “single piece flow”)
Decreased cycle time
Credit for Lean is also sometimes given to US thinkers such as W. Edwards Deming, Peter Juran, and the theorists behind the Training Within Industry methodology, each of whom played influential roles in shaping the industrial practices of post-war Japan.
Kanban is a term originating from Lean and the Toyota Production System. Originally, it signified a “pull” technique in which materials would only be transferred to a given workstation on a definite signal that the workstation required the materials. This was in contrast to “push” approaches where work was allowed to accumulate on the shop floor, on the (now discredited) idea that it was more “efficient” to operate workstations at maximum capacity.
Factories operating on a “push” model found themselves with massive amounts of inventory (work-in-process) in their facilities. This tied up operating capital and resulted in long delays in shipment. Japanese companies did not have the luxury of large amounts of operating capital, so they started experimenting with “single-piece flow”. This led to a number of related innovations, such as the ability to re-configure manufacturing machinery much more quickly than US factories were capable of.
David J. Anderson was a product manager at Microsoft who was seeking a more effective approach to managing software development. In consultation with Don Reinertsen (introduced below) he applied the original concept of Kanban to his software development activities [Anderson 2010].
Scrum (covered in the previous chapter) is based on a rhythm with its scheduled sprints; for example, every two weeks (this is called cadence). In contrast, Kanban is a continuous process with no specified rhythm. Work is “pulled” from the backlog into active attention as resources are freed from previous work. This is perhaps the most important aspect of Kanban – the idea that work is not accepted until there is capacity to perform it.
You may have a white board covered with sticky notes, but if they are stacked on top of each other with no concern for worker availability, you are not doing Kanban. You are accepting too much work-in-process, and you are likely to encounter a “high-queue state” in which work becomes slower and slower to get done. (More on queues below.)
Eliyahu Moshe Goldratt was an Israeli physicist and management consultant, best known for his pioneering work in management theory, including The Goal [Goldratt 2004], which is a best-selling business novel frequently assigned in MBA programs. It and Goldratt’s other novels have had a tremendous effect on industrial theory and, now, digital management. One of the best known stories in The Goal centers around a Boy Scout march. Alex, the protagonist, is struggling to save his manufacturing plant. He takes a troop of Scouts on a ten-mile hike. The troop has hikers of various speeds, yet the goal is to arrive simultaneously. As Alex tries to keep the Scouts together, he discovers that the slowest, most overweight scout (Herbie) also has packed an unusually heavy backpack. The contents of Herbie’s pack are redistributed, speeding up both Herbie and the troop.
This story summarizes the Goldratt approach: finding the “constraint” to production (his work as a whole is called the Theory of Constraints). In Goldratt’s view, a system is only as productive as its constraint. At Alex’s factory, it is found that the “constraint” to the overall productivity issues is the newest computer-controlled machine tool – one that could (in theory) perform the work of several older models but was now jeopardizing the entire plant’s survival. The story in this novelization draws important parallels with actual Lean case studies on the often-negative impact of such capital-intensive approaches to production.
Joint activity depends on interpredictability of the participants’ attitudes and actions. Such interpredictability is based on common ground – pertinent knowledge, beliefs, and assumptions that are shared among the involved parties. [Klein et al. 2005]
“Common Ground and Coordination in Joint Activity"
The above quote reflects one of the most critical foundations of team collaboration: a common ground, a base of “knowledge, beliefs, and assumptions” enabling collaboration and coordination. Common ground is an essential quality of successful teamwork, and we will revisit it throughout the book. There are many ways in which common ground is important, and we will discuss some of the deeper aspects in terms of information in Information Management. Whether you choose Scrum, Kanban, or choose not to label your work management at all, the important thing is that you are creating a shared mental model of the work: its envisioned form and content, and your progress towards it.
Below, we will discuss:
Visualization of work
The concept of Andon
The definition of done
Time and space shifting
Visualization is a good place to introduce the idea of common ground.
As simple as the white board is, it makes work-in-process continuously visible, it enforces work-in-process constraints, it creates synchronized daily interaction, and it promotes interactive problem solving. Furthermore, teams evolve methods of using white boards continuously, and they have high ownership in their solution. In theory, all this can be replicated by a computer system. In practice, I have not yet seen an automated system that replicates the simple elegance and flexibility of a manual system. [Reinertsen 2009]
Principles of Product Development Flow
Why are shared visual representations important? Depending on how you measure, between 40% to as much as 80% of the human cortex is devoted to visual processing. Visual processing dominates mental activity, consuming more neurons than the other four senses combined [Sells 1957]. Visual representations are powerful communication mechanisms, well suited to our cognitive abilities.
This idea of common ground, a shared visual reference point, informing the mental model of the team, is an essential foundation for coordinating activity. This is why card walls or Kanban boards located in the same room are so prevalent. They communicate and sustain the shared mental model of a human team. A shared card wall, with its two dimensions and tasks on cards or sticky notes, is more informative than a simple to-do list (e.g., in a spreadsheet). The cards occupy two-dimensional space and are moved over time to signify activity, both powerful cues to the human visual processing system.
Similarly, monitoring tools for systems operation make use of various visual clues. Large monitors may be displayed prominently on walls so that everyone can understand operational status. Human visual orientation is also why Enterprise Architecture persists. People will always draw to communicate. (See also Modeling and Visualization.)
Card walls and publicly displayed monitors are both examples of information radiators. The information radiator concept derives from the Japanese concept of Andon, important in Lean thinking.
The Andon cord (not to be confused with Andon in the general sense) is another well-known concept in Lean manufacturing. It originated with Toyota, where line workers were empowered to stop the production line if any defective materials or assemblies were encountered. Instead of attempting to work with the defective input, the entire line would shut down, and all concerned would establish what had happened and how to prevent it. The concept of Andon cord concisely summarizes the Lean philosophy of employee responsibility for quality at all levels [Ohno 1988]. Where Andon is a general term for information radiator, the Andon cord implies a dramatic response to the problems of flow – all progress is stopped, everywhere along the line, and the entire resources of the production line are marshaled to collaboratively solve the issue so that it does not happen again. As Toyota thought leader Taiichi Ohno states:
Stopping the machine when there is trouble forces awareness on everyone. When the problem is clearly understood, improvement is possible. Expanding this thought, we establish a rule that even in a manually operated production line, the workers themselves should push the stop button to halt production if any abnormality appears.
Andon and information radiators provide an important stimulus for product teams, informing priorities and prompting responses. They do not prescribe what is to be done; they simply indicate an operational status that may require attention.
As work flows through the system performing it, understanding its status is key to managing it. One of the most important mechanisms for doing this is to define what is meant by “done simply”. The Agile Alliance states: “The team agrees on, and displays prominently somewhere in the team room, a list of criteria which must be met before a product increment, often a user story, is considered “done” [Agile Alliance 2015]. Failure to meet these criteria at the end of a sprint normally implies that the work should not be counted toward that sprint’s velocity. There are various patterns for defining “done”; for example, Thoughtworks recommends that the business analyst and developer both must agree that some task is complete (it is not up to just one person). Other companies may require peer code reviews [Narayam 2015a]. The important point is that the team must agree on the criteria.
This idea of defining “done” can be extended by the team to other concepts such as “blocked”. The important thing is that this is all part of the team’s shared mental model, and is best defined by the team and its customers. (However, governance and consistency concerns may arise if teams are too diverse in such definitions.)
At some point, your team will be faced with the problems of time and/or space shifting. People will be on different schedules, or in different locations, or both. There are two things we know about such working relationships. First, they lead to sub-optimal team communications and performance. Second, they are inevitable.
The need for time and space shifting is one of the major drivers for more formalized IT systems. It is difficult to effectively use a physical Kanban board if people are not in the office. The outcome of the daily standup needs to be captured for the benefit of those who could not be there.
However, acceptance of time and space shifting may lead to more of it, even when it is not absolutely required. Constant pressure and questioning are recommended, given the superior bandwidth of face-to-face communication in the context of team collaboration.
But not all work requires the same degree of collaboration. While we are still not ready for full-scale process management, at this point in our evolution, we likely will encounter increasing needs to track customer or user service interactions, which can become quite numerous even for small, single-team organizations. Such work is often more individualized and routine, not requiring the full bandwidth of team collaboration. We will discuss this further with the topic of the help or service desk, later in this Competency Area.
Even at this stage of our evolution, with just one co-located collaborative team, it is important to consider work-in-process and how to limit it. One topic we will emphasize throughout the rest of this document is queuing.
A queue, intuitively, is a collection of tasks to be done, being serviced by some worker or resource in some sequence; for example:
Feature “stories” being developed by a product team
Customer requests coming into a service desk
Requests from a development team to an infrastructure team for services (e.g., network or server configuration, consultations, etc.)
Queuing theory is an important branch of mathematics used extensively in computing, operations research, networking, and other fields. It is a topic getting much attention of late in the Agile and related movements, especially as it relates to digital product team productivity.
The amount of time that any given work item spends in the queue is proportional to how busy the servicing resource is. The simple formula, known as Little’s Law, is:
In other words, if you divide the percentage of busy time for the resource by its idle time, you see the average wait time. So, if a resource is busy 40% of the days, but idle 60% of the days, the average time you wait for the resource is:
Conversely, if a resource is busy 95% of the time, the average time you will wait is:
If you use a graphing calculator, you see the results shown in Time in Queue Increases Exponentially with Load.
Notice how the wait time approaches infinity as the queue utilization approaches 100%. And yet, full utilization of resources is often sought by managers in the name of “efficiency”. These basic principles are discussed by Gene Kim et al. in The Phoenix Project [Kim et al. 2013], Chapter 23, and more rigorously by Don Reinertsen in The Principles of Product Development Flow [Reinertsen 2009], Chapter 3. A further complication is when work must pass through multiple queues; wait times for work easily expand to weeks or months. Such scenarios are not hypothetical, they are often seen in the real world and are a fundamental cause of IT organizations getting a bad name for being slow and unresponsive. Fortunately, Digital Practitioners are gaining insight into these dynamics and matters are improving across the industry.
Understanding queuing behavior is critical to productivity. Reinertsen suggests that poorly managed queues contribute to:
Longer cycle time
These issues were understood by the pioneers of Lean manufacturing, an important movement throughout the 20th century. One of its central principles is to limit work-in-process. Work-in-process is obvious on a shop floor because physical raw materials (inventory) are quite visible.
Don Reinertsen developed the insight that product design and development had an invisible inventory of “work-in-process” that he called design-in-process. Just as managing physical work-in-process on the factory floor is key to a factory’s success, so correctly understanding and managing design-in-process is essential to all kinds of R&D organizations – including digital product development; e.g., building software. In fact, because digital systems are largely invisible even when finished, understanding their work-in-process is even more challenging.
It is easy and tempting for a product development team to accumulate excessive amounts of work-in-process. And, to some degree, having a rich backlog of ideas is an asset. But, just as some inventory (e.g., groceries) is perishable, so are design ideas. They have a limited time in which they might be relevant to a customer or a market. Therefore, accumulating too many of them at any point in time can be wasteful.
What does this have to do with queuing? Design-in-process is one form of queue seen in the digital organization. Other forms include unplanned work (incidents and defects), implementation work, and many other concepts we will discuss in this chapter.
Regardless of whether it is a “requirement”, a “user story”, an “epic”, “defect", “issue”, or “service request”, you should remember it is all just work. It needs to be logged, prioritized, assigned, and tracked to completion. Queues are the fundamental concept for doing this, and it is critical that digital management specialists understand this.
Multi-tasking (in this context) is when a human attempts to work on diverse activities simultaneously; for example, developing code for a new application while also handling support calls. There is broad agreement that multi-tasking destroys productivity, and even mental health [Cherry 2021]. Therefore, minimize multi-tasking. Multi-tasking in part emerges as a natural response when one activity becomes blocked (e.g., due to needing another team’s contribution). Approaches that enable teams to work without depending on outside resources are less likely to promote multi-tasking. Queuing and work-in-process therefore become even more critical topics for management concern as activities scale up.
So, do you choose Scrum, Kanban, both, or neither? We can see in comparing Scrum and Kanban that their areas of focus are somewhat different:
Scrum is widely adopted in industry and has achieved a level of formalization, which is why Scrum training is widespread and generally consistent in content
Kanban is more flexible but this comes at the cost of more management overheads; it requires more interpretation to translate to a given organization’s culture and practices
As Scrum author Ken Rubin notes: “Scrum is not well suited to highly interrupt-driven work” [Rubin 2012]; Scrum on the service desk does not work (but if your company is too small, it may be difficult to separate out interrupt-driven work; we will discuss the issues around interrupt-driven work further in Digital Operations
Finally, hybrids exist (Ladas’ “Scrumban” [Ladas 2009]
Ultimately, instead of talking too much about “Scrum” or “Kanban”, it is worth looking more deeply into their fundamental differences. We will return to this topic in the section on Lean Product Development.
Finish what you start, if you can, before starting anything else – when you work on three things at once, the multi-tasking wastes time, and it takes you three times longer to get any one of the things done (more on multi-tasking in this chapter)
Infinitely long to-do lists (backlog) sap motivation – consider limiting backlog as well as work-in-process
Visibility into work-in-process is important for the collective mental model of the team
There are deeper philosophical and cultural qualities to Kanban beyond workflow and queuing. Anderson and his colleagues continue to evolve Kanban into a more ambitious framework. Mike Burrows identifies the following key principles [Burrows 2014]:
Start with what you do now
Agree to pursue evolutionary change
Initially, respect current processes, roles, responsibilities, and job titles
Encourage acts of leadership at every level in your organization – from individual contributor to senior management
Make policies explicit
Implement feedback loops
Improve collaboratively, evolve experimentally (using models and the scientific method)
Evidence of Notability
Work and task management is a fundamental problem in human organizations. It is the foundation of workflow and BPM. Lean generally is one of the most significant currents of thought in modern management; see Ohno 1988, Womack & Jones 2003, Womack 1990, Rother & Shook 2003, and Rother 2010. Kanban is widely discussed at Agile and DevOps conferences. Using a lightweight, generalized task tracking tool, often physical, is seen in digital organizations worldwide.
Kanban’s generalized workflow does not scale to complex processes with many steps and decision points. This will be covered further in the section on workflow management. Not all activities reduce well to a list of tasks. Some are more intangible and outcome-focused. As Bjarne Stroustrup, the inventor of C++, stated: “The idea of software development as an assembly line manned by semi-skilled interchangeable workers is fundamentally flawed and wasteful” [Stroustrup 2010]. It is critical to distinguish Lean as applied to digital systems development (as a form of applied R&D) versus Lean in its manufacturing aspects. Reinertsen’s contributions (Reinertsen 1997 and Reinertsen 2009) are unique and notable in this regard and are discussed in the next section.