How One Team is Making the Transition to Agile

It can often be difficult for organizations to adopt agile methodologies in their truest sense. Water-Scrum-Fall is a hybrid approach of waterfall and agile practices, blended together in a manner that facilitates the transition to agile.

A waterfall-based approach to software and product development tends to treat the entire activity as a single segment with a definitive start and a definitive end. In very large and complex projects, this can leave little room for flexibility (i.e. iteration of some component of that project) and cause headaches when the project is “complete,” yet not delivered as expected. On the contrary, scrum (or agile) practices break these complex projects down into smaller components that are developed rapidly and iterated on in a more isolated, controlled environment. These agile practices require a “full stack” or comprehensive perspective of a project that may be difficult to track and monitor in a highly departmentalized organization. A hybrid approach, known as “Water-Scrum-Fall” can be beneficial to help adopt some best practices from each by defining a plan at the start, iterating on that plan, and delivering throughout the duration of that plan.

I discuss how Water-Scrum-Fall has been implemented and how it has helped bridge gaps in planning within a deeply hierarchical organization, to tracking project progress across departments, to understanding unexpected bumps in the road, and adjusting plans and goals accordingly. In our attempt to become more agile over the span of several years, we at the National Geospatial Technical Operations Center have found ourselves in a Water-Scrum-Fall paradigm. Our initial efforts were to apply an agile framework into development. A true agile framework cannot exist within development only and we found the agile principles gradually expanding into the planning and delivery components. Based on these initial efforts, we enabled ourselves to be more flexible and agile in both planning and delivery components. We feel that we have hit a good stride in our ability to monitor and communicate a project’s status with enough flexibility to iterate and alter plans and schedules when necessary. In all things agile, this is most likely our current stepping stone into a more innovative and effective future.

Water: Converting an Annual Plan Into an Agile Backlog

It is crucial to have as many levels of involvement as possible during annual planning exercises. Having a high-level executive say, “We shall improve our product this year to meet this customer’s need,” is the start and direction that is needed, but simply not enough. The product owner then needs to divide this visionary plan into smaller, measurable “milestones.” Examples are, “In Quarter 1, we will enhance our product delivery mechanism,” or “Through Quarters 2 and 3, we will enhance our product’s functionality in these three ways” or “We will allot this much time in Quarters 2 through 4 to resolve [critical] bugs”. Defining the beginnings of a critical path through milestones at this stage allow for the realization of priorities and dependencies.

The development team and product owners must get together during annual planning and determine if these are truly achievable goals. They review known knowns and known unknowns and develop a rudimentary plan. As the unknowns are identified throughout the year, the original plan will be amended and iterated on to account for these unexpected efforts.

• What major tasks must be done to achieve a milestone?
• What dependencies exist?
• How accurate are the estimates?

As this happens, the first round of plan refinement ensures that expectations can be met. It slowly evolves into a large plan of moving parts and time estimates. Lists of tasks and objectives have an initial round of categorization into “must have,” “like to have,” and “wish to have.” Based on budgeting, timing, unexpected hurdles, and multi-year plans, this type of categorization begins forming the framework and scope of annual work while setting expectations at multiple levels of management within the organization.

The breakdown of work will likely need to be coordinated across many functional teams (e.g., software developers, data architects, IT personnel). On enterprise-level projects, stakeholders may have vastly varying priorities within the annual plan. It is important for all stakeholders, product owners, development teams, and executives to have a forum to track cross-functional tasks, collaborate on appropriate priorities, and accept changing and evolving plans throughout the year. The agile framework will not reach its maximum effectiveness if only implemented at the development team / scrum level, but only through the entire plan-develop-deliver process.

Scrum: Prioritizing and Iterating on Actionable Items Across Teams

bakhtiar_zein/iStock/Thinkstock

Just as the iterative scrum cycle occurs for the development teams, so must it occur at the stakeholder and project management levels. If scrum teams simply focus on “continue development” through each scrum cycle with no finer granularity of direction from a product owner or understanding of dependencies in other teams, the agile framework disintegrates. The product owner must continually groom the backlog so that the highest priorities are worked on and that the most effective critical path is being followed through development cycles. This becomes challenging for enterprise systems and projects where many stakeholders (or product owners) are part of the picture.

As part of our agile structure, we have implemented Innovation Working Groups, which are cross functional teams of stakeholders, subject matter experts, and operational staff that collaborate to provide guidance to a development team. Within these groups, discussions are led by a project manager that may have oversight on multiple projects or that brings multiple stakeholders together to cross-prioritize. These collaborative meetings allow stakeholders or production teams to work independently while defining the critical needs and blockers. Unless immediate action is required, these working groups meet less frequently than the scrum teams. Their goal is to periodically make sure that all teams stay true to the defined critical path and make planning adjustments and iterations as necessary throughout the year.

Coordinating working tasks, priorities, and dependencies across teams can also pose significant challenges to ensure development is not blocked unnecessarily. Some project managers have started to perform weekly “scrums” or status meetings to facilitate a higher-level functional team to coordinate cross-team tasking. It is an opportunity for multiple product owners and scrum masters to gather in one room and discuss recent activities.

• What is the mid-sprint status?
• Is one team soon to be dependent on another?
• Do sprint priorities need adjustment to ensure that all teams are working harmoniously?

This ability to communicate, collaborate, and even pre-plan sprint goals and issues enables smoother cross-team coordination and prevents siloing of teams that do not have “full stack” control over their projects.

Fall: Adjusting Plans Based on Agile Metrics

As work continues through the year, it is critical to reevaluate the plan as unknown tasks become knowns. Adjusting plans throughout the year allow communications of change to propagate through the organization and set expectations so the “end-of-year” deliverable does not contain any unexpected surprises. Within our development teams, we are beginning to link development tasks to planned, annual activities. This enables us to extract metrics of the distribution of work across a vast set of annual goals and teams. Through this, we can observe where priorities may have shifted (one activity being worked more frequently than another) and where planned estimates were off (one activity experiencing many unknown unknowns and exceeding the originally scoped resources).

Comparing the distribution of work extracted from the agile boards to the planned distribution of work from the annual plans serves as a catalyst to straightforward insight into how priorities need to be adjusted. Should resources shift from one project to another? Does the scope of one project need to change? Though this, we can identify how some of our more important priorities (often maintenance of existing projects) can easily be overlooked during annual planning activities but are still well attended to throughout the year. This comparison forces us to realize how our day-to-day business can conflict with ambitious annual goals and allows the reality of priorities to become tangible.

Mid-year adjustments to plans should not be considered a point of failure but demonstrate the focus we attain by clearly identifying and adjusting expectations based on the reality of the situation. If it becomes apparent that the planned goals become insignificant as newly discovered priorities are defined, why would we adhere to something that is no longer valuable or “fail” the documented annual deliverable? Adjusting and iterating through the annual plan can reset these expectations and clearly define updates to the critical path so the project can be delivered when expected with the most earned value.

Lessons Learned: Iteratively Improving Is What Agile Is All About

The lesson that we learned over time was that forcing an agile framework into an existing waterfall paradigm to “improve our organization” will not make immediate improvements to the organization. Instead, the agile framework needs to be happening in all facets of our organization and that practices of both waterfall and scrum can be blended so that the organization, overall, becomes gradually more agile. Having tiers of functional teams enable cross-prioritization, coordination among teams, and ensuring functional development teams have the “full stack” perspective they require. Through iterative adjustments to planning, we can ensure expectations are met and all priorities are effectively documented; through iterative development activities, we can ensure the highest priority items are worked and delivered as expected; through iterative deliveries, we can ensure projects are delivered with intended goals met and thoroughly vetted by product owners. While this post focused on our current iteration of planning within an agile framework, we need to ensure we remain focused on enabling agile throughout our entire organization so we can continuously and iteratively improve our own processes.

Check out some of DigitalGov’s recent articles on the agile methodology. Have a .gov or .mil email address? Join the Agile/Lean Community of Practice to connect with others at federal agencies working to increase team efficiency and customer satisfaction, while reducing project risk and cost.