The Plight of an Architect in an Agile Project

Agile methodology in software development has emerged as a guiding light, promising flexibility, collaboration, and adaptability. But organizations have mistaken it for a luxury cruise liner while treating it like The Pirates of Carribeans, Black Pearl on the high seas of chaos.

Agile, with its sprints, stand-ups, and user stories, was supposed to be the antidote to the rigid and often cumbersome Waterfall methodology. However, in the real world, Agile is sometimes wielded like a double-edged sword – misused by developers and misunderstood by business leaders.

The Agile coaches are like the Pirate Captain, are the ones mainly responsible to steer the meetings, and are the ones who navigate the ship without an ounce of technical know-how. Picture the Agile stand-up meetings as the meeting of the Brethren Court, which typically turns into recitations of individual developers achievements. Each developer trying to resolve epics and making their own stories for their everyday chores, trying their best to please their captain. 

Then there are the Product Owners who act as the Pirate Lords, holding the keys to the treasure chest of project priorities. These lords of prioritization often struggle to let go the old ways of the Waterfall, treating Agile like a mere parrot on their shoulder rather than a shipboard companion. They treat technology debt as The dead man's chest, which is not supposed to be opened or seen.

Amidst all of them, the Architects are often left in the lurch as Agile teams treat their decisions as an afterthought. Their long-term vision gets lost in the relentless pursuit of project priorities and sprint goals. Good Architects are aware of the so called mirage on the horizon. But, often find themselves relegated to the backseat of the ship, much like a passenger becoming mere spectator watching their maps of successful navigation become damp and tattered in this unpredictable Agile Storm. 

Product Oriented Development Methodolgy

In several Organizations, the focus of project execution has recently changed from more product-specific deliveries replacing project-specific deliveries. This requires a complete organization-wide change to make every internal/external project run on AGILE ways of working.

What this essentially means is that every project needs to be viewed from an MVP approach. i.e. Most valuable product.  This requires

1. Changes in the organization and team structures

Teams are more defined based on specific goals. Teams are asked to think more in terms of product innovation and deliver specific features to evolve the product. There is no one-time delivery system and the business teams are aligned directly with these teams to incorporate faster feedbacks on the product roadmaps. Each team is attached to a dedicated product owner who helps create the specific goal required by the business teams. 

2. Changes to the ways of working.

Different stakeholders (Both IT and business) meet in every few weeks and define the product goals. Teams are more cross-functional so that they can be moved around once the specific need of the goal is achieved. From the business perspective, the money allocation to building these products is more incremental and on a quarter by quarter basis rather than estimating long term projects. If an idea is not flying it can be terminated at any point in time. Teams are more strengthened with Agile and DevOps culture. It’s a more flat structure where the individual teams mostly interact with an Agile Coach, PO, and a PMO in the hierarchy. 

3. Product Goal Definition

The product goal definition is based on the lifecycle of the idea from start to production and the idea is split into the specific weeks road map. The goals are created based on different objectives related to parameters that can be real-time customer behavior or any quantifiable metric. Each team is asked to present the product's progress in incremental demos every second week. Several of the goals are also based on the market competition and the main idea is faster velocity. 

Are Technical design documents really helpful and how to adapt to minimal documentation approach ?

As Architects and Developers, we have all been in projects writing high-level to low-level design documents. Design documents are a common artifact in software development projects, but their usefulness and effectiveness in an Agile organization have been the subject of debate among IT teams. The main intention of the design documents is to ensure that all stakeholders understand the project goals, technical blueprint, and requirements at a granular level. But, limited audiences are reading this document, and very seldom do these documents get updated. Hence, these documents tend to become cumbersome, time-consuming, and ineffective. 

The Case for writing Technical Design Documents

In a waterfall project software lifecycle moves from one stage to another, and writing design documents made a lot of sense as these were a deliverable and inputs to the next stage. Also, it served as a reference for various teams throughout the project lifecycle without many changes to the document. Additionally, in large projects where the development team expanded rapidly in different stages, design documents became helpful for onboarding new team members. 

However, developers still complained if the document became too long or technical for business stakeholders. Additionally, in a fast-paced project where requirements changed abruptly, design documents quickly became out-of-date or irrelevant as the project progressed, rendering them ineffective as a reference.

The Case for Minimal Documentation

In recent years, with several organizations adopting the Agile work model, there has been a growing trend toward minimal documentation. This approach typically emphasizes lightweight and flexible documentation and more on communication and collaboration among team members. Facilitation happens via regular meetings, syncs, retrospectives, tech guilds, stand-ups, and other forms of communication. Different Agile teams adopt different strategies:-

One of the minimal documentation approaches is to use lightweight documentation tools, such as Wiki or shared Google Docs, to document project goals, requirements, design, Architecture artifacts, decisions, diagrammatic representation of solutions, etc.

Some teams that involve business teams closely follow a BDD/TDD approach and want that as a starting reference for any requirement or even design-specific decision. The development teams also follow a strategic approach towards committing code and documenting every release into production. 

Development teams that hate to write documentation can use code documentation like Doxygen, Javadocs, etc or template-based tools like Markdown, Asciidoc, etc that generate documentation based on source code annotations, structures, or comments using automatic scripts.

Conclusion

Most development teams in today's Agile Organizations hate design documents as they are time-consuming and have maintenance overhead. They can slow down the development process leading to Analysis Paralysis. Also, many tools and techniques are available to automate the creation of diagrammatic representations and technical blueprints in the development process.