Part one of a three-part blog series.
“The major problems of our work are not so much technological as sociological in nature.” 
The above quote is a timeless summation that technology continues to hold a limiting influence on software productivity. To be sure, innovations such as OOP, refactoring, test-driven development (TDD), architecture-driven software development, SOA and the like are positioned (assuming appropriate application) to positively influence the productivity of any software development project. However, the purely technological silver bullet will remain elusive, necessitating a continuous focus on the human aspect of software development. The Agile Manifesto of software development espouses a people-oriented, concerted approach, proposing among its four value statements “Individuals and interactions over processes and tools”. . Agile methodologies cast an evolutionary influence over software development, in which the iterative and incremental spirit of the process emphasizes continuous improvement: “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.” . How does the Agile team “reflect”?
Retrospectives embody the spirit of team collaboration, self-reflection, inspection and adaptation by offering a framework of activities in which the members of a team are encouraged to provide feedback and identify lessons learned. Retrospectives, as applied in software development, have historical roots in Project Retrospectives, described as:
“A ritual held at the end of a project to learn from the experience and plan changes for the next effort.” 
While Retrospectives remain valuable as an end-of-project activity, the spirit of Retrospectives should be imbued across the project continuum, conducted at key project milestones: at the end of project iterations, releases, and immediately upon occurrences of key project incidents (unexpected events). We will see that these interim Retrospectives diverge in purpose from the traditional concepts of Project Retrospectives, and the Iteration and Milestone Assessments proposed in the Rational Unified Process (RUP) and its derivative process methodologies.
Interim Retrospectives: Supplementing Iteration and Milestone Assessments, and Using the “F” Word
Traditionally, the effectiveness or performance of a software development team has been measured using a combination of budget variance, schedule variance, software build quality, and conformance to standards, comparing these metrics to various planned values. The implication of such traditional measures of team efficiency and progress is that the health of the team is secondary to the team’s productive output; this is analogous to the long-standing debate on the utility of deriving a country’s level of social welfare, progress, state of its natural environment, etc. (exclusively) from a purely economic statistical measurement (e.g. in the form of its GDP). The Iteration and Milestone Assessments recommended by the Rational Unified Process codify these objective measurements of success or failure.
The RUP Iteration assessment is designed to continually assess risk and progress by evaluating project metrics (such as project stability, software modularity, software quality, expenditure profile, and the amount of rework conducted during the iteration), assessing iteration results against planned goals, considering external change(s) (such as reassessing the realistic expectation(s) of the iteration goals), and updating the project work products (such as creating change requests for the Vision, Risk List, Iteration Plan, Software Development Plan, and Software Requirements).
The RUP Milestone Assessment is primarily designed to address the management of the project, considering issues such as project risk profile, assessing the clarity of the project’s scope, measuring progress based on delivery capability, quality and planned work products, and considering the project’s level of cost and schedule performance. While the RUP Iteration and Milestone assessments continue to bear relevance and importance to the project’s health, these approaches differ from a Retrospective, diverging in their respective emphasis on team collaboration, content, and steps of execution. The guiding spirit and process of the Agile Retrospective has been established in the space of the OpenUP (involving yours truly, serving as the “committer” of the original content), formalizing the overarching steps, roles, etc. of the activity.
Retrospectives can be seen as a nuance of self-organization, avoiding the overly ornate measurement criteria and totalitarian characteristics of various software process improvement initiatives. Retrospectives are intended to evaluate the “How” of the team’s work, not the “What” of the process. The focus is on identifying factors that affect the team’s health, such as processes, artifacts, the project environment, and the tools used by the team. The Retrospective acknowledges the sociological challenges of the project, coupled with the technical challenges faced by the team. In the context of a Retrospective, the team is the preeminent driver of reflection, empowerment and process improvement, not the passive participants in a meeting that is strong-armed by a controlling leader. There are steps (detailed in a subsequent blog post) employed in a Retrospective setting to create and support an environment in which the team practices honest and unconstrained communication. The output of the Retrospective is more than a perfunctory list of recommendations; the team uses the Retrospective as a catalyst for continuous inspection and adaptation, integrating its lessons learned in the next iteration or project, and calibrating the team’s progress with the goals of the project.
Agile software development methodologies recognize the importance of open, face-to-face team collaboration, institutionalizing such practice in the fabric of their work. The Scrum Sprint Retrospective is exemplary in its approach to supporting the collective voice of the team, and providing an ecosystem in which teams can reach convergence and a state of high performance.
By focusing on the “how” aspect of a team’s experience in a project iteration, the collective discourse requires more than elicitation of cold facts and diplomatic viewpoints. A Retrospective welcomes the communication of emotionally charged topics and candid conversation, humorously characterized as the use of the “F” word: Feelings . To effectively provision a forum for the expression of subjective thought, a Retrospective must be properly structured and involve the finesse of a team facilitator (discussed later). The simple method of choosing strategic questions can incite the expression of feelings, without the impression that one is engaging in psychotherapy:
- When were you excited to come to work?
- When was coming to work “just a job”?
- When did you dread coming to work?
- What were the high points during the iteration? What were the low points?
- Describe the experience of being a team member during this iteration. 
A successful Retrospective requires an organizational culture that supports team collaboration, provides a sufficient time for reflection, encourages honest feedback, and invests in the effort to identify the appropriate person to facilitate the meeting; these are the topics that will be addressed in my next blog post.
–Ben Haith, firstname.lastname@example.org
- Tom DeMarco and Timothy Lister. Peopleware: Productive Projects and Teams. Dorset House, 2nd edition, 1999.
- Manifesto for agile software development. http://agilemanifesto.org
- Norman L. Kerth. Project Retrospectives: A Handbook for Team Reviews. Dorset House Publishers, New York, 2001.
- Esther Derby and Diana Larsen. Agile Retrospectives: Making Good Teams Great. Pragmatic Bookshelf, Dallas, Texas, 2006.
- Guest Lecturer Jutta Eckstein: Agile Methods – Coming of Age 6th International Conference, XP 2005, Sheffield, UK, June 18-23, 2005, Proceedings
The Blog Series
Part Three: Anatomy of a Retrospective