Stephan Dahl, July 23., 2009
Software, being the most fluid medium of construction known, is constantly changing. And while the medium is fluid, the cost of change in terms of wasted effort and time is every bit as crippling to a software project as to any other construction effort.
This article deals with tools and techniques to manage and describe change, and control its impact on the project at hand.
First and foremost, to control change we must be able to observe it - if we can't see and measure changes as they occur, we have little chance of keeping changes under control.
To realize this goal, it is necessary that all project artifacts are under revision control.
Keeping all project artifacts under control is both easier and harder than it sounds - easier, because every artifact in a software project is a file on a computer, and harder because many of those files are traditionally kept in many different repositories, some of which are transient in nature and many of which are not connected or synchronized with each other - emails for important decisions, USB sticks for test samples, multiple copies of spreadsheets (some attached to emails, some stored on shared drives) for plans. The list goes on.
In order to consider all artifacts under revision control, it is necessary to have a single repository for use within the project. The repository must meet a number of stringent requirements:
Once the repository is in place, it is necessary that the project actually
uses
the repository - this goes for every project participant. To this end, fostering a culture of "if it isn't in the repository, then it doesn't exist" can be helpful. If everyone as a matter of course goes to the repository for documentation, then it has succeeded - but it only takes a few exemptions where critical documents are stored "elsewhere" for the utility of the repository to collapse.
Now that all project artifacts are under revision control, we realize that there are different categories of files that we may like to group into logical units - these units are the Configuration Items of the project.
Configuration items share similar characteristics, such as
A typical minimal set of Configuration Items are:
A configuration item consists of a well-defined subset of the files that are stored the revision control system. Since all files in the revision control system must eventually be included in at least one configuration item, the configuration management tool must be able to report which files are not yet under configuration management, to serve as a to-do list for the librarian.
Once all files are under configuration management, it becomes possible to define Configuration Management Baselines. A Baseline is a key concept in managing change in general, since Baselines are the fixed points from which changes are measured.
A Baseline is defined in terms of a Configuration Item by specifying a particular revision of each of its constituent files. Typically, this is done by selecting the revision of all files in the configuration item as of a particular time, but it is entirely possible (and frequently desirable) to select some file versions based on time, and other files based on revision number - for example, if the baseline need to include a late-arriving but critical change.
A Baseline, once defined, is considered Open - while the constituent files (being under revision control) can't change, the Baseline itself can. However, once the Baseline is approved by all concerned parties, it is Locked, and then it is a true fixed point - a Locked Baseline can be used to represent final requirements, or an approved plan, or a reviewed design, or a shippable product. It is useful and desirable to be able to access the locked baseline as a complete package; usually this is done by a copy and export script.
Software projects being what they are, there are of course constant changes to the constituent files; Change control is never about preventing change, but always about observing change, and change management must never become a hindrance to the activities of the project participants. To this end, a configuration management system must be able to show the difference between the latest locked baseline and the current state of the documents under revision control. Likewise, it should be able to list the differences between any two baselines. As with the revision control reports, it should be possible to pose such queries ad hoc and expect a prompt answer.
Baselines should be labeled in a consistent manner; A three-level scheme with Major, Minor and Fix version increment counters usually works well for most purposes. The Major version is usually incremented at the end of a new project, when the first shippable version of the product is built. The Minor version is usually incremented when a new feature is included in a product, and Fix version is usually incremented when a critical bug is fixed. Lower counters are reset to zero when a higher counter is incremented. A product that has not yet shipped is usually assigned a major version of zero, with incrementing minor versions for each planned development timebox.
Note that labeled baselines as described can accommodate development branches. If a new release (eg, "2.0.0") of a product branches off the previous production version 1.4.13, then a bugfix (version 1.4.14) can be built and shipped and include files or file revisions that are not yet included in the current working set for release 2.0.0. Of course, the fix should eventually be merged back into the new version, as is usual.
A locked baseline can be reflected in the underlying revision control system, for example by tagging files with (Configuration Item)-(Baseline label). This makes it conveniently accessible to project participants (who usually view files through the revision control system) that a particular file is included in "Source-1-4-13". Note that an unlocked baseline (being mutable) is never reflected in the revision control system.
With the configuration items in place, we can now specify a Baselining Procedure:
The project librarian frequently (at least weekly) reviews the list of unconfigured files, and (possibly with the help of the file creator) assigns them to a configuration item.
In preparation for Project check points, the librarian prepares a Baseline of the relevant configuration items and a Baseline Report for each.
There is typically a project checkpoint once the requirements (usecases, interface models, performance requirements etc) are gathered and approved by the client. For this checkpoint, the Initial Requirements Baseline is created and locked, and it is from this baseline that the inevitable project change requests are measured.
The completed systems architeture (architectural overview, system context, logical data model etc) is sometimes subject to external review. The review material is a natural baseline, as is the material after the review findings are incorporated. If such a review is included in the project, it is advisable to have separate configuration items for Architecture and Design.
The project plan (budget, allocations and schedule) is also baselined once the client approves the plan.
The design and source configuration items are usually baselined together. An open baseline is created for the next release of the product (this may be a timebox in a project, or it may be a fix release). File changes for the release are then updated in the baseline, and once all changes are complete and test approved, the baseline can be reviewed and locked.
Project changes are usually gathered into releases, and each release has its own set of checkpoints; At least one for the updated requirements and one for the updated design and source files.
It is usually the project librarian who performs the day-to-day work with assigning files to configuration items and file revisions to baselines. However, the critical work of approving and locking baselines are done by a formal Configuration Control Board, which consist of the product owner or client representative, the Project Manager, and the librarian. For some configuration items, the technical lead or IT architect must also be included.
The control board meetings have a fairly fixed agenda, and for minor fix releases they can be completed in a few minutes on a conference call. For large releases, more time is of course needed.
The control board meetings may never be skipped; The approved minutes of a control board meeting are formal evidence of approval of changes.
Observe that while a revision control system is necessary in order to perform configuration management, it is not sufficient - in particular, managing open baselines is not normally possible in a revision control system. A separate tool is normally needed to perform configuration management; This is not normally a problem, since the configuration management system user group is usually limited to the project librarian.
Since the number of files being managed in a typical project can easily run into the tens of thousands, some sort of user-friendly automation must be available to create and edit configuration items and baselines, select and assign files and revisions using configurable filters, and to generate the necessary reports.
Implicit in the above section is that the project has an assigned Librarian; This is not a full-time role for any except the largest projects, and would usually be assigned to a project secretary. For very small projects, the role is assigned to either the project manager or the lead developer.
Building on established procedures for Configuration Management, we can now define procedures for Change Management.
Changes usually start with a Feature Request for a production application, a Project Change Request for a project in progress, or a Bug Report when a user or project participant encounters an error. For the purpose of Change Management, these are all equivalent; The processes to follow are always the same. The distinction in labeling is still useful, since prioritization and budgeting would differ between the different types of changes. For the remainder of this section, all the change types will be called Change Requests.
The first step when a Change Request is received is to perform a rough validation (is the bug real, is the feature requested relevant and possible?). Some Change Requests can be rejected early this way.
Once a Change Request is accepted as valid, its cost must be roughly estimated. Small bugs (eg, "typo in a screen label") can be estimated reasonably precisely, large changes (eg "application must be available in multiple languages") can only be guesstimated. If a large, imprecise guesstimate is provided, then a more precise estimate on how much a detailed analysis would cost must also be provided.
Once the Change Request is estimated, it must be assigned to a release. This is done by a Change Management Board, consisting of the product owner or client representative, in cooperation with a project manager. For a critical bug, a new release may be scheduled, small feature requests can be assigned a planned release where there is still space in the plan, and larger or low-priority changes can be assigned to "future release". During an active project, these assignments should happen on weekly meetings between the client representative and the project manager. The project manager should have authority to make these assignments on an immediate basis and just report his ations during the formal meeting, in case a critical problem arises. The formal meeting also reviews the funding for the change - bug fixes are usually funded by the project budget, changes in requirements are funded by the client, while feature requests from external groups may include funding for the work.
Change Request that has received approval for development are then assigned to the relevant project participants; Large changes can be subdivided and assigned to different people, and the subtasks may be marked with dependencies - eg, a task may be assigned to a business area expert for usecase updates, a lead developer for design work, two developers to write and review code, and a tester to write new testcases - naturally, some ordering between these tasks are in order.
If the estimate for the Change Request was large and uncertain, then it must be possible to approve work on a detailed estimate only (without naming a target release, but still assigning funding for the estimate preparation work); Once the detailed estimate ("bid") is prepared, the Change Management Board can assign the actual work to a release.
Once the project participants are working on a particular Change Request, they must be diligent in tagging their checked in work with the ID of the Change Request. This makes it possible to identify all changes across the project artifacts that are due to a particular change.
Once the changes are complete and the (new or changed) related tests all pass, the change may be included in a locked baseline as described above. The configuration mangement board can mark the work complete, and the Change Request submitter can mark the work accepted.
It must be possible to create project changes to include file changes already done by the project team on their own initiative (eg, code refactorings, document cleanups, requirements clarifications etc). Well-behaved team members create the change requests before they commit their changes to revision control, so they have a label to use. Such changes should generally pass without comment (and without funding) through the Change Management Board, under the assumption that there is a good reason for them, and that they can be contained within the already allocated time. Including and following this procedure prevents the Change Management processes from being a hindrance to the day-to-day work of the project.
Change Management for stable applications differ slightly from project Change Management - in particular, a stable application may not have a project group at hand to assign changes to. In these cases, approving the change and assigning funding also implies securing the resources needed to execute the work (both in terms of people and work environments).
Change Requests are usually tracked through a separate system; There exist a large variety of change-tracking packages in the market, and most would (if properly configured) meet the requirements of the process described above.
It is desirable that the same system is used for all types of Change Requests, and that the same system is used during the project phase as well as during the subsequent stable lifetime of the product: This makes it possible to extract stability metrics in a structured manner throughout the lifetime of the application.