The series so far:
- Should You Consider Agile for Very Large IT Projects?
- Providing the Necessary Tools and Reports for Very Large IT Projects
To ensure the success of complex IT projects, it is important to have a close collaboration between the development team and the project stakeholders, easy access to centralized project data, cross-project visibility, precise project monitoring, integrated tools, prioritization and reporting in all phases of the project. The larger the team, the more important it is that the development software supports all this.
The larger the project, the more important it is to have development tools that will handle the complexity, size, variables and vast inputs that are associated with such projects. Whereas it is possible to deliver small or low complexity projects with many off-the-shelf Application Lifecycle Management (ALM) products and some manual intervention, this becomes much harder with larger projects. As the project complexity or the size increases, these tools will require more advanced capabilities. Without these capabilities, the tasks of managing and monitoring the health of the project will be a nightmare. If your software tools don’t allow you to identify the problems quickly, the cost to the project can be impossible to recover from.
To ensure success in such a project, you must therefore ensure from the start that your software tools meet your requirements. To do this you need to identify the support features that are required for every team process that will be used. This will vary with the development methodology, organization specific processes, and the pre-existing tools which must be used in one’s particular development environment.
As soon as one has identified the support that is necessary, the project leadership team within the governance board should plan to customize, develop and integrate these tools, train the team members and be ready with a working system that is capable of generating the project metrics and advanced reports before starting the project of this magnitude.
The team processes and the nature of the support required
Whether you plan to use pure Agile or a tailored Agile methodology, you will need the tools that support all the different types of team processes.
Team processes and the support required
So what are the team processes that need support and how should they be supported? Here is a typical list that is used to identify the important support features that are required for every team process.
| Team process | Type of support needed |
| Project Bank | Contains all the project data and materials in one place. It must be possible to track changes, and alert the owner. |
| Integration | Must be integrated with other tools for collaboration. Must allow for Synchronization of project artifacts across the lifecycle, allowing information to flow freely between ALM and other tools. |
| Flexibility | Able to fine-tune the review & approval process for project artifacts. The tool must have the flexibility to add a few project relevant fields and have a reporting capability. |
| Collaboration | Speed up the communication between team members using techniques derived from social media, including video chat, threads etc. |
| Review, Approvals | Deliver documents for review online, manage review cycles, define how many review cycles are allowed, approval deadline (calculated automatically based on review cycle & status). Track requisite review and approvals (sign-offs) in one place. |
| Roll Up | Must be able to roll-up and connect all the lowest level items to the highest level i.e., E2E process requirements, User Stories, Issues, Defects, Documentation, Approval, their status, expected date of completion must be connected to each other. In case some of these are managed outside of Collaboration tool, the Collaboration tool must have the complete picture. |
| Drill Down | Must be able to drill down to the details at the lowest level to determine the cause of bottlenecks. |
| Progress Reports |
|
| Defect Reports |
|
| Project Health Dashboard | An automated up-to-date dashboard for all key project success matrices*. * This will require roll-up. |
Feature capabilities
The following matrix provides a summary of the feature capabilities that are likely to be needed.
Capability Matrix
Meeting the requirements
The existing Application Lifecycle Management (ALM) tools have some of these capabilities, but there is a problem. They all aim to be all-encompassing: they are designed with the assumption that the team would use the product for all their requirements and nothing else. This means that none of them are easy to integrate with other tools, and if you must have integration you are offered the unprepossessing do-it-yourself route of an API or extract using CSV, Excel or PL/SQL. The DevOps approach is by definition ALM but its tools are different as DevOps brought into play the key element of culture that was missing in ALM. However, the DevOps alternative of the toolchain has its downside as well, in that the number of sources of information that has to be gathered increases making integration even more important to pull together data from a variety of discrete applications and make sense of it.
The Tools
Unsurprisingly, ALM tools are very capable for Sprint planning, progress reports, basic dashboards and can roll up the stories up to Epic (a very large user story consisting of smaller and more measurable stories). However, ALM tools generally contrast with the more recent DevOps tools in that they aren’t easily integrated to other vendor’s tools.
It is Epics that can cause particular difficulties. An Epic can be considered as being equivalent to a Business Requirement for all practical purposes. However, it is important to be able to roll up Epics into End-to-End business process in order to visualize the completion of the end-to-end progress. Such a view is important to decide mid project release among other reasons. The off-the-shelf tools do not have capability to manage End to End business processes out of the box. However, some of these tools can be configured to accommodate it. Where, many of these tools provide an API to allow you to integrate ALM tools with other enterprise tools.
A few prominently used ALM tools are listed below with their claimed features.
| Product Name | Description (as per vendor) |
| CA Agile Central | “CA Agile Central is an enterprise-class platform that’s purpose-built for scaling agile development practices. Provide a hub for teams to collaboratively plan, prioritize and track work on a synchronized cadence. Connect your development work to your company’s most important business initiatives. Measure productivity, predictability, quality and responsiveness with real-time performance metrics.” 1 |
| HPE ALM | “HPE Application Lifecycle Management (ALM) software and solutions enable teams of all sizes to deliver high quality apps with greater speed and agility. Combining ALM software products, DevOps and development tool integrations via an open REST API, a broad partner ecosystem, and a methodology-agnostic approach, HPE ALM increases release velocity while balancing application quality and enhancing collaboration throughout the lifecycle.” 2
It features Application Lifecycle Management, Agile Project Management, Quality Management (HPE QC), Open Source Integrations etc. |
| PTC Integrity | “Integrity Software Suite provides a comprehensive set of Application Lifecycle Management (ALM) and Model-Based Systems Engineering capabilities that enable a holistic software and systems engineering approach by improving collaboration, automation, and reuse across teams and disciplines. Simplify the complexity involved in developing today’s complex engineered and smart, connected products.” 3 |
| IBM CLM | “Delivers requirements management, quality management, change and configuration management and project planning and tracking.
IBM® Rational® Collaborative Lifecycle Management combines IBM Rational Team Concert, IBM Rational DOORS Next Generation and IBM Rational Quality Manager. It delivers requirements management, quality management, change and configuration management and project planning and tracking. These integrated capabilities foster greater communication, collaboration and visibility to accelerate delivery, improve quality and support better development decisions.” 4 “IBM® Rational Team Concert™ helps companies build better software and products with an all-in-one agile environment for development teams. This includes agile, formal and hybrid planning and reporting that are all on a common platform.” 5 |
| Polarion® ALM™ | “Facilitate synchronicity and easy access via 100% Browser-Based access to all data.” 6 |
| JIRA |
|
Most of these tools provide REST APIs for data extraction. For all practical purposes it will be a project in itself to integrate these products from different vendors. When it comes to Roll-Up reports, it is essential to use a specialized off-the-shelf tool and integrate it with ALM (e.g. Report Studio8).
Many of these tools are able to connect requirements management, quality management, change and configuration management and project planning and tracking. But roll up reports of E2E Process Flow, Epic, User Story level with connected Test Cases, the execution status and defects are not straightforward. If an enterprise is using several tools from different vendors, then this kind of view becomes even more difficult. Lack of integration across these tools can cause significant time lag and productively loss.
The DevOps approach
Consider a development environment where –
- Application Lifecycle Management is being done in an industry standard collaboration tool. ALM is master of E2E business processes, Business Requirements (Epics, User Stories), Design documents, Review process, Approvals etc.
- End to end QA for project is being done by all vendors using a separate QA system. QA system is master of Test Plan, Test Cases and Test Runs but not used for Defects.
- Software change, configuration and release management functions are in different tool(s). It is used in Continuous Integration, and sometimes in automated code deployments too.
- Defects are managed in common enterprise tool for Development and Operations. This defect system is different from the QA tool and used by all vendors.
- Testers create all defects in defect system and update the defect numbers in the QA tool in a field against the Test Case.
While there are obvious advantages in using a complete out-of-the-box ALM product, there is real possibility that you will have to spend a lot of time and effort to customize the tools and create your own custom reports.
ALM does not demand a single all-encompassing tool that supports the entire development lifecycle. These monolithic tools have been developed to meet a demand. The alternative is to adopt a DevOps approach and create a toolchain of specialist tools that each support a particular team process in depth, and which allow for easy transfer of information between them. However, there will still remain the specialist reporting that needs to be fine-tuned to the specific demands of your project.
Connecting the dots
In any Agile development, there are a number of entities that need to be tracked and reported on, with a number of inter-relationships. Here is a conceptual ER Diagram that depicts the most important entities. Depending on the systems used, these entities may be handled by one or more tools.
It is impossible to have a single unified approach to reporting on these entities because of the range of different systems in each enterprise. However, the required information can be extracted from all these tools. The remaining task is to make sense of this data and to generate the reports. But why not just use the reports that you can get out-of-the-box? Unfortunately, it is very unlikely that these will meet the complex reports that the team will require. This is because it is extremely difficult to connect all systems using out of the box capabilities regardless of how many tools that are being used. When you are using a range of disparate systems, it becomes even harder.
A simple reporting system to do this can be created via scripting in PowerShell or VBA within two to three weeks by using the automation of office tools such as MS Word, Excel and Access. As an example, data can be imported into a desktop database such as Access or SQL Server Express and then reports can be generated using Excel. Though this custom tool will require some effort to build it but it will not be substantial. Assuming Office tools are available, there will be no additional cost involved other than the time it takes to do the scripting. There are plenty of alternative ways of doing it, even using Python and SQLite with R or GnuPlot.
For collaboration purposes, these reports can be uploaded into the Collaboration (ALM) tool once a day so that these are visible to the all teams and stakeholders as well.
Reporting Tool Design
Data Extraction from Tools
It is impossible to describe how to extract data from every tool that you use. Most tools allow export via CSV, Excel, XML or RESTful interface. An established monolithic ALM tool such as HPE ALM supports REST API and SQL queries to extract the information.
Reporting
Sample Project Progress Reports
Typically, a transformation project involves changes in multiple systems. A typical Burn-Up Chart shows how much work has been completed. Below chart is a composite Burn-Up chart which not only shows the total work, but also gives view of contribution from each system against planned contribution.
An Epic contains a number of smaller stories. This report provides a “work completion” view of Epics along with work added and work remaining.
Below chart is a composite projection chart that provides an End-to-End view at Business Requirements level. It is very useful to give bird’s eye view to senior management and project stakeholders. It gives view in four dimensions – Baseline, Requirements Developed, Continuously Integrated, and End to End Tested.
Below chart is similar to the previous one except that it gives additional details at Sprint level.
Rolled-Up Progress Table
This gives low level details of each Epic’s status and which stories of the Epic are complete, along with completion dates of pending stories.
Sample Defects Dashboard
This is a simple report showing number of systems involved in the project, and defects identified in those systems along with their status. This view comes handy if there are multiple teams (or vendors) involved in the delivery. This helps in focusing on the right team and also can help in ascertaining quality of delivered work.
Sample Defect Impact Report
This report shows defects spread across systems and their status, along with how many test cases a particular defect is affecting. This helps in prioritizing the defect fixes.
End-to-end test cases
In this section, we show a sample report for end-to-end test cases. This report provides all the defects that are impacting a particular test case. It helps in deciding the priority of fixing every defect in the test case, and also provides overall execution results of Test Cases. This also provides a forecast of the presence of any particular defect, and forecast of when a Test Case can be re-tested. This becomes very helpful if a test case has many defects/enhancements from different systems.
Project Managers can plan for a week (or further) ahead, plan test schedules, plan developer resources, find out bottlenecks, ascertain teams responsible and respective vendors etc.
Conclusion
Because each organization has distinct differences in their governance and the extent of agile adoption, what might be easy for some organizations may prove to be challenging for others. You are the best judge of the right approach for your organization. What is important for larger projects is that all team processes are properly supported and that the reporting is effective. How you do it depends on your organization: You may identify a tool that can truly integrate the other development tools your team uses, or whether it is better to use a custom made tool or script to collate the data from all separate systems and create reports and draw conclusions. The project size, the complexity of the requirements, the systems impacted by the changes, the organizations’ Agile culture and organization’s governance will be the five key parameters that will help you make the right decision.
References
- CA Agile Central. Retrieved August 5, 2017, from https://www.ca.com/us/products/ca-agile-central.html
- Hewlett Packard Enterprise, Application Lifecycle Management. Retrieved August 5, 2017, from https://saas.hpe.com/en-us/software/application-lifecycle-management
- PTC Integrity. Retrieved August 5, 2017, from https://www.ptc.com/en/application-lifecycle-management/integrity
- IBM Rational Collaborative Lifecycle Management. Retrieved August 5, 2017, from https://www.ibm.com/es-en/marketplace/application-lifecycle-management
- IBM, Rational Team Concert. Retrieved August 5, 2017, from http://www-03.ibm.com/software/products/en/rtc
- Polarion ALM. Retrieved August 6, 2017, from https://polarion.plm.automation.siemens.com/products/polarion-alm
- JIRA Software, Atlassian. Retrieved August 20, 2017, from https://www.atlassian.com/software/jira
- Rational Insight 1.1.0, Multiple Rational solution for CLM and Rational Insight. Retrieved August 6, 2017, https://www.ibm.com/support/knowledgecenter/SSRL5J_1.1.0/com.ibm.rational.raer.integration.doc/topics/t_integrate_multiple_clm_with_insight.html
Load comments