Evaluating your PPM software vendor

By Kristen Caretta, Associate Editor | Jun 18, 2009

0 comments

Delicious

Digg

Email

Print

Thumbnail: 

Choosing a project and portfolio management solution, like any enterprise application, can be a challenge. But never more so than when a company is seeking one standard PPM software package to replace some 10 different packages residing in 10 different project management offices (PMOs) -- the result of merger and acquisition activity over the years. Such was the case at Dalton, Ga.-based Shaw Industries Inc., a manufacturer of flooring products, which had varied tools and processes surrounding project reporting, logging and managing in its PMOs companywide.

'The groups were siloed in how they managed projects and communication,' said Greg Livingston, director of IS planning and systems development. 'As we grew, project management tools were created as needed amongst the groups, and we lacked consistency in how we handled tasks.'

This lack of consistency posed problems in meetings. When different groups brought reports together, 'the reports looked different, the prioritization processes looked different,' Livingston said. 'Business partners never knew which version they would be getting.'

The application mix included Microsoft Excel, Lotus Notes, Evernote (a point solution to manage Scrum projects), a custom Java solution called ShawOnline Activity reporting, .NET work order systems and Jira, an open source tool. Although some were effective as point solutions for functions such as resource management, application portfolio management (APM), project and portfolio management (PPM) and basic logging and tracking, none was an enterprise-centric bundle.

For example, people found the open source solution, Jira, effective for logging and prioritizing requests, according to Livingston, but it wasn't robust enough for enterprise use. 'It wasn't enterprise level for us. There were no tools for resource allocation, financial management or dashboards, and the software required multiple instances of data,' he said.

So when Shaw Industries decided it was time to standardize on one kind of PPM software, its requirements included a focus on process improvement and effective resource allocation, as well as alignment with methodologies already in place like Lean IT and Scrum.

'We needed a solution that would meet our two main needs: consistency in information extracting and reporting in our project execution, and a better picture of our application portfolio,' Livingston said. 'You cannot have an effective Lean Six Sigma strategy without a measurement process and metrics. We needed a single solution to help us manage our business.'

Developing a vendor scorecard for vendor evaluation

The next step was turning these requirements into a scorecard by developing a list of evaluation criteria and a scoring system. Livingston created a vendor scorecard that covered the six dimensions Shaw wanted in a product, from PPM to APM, financial management and more. Product evaluators would then score products on specific attributes in these categories, using a rating scale of one to 10. Categories were not weighted per se, but some had more features to rate than others.

For example, for PPM software, the evaluation criteria were:

• Project prioritization and business and effort analysis.
• Project planning and tracking from conception to final delivery.
• Capacity planning, including resource allocation and assignment.
• Time entry and tracking, with a focus on functionality and ease of use.
• How well the product supports Scrum methodology.

  Michael Hanford, a research director at Gartner Inc., said he is a big fan of using a vendor scorecard, as long as (like Shaw's) it's developed with one's organization in mind.

  'I've seen some problems where people borrow a scorecard, find one in a book or look at one that someone else is using and think, 'If someone else used it, it must be good,'' Hanford said. 'The thing you have to remember about scorecards is that it's a distillation of your goals, your needs and your corporate culture. What works for someone else won't necessarily work for you.'

  Hanford recommends being thorough but not going overboard. 'Some monstrously complex thing won't serve you well either because you're so focused on the scorecard, you forget what it is you're supposed to be evaluating,' he said. Most important are simplicity, ease of use and the knowledge that the scorecard is just one aspect of your vendor selection process, along with other research and customer references, he said.

  On to the vendor evaluation process

  With an evaluation framework established, the Shaw team turned to Gartner's Magic Quadrant to identify the PPM industry leaders. Using the profiles and general evaluations in Gartner's research, it narrowed the search to two vendors: CA Inc. and Compuware Corp. 'This is where the in-depth evaluation started,' Livingston said. 'We had two valid products from two well-respected vendors.'

  Evaluation teams then used the vendor scorecard during product demos and vendor meetings. At the end, the scores were close, but Shaw Industries eventually chose Compuware's Changepoint solution over CA's Clarity.

  What put Compuware on top? The biggest factor was the value of its PPM software, according to Livingston. 'What you get in relation to what you pay was a deal for us. The features that were important to us just came out to be a better price,' he said.

  The partnership and relationship between the two businesses was also a big selling point. After all of the evaluations and formal meetings, 'we just related better on a personal and business to business level with Compuware.'

  And finally, Changepoint had a feature called the agile accelerator, developed for agile IT and Scrum methodologies. 'CA offered to adapt the product to work with our Scrum methodology, but Changepoint was already designed for this,' Livingston said.

  At this writing, Shaw is still in the implementation process, which will take place across all 10 PMOs, with the project governance processes remaining the same. Livingston is satisfied so far with his team's decision. 'It's strategic. We can allocate staff more effectively and stay competitive,' he said.