Jamie Valenti-Jordan is Food Brand Program Manager at the Food Finance Institute ,and CEO of Catapult Commercialization Services.
For most organizations, when a capital project is developed for executive and organizational alignment, there is a required level of due diligence necessary as a function of corporate policy to ensure that the optimum system is purchased. This policy often specifies that two to five quotes for the same system must be sourced and compared in an “apples to apples” comparison. For those of us that have had to attempt to make these analyses, they can be more akin to “apples to harmonica-playing toaster.”
In order to provide an objective, systematic analysis based on organizational values, resources, and strategic priorities, normalization of these subjective elements must first occur before the options at hand can be scored and classified. Only after this is complete can a decision matrix be used to fully align the cross-functional stakeholders and represent all concerns on a common basis. “This is even more important now as the impact of looming tariffs are likely to change the supply chain calculus,” says Matt Tom, President of MTCC.
Stakeholders
The specification of equipment is nearly equal parts technical and political. Before attempting to source quotes to address project needs, ensure that all the relevant stakeholders are accounted for in the equipment need specification. Clearly the basics, like throughput, functionality, and space constraints, are necessary just to frame the universe of options. From there, it is important to engage the various direct and indirect stakeholders to understand how many needs can be met with a single system. This approach becomes helpful when enlisting multiple departments to support the decision that this process yields.
The direct stakeholders are often referred to by experienced professionals as the ones who have to “live with” the system. This list includes plant operators and supervisors, maintenance, plant engineers, quality team members, schedulers, and plant managers. These team members are important to have onboard even before the first quote is developed because they can reveal key features of the quotes that may change your sourcing strategy.
The indirect stakeholders will sometimes say that they do not have input on a particular system, but they are the soft decision makers that can influence the success of the project approval process substantially. R&D and marketing are two such groups that may be able to share some strategic midterm insights on equipment specifications, such as changeover needs and design requirements. Finance will also want to share insight on the impact of maintenance and changeover costs on the total cost of ownership, which is an important metric when communicating to executive leadership—who care about pursuit of strategic organizational goals.
Non-starters
With the stakeholders onboard, requirements for appropriate design can be gathered in a large (often virtual) cross-functional team meeting. Some of these requirements will be represented as optimum needs instead of pass/fail requirements. Split the requirements (i.e., must be able to run our packaging as it exists today) from the criteria for evaluation. By making a list of absolute pass/fail requirements, quotes gathered can be quickly evaluated as possible options for further evaluation or necessarily pushed back to the vendor for additional refinement.
Criteria
The first on every list everywhere is raw capital cost. This cost should be inclusive of necessary or preferred options. This process of selecting options is where engineers add substantial value because they can picture the future and the needs of a future team living with the equipment.
When gathering criteria, make sure that each stakeholder group has at least one criterion that directly relates to their role in the organization. For example, proximity in miles to the nearest parts storage facility is something that can lead to substantial outages and maintenance team member complaints about a system. When developing the list of criteria, consider a cross-functional brainstorming that is focused what is important, not how important.
Each criterion should represent an independent metric that can be quantified, ideally objectively. For example, consider the metric “changeover time (in minutes)” as preferable to “ease of changeover,” which is subjective.
Weighting
Weighting is the allocation of relative numerical importance of a particular criterion. For example, raw capital cost (including equipment option selection) is often valued between 30% and 70% of the total decision alone. Other weighting schemas look to apply a relative weight, such as lead time (in weeks), as a more significant criterion than cost of common wear parts (in dollars/year). Therefore, lead time might have a relative weighting of three while the cost of common wear parts might have a relative weighting of one. Note that relative weights above five and below one generate problems in the algorithm’s ability to analyze the data correctly.
Another technique to aid in sophisticated weighting needs is the use of a minor criteria subcalculation. This method works by restricting the minor criteria, in aggregate, to have a smaller impact on the final score by capping the overall impact of the score—for example, the minor criteria can be capped, in aggregate, to only be 20% of the total decision making. This method is useful when the team generates more than about 15 criteria.
When developing the weighting of the different criteria, leverage project champions and other senior leadership to provide strategic guidance on organizational priorities and opportunities. This division of responsibilities for criteria generation and weighting aligns with team expertise and job functions.
Scoring
Scoring, or allocating values for each option and criteria combination, can be highly contested and often debated if subjective values are used. This is why criteria should be defined as objectively as possible such that the numbers come from quotes and documentation. If subjective criteria must be used, leverage the team that pushed for that criteria to score the options for you to avoid arguments later in case another team member’s favorite proposal did not end up being selected.
Decision matrix algorithms
There are two primary decision matrix calculation algorithms out there: the additive and the multiplicative. The additive uses the weighting score as a multiplier, and the scores for each option are added together to give a comparative number. The multiplicative method uses weighting as an exponent for a particular criterion and then multiplies the resulting scores together.
Both work well, but they can give different results. Method selection can be as important to the final decision as the scoring and weighting. For this reason, this author prefers a third, self-developed method based on the number of standard deviations away from the mean that a particular criterion is to determine its relative score with weighting applied as a multiplier in the number of standard deviations. This minority-reporting method is usually only used in tiebreaking and does not necessarily yield the additive or multiplicative methods’ results preferentially. Only once has the third method yielded a third, unique result, and that was primarily due to intentional numerical manipulation to explore whether that result was possible.
Regardless of the method used, the result of a decision matrix is a decision—explicitly, “Option X is the best option.”
Calculation considerations
When actually building a decision matrix, think about whether a criterion is considered better when it is a higher value or a lower value. Simple toggle coding can allow for the functionality to flip without having to recode a cell. The mathematical compensation takes place with a sign in front of the weighting factor: higher is better uses a positive sign, while lower is better turns the weighting factor negative.
Finally, if building the decision matrix from scratch, consider normalizing each dataset by dividing by its mean value to start. The normalized data is easier to visualize and spot any errors.
Output
At the end of the day, the fanciest spreadsheets and data visualization tools are attempting to approach a single decision: “Option X is the best.” Once the matrix returns an answer, it’s important to gut check the analysis with stakeholders to ensure that nothing “seems wonky.” Review the matrix and results with the stakeholders originally engaged in the criteria development. This will not only validate the results as accurate, but it will bolster additional stakeholder alignment to the project goals and execution itself. Armed with this completed exercise, executive management can ensure that a recommended option has already been validated by the entire organization trusted to run the business.
