We use "appraise" instead of "evaluate" or "measure" to emphasize that in considering supplier performance, we ultimately measure the value they deliver to our organization. We review practices which we have found of value below.
Supplier performance is tracked on an engagement level and reported overall on a monthly basis as part of our ongoing governance process. There is an additional aspect to performance, however, looking at which relationships are expanding, contracting, or are stable; Client customer satisfaction with the supplier's staff and services; how our suppliers compare to each other at the end of the day. We will be discussing both current activities and ones in development.
- Relationship summary
- Customer satisfaction survey
- Supplier satisfaction survey
- Key performance metrics
- Supplier scorecard
We recommend a periodic revew and documentation of the current state of key supplier relationships. Most of this information would be provided by suppliers in their monthly governance reporting deck:
- Relationship duration
- Current $ volume
- Current headcount
- Relationship status — growing? shrinking? risks? etc.
- Supplier type — "Tier 1"? Specialty?
- Services being delivered — range of in-progress engagements currently in progress
- Delivery locations— staff and equipment locations
- Contract type — summary of MSA, SOWs
- RFP Bid:Win:Loss — history of bids and losses on prior opportunitie
- Customer satisfaction survey — last survey results, scores, and trend
- Management asessment — strengths and weaknesses, client and supplier actions which can strengthen relationship
Surveying customer satisfaction is vital to understanding the value of a supplier relationship. All major suppliers conduct surveys, minimally of client management annually. This is helpful for the suppliers, however, there is marginal value to the client as each supplier survey is different.
Our approach is to work with all key suppliers to adopt a common consolidated customer survey:
- Ideally, suppliers adopt the sourcing survey in place of their own. In return, sourcing administers the survey process, collates and scores the responses, and maps the scale, scores, and questions to the supplier's if requested. While this is generally not possible for senior management surveys—typically conducted by third parties for a supplier, does address engagement surveys. In the closest client-supplier relationships, suppliers may elect to incorporate the client survey into their own performance tracking and employee reward processes.
- A common set of questions and a common process across suppliers allows the client to compare performance across suppliers. This helps in assessing the value delivered across suppliers and informs decisions on potential reallocations of work.
- A single standardized survey also provides a venue for supplier competition other than cost. Our experience is that suppliers are always eager to find out where they stand regarding their competition.
We have also found that annual surveys do not provide timely-enough information and suggest conducting them semi-annually. All basic areas should be covered:
- Meeting commitments
- Quality of communications
- Quality of services — Applications
- Quality of services — BPO and Help Desk
- Quality of services — Infrastructure management
- General service competency
- Overall service
Reporting should include breakdowns regarding ranges and trends of responses and history.
For key engagements, we recommend a short bi-monthly scorecard to be completed by both client (customer rating) and supplier (self-assessment). This is another early warning system for any engagement issues, and is particularly valuable in identifying any "disconnects" between the supplier's perception of their performance versus the client's. A sample scorecard, including supplier and client scoring on a simple Fails/Meet/Exceeds scale:
|Project execution||Description||Supplier F/M/E||Client F/M/E|
|Project Scope Management||Ability to clearly define scope throughout project lifecycle, setting the right expectations, managing scope creep, and so forth.|
|Change management||Flexibility to adapt to changing user requirements. Prioritization and management of change requests.|
|Communication||Ability to clearly communicate risks, regular status updates.|
|Quality and timeliness||Description||Supplier F/M/E||Client F/M/E|
|Compliance with user requirements||Accuracy of deliverables versus stated requirements. Percentage of requirements addressed in deliverables.|
|Defects and remediation||Number of defects reported in deliverables, time to remedy.|
|Defect severity||Deliverables quality as measured by defect severity and effort to remedy.|
|Compliance with committed timelines||Deadlines missed as percentage of total.|
|Utilization and productivity||Description||Supplier F/M/E||Client F/M/E|
|Productivity||Team productivity for a relevant unit of work, as appropriate.|
|Resource utilization||% utilized versus % available.|
|Resources||Description||Supplier F/M/E||Client F/M/E|
|Quality of resources||F/M/E qualitative "Fails/Meets/Exceeds" score.|
|Time to fill open positions||Average time to fill vacant positions.|
|Average attrition rate and time to transition||% and time until replacement resources achieved expected productivity levels.|
|Added value||Description||Supplier F/M/E||Client F/M/E|
|Innovations||Technical innovations with regard to architecture and methodologies.|
|Process improvements||Process improvements positively impacting deliverables.|
|Other improvements||For example, productivity.|
One of the key factors in a successful supplier-Client relationship is the degree to which the partnership is truly reciprocal. We advocate clients preiodically conduct a "supplier view of client" survey. As an ombudsman for the success of supplier engagements, such a survey can be an important tool in uncovering and addressing client process issues.
Our experience is that a robust qualitative survey across all suppliers provides the most value—particularly as an early warning system for developing issues before they materially, adversely, affect an engagement.
There are, however, metrics outside those tracked by project management which can offer additional insights into engagement health. One such example is effort utilization, which is effort divided by available hours. If low, it may indicate resource productivity or availability issues. If high—over 100$—then the supplier is assigning shadow resources or investing additional resources. Over an extended period, or if significantly over 100%, this may signal the engagement is in trouble.
We discuss metrics further here.
Suppliers generally provide summaries of relationship health based on their own methodologies and reporting priorities. These can be particularly valuable in surfacing any client issues for sourcing follow-up.