Have you ever contracted an engineering vendor who checked all the boxes … only to see the schedule derail anyway?
It seems like everyday lately. Everything was fine with the proposal. Solid rates. Good references. But six months down the road things fall apart and no one can identify when things started to go bad.
Here’s the problem:
Most vendor scorecards evaluate what’s easy to measure instead of what will actually determine performance. Price. Number of employees. Years in business. Certificates on the wall.
None of that gives you any indication if a vendor can do the technical work that determines if your project gets delivered on time.
The good news is? A scorecard designed around true capability will identify a weak vendor well before they cost you money.
What you’ll uncover:
- Why Most Vendor Scorecards Miss The Point
- The Four Categories That Actually Predict Performance
- How To Weight And Score Your Vendors
- Turning The Scorecard Into A Real Decision
Why Most Vendor Scorecards Miss The Point
Most procurement scorecards were designed to score buying commodities. They excel at valves, fittings and steel. They crumble when you try and score buying engineering judgement.
Look at what a typical one scores:
- Price per hour
- Company size
- Years in operation
- Certifications held
- Number of references
All of those are lagging indicators. They tell you what a vendor did, not what a vendor will do for you on your project. A company can be thirty years old and fully certified and still staff the most dynamically challenging line in the plant with a junior engineer.
Here’s a textbook example: piping vibration analysis. It’s an innocuous-looking line item on a proposal. But flow induced and acoustic vibration sneak-up and ruin schedules and cause costly field rework. Vendors who build vibration screening into their Piping Stress Analysis Services deliver a design that clears thermal code requirements and stays stable once the plant starts-up. Vendors who look for vibration risk during design catch it before cutting steel.
Same price. Completely different outcome.
The Four Categories That Actually Predict Performance
Leading indicators is what makes a good scorecard. Things that happen before the issue occurs.
There are four categories worth scoring, and everything else is noise.
Technical Depth In The Hard Disciplines
This category gets skipped most often, and it is the one that matters most.
Projects almost always funnel their complexity into two or three disciplines. Solve those two/three problems and everything else falls into place. Get them wrong and no project management will fix you.
Consider vibration. Statistics released by the UK Health and Safety Executive indicate that fatigue/vibration causes 21% of hydrocarbon releases in the North Sea. Approximately 1 in 5 containments can be linked back to something that should have been caught by a proper dynamic analysis.
So what should you actually be scoring?
- Does the vendor screen lines for vibration risk, or only on request?
- Do they work to recognised guidelines such as the Energy Institute AVIFF methodology?
- Have they ever demonstrated a failure analysis where the root cause was design instead of field?
- Who approved the analysis? How many years experience do they have?
Ask who the engineer is that will perform the piping vibration analysis on your scope. Good vendors name them right away. Bad vendors talk about “team capabilities” and deflect.
Delivery History On Comparable Scope
Generic reference checks are about as helpful as cereal box prizes. “Did you enjoy working with them?” ALWAYS results in a nice reply.
Compare apples to apples. Same class, same service fluid, same regulatory environment, same order of magnitude roughness. Just because a vendor succeeded at a small utility enhancement doesn’t mean they can deliver on a large process unit.
It’s worth noting why the stakes are so high. McKinsey looked at over 500 mega projects and found that cost overruns averaged 79% against initial budgets, while they took about twice as long to complete.
Most of that damage is technical rework nobody scoped at award.
Team Continuity
This one is sneaky.
The attendees of the kickoff presentation rarely are the ones doing the work. Engineering procurement oldest trick in the book.
Score it directly:
- What percentage of the proposed team is named in the bid?
- What is the vendor’s staff turnover over the last two years?
- Are key personnel contractually committed, or just “anticipated”?
Named people with contractual commitment score high. Anonymous resource pools score low.
Communication And Change Behaviour
How a vendor acts when things go wrong is worth more than how they act when everything is going right.
Ask yourself – what was the most recent change order they came across in their business? Did they own up to it as soon as possible, or did they wait as long as possible? Did they price it fairly, or look at it as a revenue stream?
Past clients will answer that honestly if you ask it specifically.
How To Weight And Score Your Vendors
Weighting is where the majority of scorecards silently fail. If price has a weighting of 50%, price ALWAYS wins and the scorecard is merely wallpaper.
A weighting that reflects reality looks closer to this:
- Technical depth in hard disciplines — 35%
- Delivery history on comparable scope — 25%
- Team continuity — 20%
- Communication and change behaviour — 10%
- Commercial terms — 10%
Rate each category on a scale of 1 to 5 with defined parameters for each number. Ambiguous rules lead to scores biased toward the center. You learn nothing from a scorecard full of 3.5’s.
Then add one rule that throws everything out: a floor on technical depth. If any vendor scores below a 3 there they are out, regardless of total score. Cheap engineering on the hardest part of the scope is never cheap.
Putting The Scorecard To Work
Build the scorecard before the proposals arrive. Not after.
After you see the bids, you can be tempted to manipulate the criteria to match whomever you already favored. Setting the weighting in stone beforehand prevents that.
Continue scoring after award. Score the same vendor at 30% design completion and once again at turnover. Two results occur:
- You build a database that makes the next selection dramatically easier.
- Vendors who know they are being scored behave differently.
That second point is underrated.
Tying It All Together
A vendor scorecard is only worth building if it predicts something.
Price, # of employees, and years in business are almost worthless predictors. Technical depth, similar delivery track record, team stability, and attitude towards change are solid predictors. Give them appropriate weight, set a minimum pass score for technical ability, and be consistent with your scoring.
To recap quickly:
- Build the scorecard before proposals land
- Weight technical depth heaviest
- Name the engineers, not the company
- Disqualify anyone below the technical floor
- Keep scoring after award
Focus on the disciplines where the fight is the toughest, and the rest of your project becomes much easier.
