The creation of an evaluation framework for public sector procurements is something that I cannot stress the importance of enough; not only is a good evaluation framework the only way to accurately measure the relative merits (or otherwise) of submitted tenders, it is a necessity for public sector organisations with regards to transparency and the avoidance of challenge in their procurement process.
With that in mind, in this post I want to focus on two things:
1) the creation of an evaluation framework and, more specifically, the evaluation matrices that seek to objectify and quantitatively score key aspects of tenders submitted for a strategic commissioning or outsourcing contract, and
2) how to go about scoring and evaluating procurement tenders to ensure you pick the right supplier and avoid a risk of procurement challenge. With a well-assembled evaluation framework in your back pocket you can be sure of not only picking the right provider for your unique needs, but also of being able to justify your selection as being correct.
The Tender Evaluation Scoring Matrix
In a nutshell, an evaluation matrix sets out to:
- Establish what your evaluation criteria are
- Set out how each criterion (and sub-criterion) will be weighted in terms of importance
One must drill down to such a level where criteria scoring can be done with no ambiguity and absolute transparency. Although your top level criteria (e.g. price, delivery, quality) may be broad, the underlying sub-criteria will leave little room for imagination. Examples of sub-criteria include:
- Cost of ownership of the solution (price)
- Robustness of the implementation plan (delivery)
- Quality of the fitness for purpose of the solution without customisation (quality).
These criteria can be preceded by “gateway” criteria. These are “deal breakers” that receive either a pass or fail score. Examples of gateway criteria would be a provider having insufficient indemnity insurance or a previous record of fraud or bribery. These gateway criteria should be considered up front so that time is not wasted assessing tenders that will automatically be rejected by default.
Each main criterion will attract a percentage weighting that reflects its importance to the procurement as a whole. Subsequently, sub-criteria will be awarded a proportion of that percentage. A finalised evaluation matrix would usually look something like this:
You will note different scoring methods on the far right of the spreadsheet. The definitions of these scoring methods are as important as the percentage weights of each element. Your evaluation matrix should be accompanied by appropriate definitions:
Determining Criteria and Establishing Scoring Methods
In order to produce an evaluation matrix that is fit for purpose, an organisation must involve all the key people and departments that will have any material involvement with the procurement and/or solution. Consider key users, people who will depend upon it, people who will input information into it or require information from it, and so on. Identify all touch points within the organisation and try to involve a representative from all relevant areas.
At this point you should hold a workshop (or workshops) in which the scoring criteria and their respective percentage weights are defined. The workshop(s) should serve as an educational tool to clearly establish the way in which the evaluation framework as a whole will function. If the procurement method being used has particular requirements or restrictions then these should be outlined and incorporated. At the first workshop stage no criteria have to be laid out — representatives can take the newly-learned information away so that they can collaborate subsequently to define, refine and agree the appropriate criteria and sub-criteria.
It is important to note that there may be additional subsidiary tender documents in addition to the evaluation matrix that aids stakeholders in scoring as objectively as possible. Further clarification of sub-criteria is encouraged if it helps in producing final scores and/or clarifying the thinking behind scores.
The key at this point is that no stone should be left unturned. All key players must be involved in the process and they all must be committed to including all relevant factors within the evaluation framework. There is no going back from this step — once the evaluation framework has been finalised and tenders have been submitted, the scoring must be done on the basis of the agreed framework and no amendments can be made.
The Tender/Evaluation Matrix Cross Reference
An evaluation matrix will be of little use if it does not align with submitted tenders — it is of vital importance that one reflects the other in such a way that systematic scoring can be carried out. It is a matter of personal preference as to whether you produce the tender template or the evaluation matrix first, but they must ultimately align before you go out to tender.
The key question to ask is this: based upon the structure of the template tender, are you likely to get back all of the information you require and will you be able to interpret that information consistently in order to produce objective scores that can be referred to in terms of picking a winning contractor and justifying that pick?
What you should aim for is a ‘grid’ in which the rows are made up of template tender headers and the columns are made up of evaluation criteria. Scorers should be able to cross reference one with the other so they know what elements of the tenders to consider when scoring any particular criterion. This can be of great assistance to scorers when working through large piles of bids.
The key is in making the scoring process an easy exercise in objective thinking, where scorers are working within a clear framework having well defined parameters and mapping of criteria to bid content.
How to Score and Evaluate Public Sector Tenders
So once you have followed the advice above and issued your tender notice and all the associated procurement documentation including the evaluation and scoring information and response templates, you can now move on to the next stage of the procurement process – evaluating your tenders.
Step 1 – Adequately train and prepare your team for the evaluation process
Once the criteria and scoring methods have been finalised we advise that you then produce a clarification and evaluation guidance document that can be taken away and used to assist those conducting the scoring process. This document should explain the evaluation framework, provide clear instructions as to how criteria should be scored and set out how the incoming bids will be clarified, scored and the scores moderated. Ensure your team have this time booked out in their diaries and they keep to the deadlines you set them to try to avoid any delays to the published timeline.
Step 2 – Initial due diligence on tenders
The evaluation process proper is rather laborious, so it makes sense to be absolutely sure that each tender is actually worth scoring before proceeding. Therefore, the first step in evaluating tenders should be to assess each one against a set of absolute (rather than quantitative) considerations known as gateway criteria. Common considerations include:
-
- Does the bidder have the appropriate level of insurances?
- Is the bidder in good financial health?
- Are there any pending legal proceedings against the bidder?
- Has the bidder ever been convicted of fraud or bribery?
All of these questions should be covered in a questionnaire included within a pre-qualifying questionnaire or the tender itself. A negative answer against any of them usually means that the bidder should be disqualified from the competition.
You must ensure that this preliminary due diligence is complete before taking any further steps in the evaluation process.
Step 3 – Tender clarifications
Although the tender template you should have supplied earlier will have been designed to provide you with the best possible clarity, there is a very good chance that each bid will contain a number of ambiguities that require clarification. Such clarifications should be made before proceeding to any substantive scoring process.
The relevant client-side groups/departments/partners should handle the respective parts of the tender with a view to uncovering any gaps or ambiguities. This should then be followed by a clarification workshop in which all questions are collated for each bidder to form a clarification questionnaire, which should be sent to the bidder.
There are two approaches you can use to clarify ambiguities within a tender:
-
- Submit your clarification questionnaire in writing to the bidder and await a formal written response.
- Hold a clarification meeting with the bidder to be followed by written answers to all outstanding questions.
You should only proceed when you have received written clarification on all your points.
Step 5 – The tender scoring process
Once all of the necessary pre-scoring due diligence has been carried out, it is time to move on to the scoring process proper.
It typically makes the most sense to split the tender up amongst the relevant stakeholder groups, departments or partners for an initial assessment and scoring exercise. After that, you should bring all parties together into a formal scoring workshop. This is a meeting in which each bid is studiously assessed and scored based upon your evaluation framework. Each group/department/partner’s score is introduced and discussed, then merged into a generally agreed score by all parties, criteria by criteria.
This is sometimes called the “first pass” scoring. Most tenders should only require one pass — only tenders that are particularly large and/or complicated require a second pass score.
The scoring process can be tedious and long-winded and will require strong governance from a chairperson who ensures that the exercise remains disciplined, objective and compliant with the rules. There are likely to be a lot of opinions (perhaps conflicting) in the room and it can require an exceptionally strong personality to control the environment!
A golden rule of scoring is that scores must be awarded on the basis of the bidder’s submission (including any written clarification responses) and in scrupulous accordance with the published evaluation framework. You MUST NOT award scores comparatively between bidders.
Step 6 – Tender moderation sessions
Once the first pass scoring process has been completed, the established scores should be reviewed and moderated thoroughly to ensure that they represent a complete and objective analysis of the tender.
This requires all relevant parties to sit down and re-examine each tender carefully against your evaluation framework, line by line. You will want to empower people who are prepared to challenge, poke and prod at every point. As stated above, this process must be done without directly comparing bids and/or scores — each individual tender must be considered independently of the others and the scoring process must not be influenced by comparable scores for other tenders.
Both the tender moderation process and the evaluation process(es) that preceded it must be documented and minuted thoroughly. Furthermore, whenever a score is assigned it should be accompanied by a narrative that justifies the reasoning behind the score. By the end of the process you should have a completed master scoring workbook that, for each tender, includes each score with the comments from the group scoring and moderation sessions at the least.
Step 7 – Final Board approval
On the assumption that you have followed the prescribed process to the letter, you should be fairly much ‘home and dry’ by this point.
Calculating the final rankings should be nothing more than simple arithmetic, since each scoring criteria is scored on a numeric scale (of say 1-5, 1-10, price, etc.). The winning tender should then be submitted to a project or programme board, which will carry out a final sanity check to ensure the validity of the evaluation process’ conclusion. The Board should formally minute its deliberations and its ultimate acceptance (or rejection) of any recommendation to award by the evaluation team. The legal/procurement teams can only be instructed to proceed with announcing the award of contract when the programme or Project Board has given the scores their seal of approval.
Step 8 – Announcement of Contract Award and debriefing
Although you should now be in a position where the winning tender is clearly identified, the process is still not quite over.
After the announcement of intention to award (ideally within a week of coming to a final decision) you should issue debrief notices to each of the bidders. One of those notices will of course be a notice of intention to award to the preferred bidder, but each of the losing bidders must receive confirmation of their failure along with an assessment of their bid. This should include a summary of how their score compared to the winning bidder and a narrative focusing on the stronger and weaker points of their bid. However, direct comparisons should not be drawn with the winning bid within this narrative.
Avoid Legal Supplier Challenges
Unfortunately, you are not out of the woods yet. Bidders have two deadlines within which to submit a supplier challenges – one 10 days after they have been informed of your decision (these 10 days are known as the Alcatel period which is the most normal time for a supplier to lodge a challenge), and one 30 days after they have become aware of, or should have recognised, the challengeable issue (a requirement by EU legislation).
If you have followed a clear and templated path, backed by strong governance and documented evidence then you should be fine, but we cannot stress enough the importance of doing all that you can to make bidders feel part of the process – that they have a say in how it will work and that they are kept informed throughout, with you living up to your timing responsibilities and making no hint of favouritism. Challenges to your decision can be costly and time consuming, and could come about for reasons beyond your control, but if the steps above are followed then you will at least have the legal high ground.
Once this challenging sandboxing period is up, you can finally award the contract to the winning bidder and start to move forward with your project.
