When you think of accidents and the space programme, most people of a certain age will remember the Space Shuttle Challenger Disaster of ’86, when the craft disintegrated just 73 seconds into the launch, due to an O-ring failure. The 7-strong crew, which included a civilian schoolteacher, perished that day and brought the number of in-flight fatalities in global space programmes to 18. The remaining 11 people died in 3 other separate incidents caused by – parachute failure, a faulty cabin vent valve resulting in decompression in space, and a thermal protection system failure on re-entry due to launch damage caused by dislodged foam insulation.
Add to this the multitude of part and equipment failures that miraculously did not result in fatalities over the years, including bundled wires preventing module separation, hatch malfunction, airlock design fault, manoeuvring thruster shut-off failure, premature engine shutdown, liquid oxygen tank rupture, sensor failures and more, and you have a scientific field littered with supplier, manufacturer and management failures.
Key Supplier Falsified Performance Data
When it comes to space travel, if something goes wrong with the process, the risk to life is extreme. Of secondary importance are the astronomical costs of such significant service failures.
So, when NASA discovered that one of its suppliers had falsified performance data it was required to submit for a component it supplied to one of their rocket manufacturers, this was a fundamental failure of management and quality assessments on all sides.
NASA’s Investigation into Supplier Falsified Performance Data
In 2009 and 2011 the Taurus T8 and T9 rockets failed to make it into space due to, according to NASA’s claims, the failure of a single frangible joint supplied by SAPA Profiles Inc (SPI) to Orbital Sciences, the company responsible for building these rockets. Fortunately, these rockets were not designed to be manned.
These joints were created to be blown apart by mini-explosive charges so sections of the rocket could separate. This was designed so that the weight to thrust ratio was correct for the module to break orbit. The cost of these component failures has been calculated at in excess of $700m. The blame is being laid squarely at SPI’s door.
NASA’s multiple investigations into these failures concluded that they were down to three elements, the first two of which were likely to have contributed to, but not caused, the failure. It was the third that was considered the primary cause.
1. Charge holder thermal contraction:
In essence the analysis indicates that the explosive energy of the charge designed to break the joint apart was not great enough or focused enough.
2. Extrusion ligament thickness:
While within tolerance levels, the thickness of the material had fractionally grown over the years to be at the upper limit of acceptable.
3. Extrusion material properties:
Independent NASA testing showed that despite having all the right certifications from SPI, the joints were not up to material property specifications.
The report from NASA’s investigative summary stated:
“NASA OIG discovered that SPI, the extrusion supplier to Orbital (the manufacturer of the Taurus XL launch vehicle and now known as Northrop Grumman Innovation Systems (NGIS), had altered extrusion material property test results from failing to passing, and had falsely provided material property certifications stating the extruded material met Orbital specification requirements.
Records of SPI’s handwritten material properties test results obtained by NASA OIG [Office of Inspector General] and later shown in court documents revealed SPI made alterations to more than 2,000 test results between about 1996 and 2006, which affected more than 200 customers.”
The investigative summary went on to further state:
“SPI disclosed to the DOJ that from at least 2006 to August 2015, test lab employees were routinely involved in altering material properties test results. According to court documents, SPI later determined its employees altered over 4,100 test results affecting over 250 customers from about 2002 through about 2015. SPI’s test lab manager pled [sic] guilty to fraudulently altering failing test results to make them pass, and to training and directing lab employees to alter test results. Court documents also state the manager created a set of written procedures for lab technicians to consult for instruction on how to alter test results.”
On the Department of Justice website, U.S. Attorney G. Zachary Terwilliger for the Eastern District of Virginia stated that: “For nearly two decades, SPI and its employees covered up substandard manufacturing processes by brazenly falsifying test results.” He went on to say: “They then provided the false test results to hundreds of customers across the country, all to increase corporate profits and obtain production-based bonuses.”
Open-book Auditing; Reducing the Opportunity for Strategic Suppliers to Falsify Performance and Cost Data
In our own experience, supplier data manipulation for critical services/projects is rare. But it does happen. More commonly, we see individuals within certain suppliers manipulate KPI results to achieve compliance with a contractual obligation – which in turn may lead to performance/bonus payments for the supplier company and/or individual being paid out when they should not have been. Where this happens, it would be unfair to tar the whole company with the same brush. Data manipulation is often not a corporate directive from the supplier, it’s often down to an individual who is usually manipulating results in an attempt to save their own reputation when services are not performing well.
What can you do to prevent data manipulation impacting your strategic project/relationship? I think we can all agree that those who seek to deceive don’t come to you wearing a label saying ‘don’t trust me’. So, assuming that you can’t (and shouldn’t) treat everyone with suspicion, though you must counter the unpredictable, how can you professionally go about preventing data falsification from suppliers – or if it has/is happening, to identify it quickly and resolve it?
Open-book ‘accounting’ is a phrase much used among those in the strategic outsourcing/relationship fraternity. It’s a predetermined level of financial transparency between a strategic supplier and client. It’s a way of transparently sharing appropriate information, usually at specified milestones, in the relationship.
It benefits clients who can see that their investment in the supplier’s services is being optimised. It also benefits suppliers because it provides evidence of a level of commercial trust that is appealing to clients and fosters a collaborative foundation for working together in an open commercial environment.
Key Principles of Open-book Auditing
Open-book ‘auditing’ (as opposed to just open book ‘accounting’) is an extension to financial transparency. Suppliers evidence both financial and operational performance data. It extends the supplier’s trustworthiness beyond pure financial data. There are many aspects to open book auditing, but some of the key principles are:
1. Clear, understood and agreed terms
As with all other important elements of a major strategic outsourcing or complex project relationship, it’s vital to set out the terms of this openness you will agree between you in the contract that both sides sign at the outset. Clarity is key here, what financial and performance information will be made available, how often, when, who will be responsible for supplying the information/hosting the inspections and so on. The more thought that goes into these terms at the start of the relationship, the less room for dispute later on.
2. Transparency clauses
A relatively recent mechanism on public sector contracts enables the publishing of performance data. The general public are able to understand more clearly how their taxpayer money is being spent. We’ve been using transparent auditing rights for many years on the complex supplier relationships we set up from the outset and on those that we are asked to get involved with that require remediation after an event.
Structured in the right way, transparent auditing rights help clients to get access to all of the underlying inputs to financial and performance data, and the process of data collection. If operated by the client appropriately, they can access enough depth of information from the supplier’s records, to help them identify whether there are any misalignments in what is being reported.
Note that access is one thing, but interpretation is something else entirely. Superficial information will not lend itself to analysis, but only by knowing where there is potential for data manipulation can you request (and ultimately enforce) the depth of detail needed throughout the relationship that will aid you in steering a more desirable course.
3. Benchmarking against best practices
You’ve probably instructed your supplier due to their specialist service delivery knowledge. Therefore, it is important to be able to assess costs and performance benchmarks from other industry experts and appropriate data sources in order to determine: a) whether you are receiving value for money, and b) whether the underlying performance/quality data of the service is actually as the supplier is representing it to you.
Conclusion
Open book auditing is one of the ways by which to keep everyone transparent in their optimisation of their agreements. The very best suppliers are rewarded for their honesty and best practice adherence. Those suppliers that do not are more quickly identified, and remediation steps can be put into place to prevent inaccurate results being reported in future. But the client has to put in the effort to make sure open book auditing is being operated so that it has value to both client and supplier.
NASA’s issues with SPI are potentially indicative of a wider issue with supplier diligence which could, under other circumstances, have resulted in a loss of life. The consequences in this particular case were purely financial and commercial – albeit hundreds of millions of dollars and years of research and engineering. NASA had no idea of the ‘doctored’ test results until a lengthy, costly and detailed investigation, prompted by a court case.
According to court documents: “SPI has agreed to plead guilty to one count of mail fraud, and SEI [Sapa Extrusions Inc.] has entered into a deferred prosecution agreement (DPA) in connection with a criminal information filed today charging the company with mail fraud. As part of the plea agreement, SPI has agreed to pay $34.1 million in combined restitution to NASA, the Department of Defense’s Missile Defense Agency (MDA), and commercial customers. SPI has also agreed to forfeit $1.8 million in ill-gotten gains.”
