Every senior engineer I know has some version of this suspicion, that the errors we encounter today are worse than they used to be.
So I went looking for the data.
Three questions:
- Are we making more mistakes per unit of work?
- Are more of these errors “escaping” into our built structures?
- Are the consequences getting worse?
Q1 could be answered using timesheets, drawings and all sorts of project data, but there’s very little incentive for anyone to do this.
Imagine a firm keeping a master spreadsheet of every error found on its projects over ten years, alongside its design output. It’s a lawsuit waiting to happen. Aviation has protected reporting. Infrastructure doesn’t.
Q2 we can’t answer convincingly either. We do audit built structures, but a design error doesn’t necessarily show up as a defect - or on Day One. If nothing shows up, we don’t know whether the design is correct or the buffer from conservatism in our codes is holding.
To answer Q3, we can look at US bridge collapse rates, which sit at around 1 in 4,700 per structure-year (Cook et al. 2015). Wardhana and Hadipriono found failure causes for US bridges broadly consistent across three decades, dominated by scour, flooding and impact, not design errors.
Q3 looks like a genuine “no”, but that’s the verdict on designs from decades ago. The work we’re doing now won’t show up for another 30 years.
There’s still the “feeling” that things are getting worse though.
Why?
I think 3 reasons.
1. Visibility
CROSS showcases near-misses that used to be invisible. Add social media, where any failure makes a catchy story.
More visible errors feels like more errors.
2. Error types
I suspect reliance on computers killed the arithmetic errors we may have seen in the past and replaced them with modelling errors. Put a wrong input in and the software still runs, still gives you an answer - hiding a possible conceptual misunderstanding beneath.
If you spend your whole day operating a tool rather than thinking through first-principles can we be surprised if conceptual understanding develops more slowly?
3. Checking under pressure
Projects have always been under intense fee pressure, but program is getting pushed further too. The first thing at risk is design checking. We rarely even scope it properly (“check this structure works”). The design is tied to a deliverable, so you can’t squash that.
But checking? When it works you don’t notice it. Matousek and Schneider found that most errors are catchable. Checking is the whole ball game, but it’s the first item cut when fee/program compress.
There’s very little data to answer these questions convincingly. We can’t prove anything about errors in the last 20 years. We could measure it, but we’re not incentivised to.
I am concerned about error types changing, and checking being squeezed. These have the potential to increase escaped errors, but, if true, we won’t see the consequences for another 30 years.