When I was labouring away in my economics classes at LSE more than 10 years ago, there was a lot of debates around accuracy of models we were using to understand the economy. And the agreement was that all models were ‘wrong’ because they are (over)simplified versions of reality.
But simplification was necessary for us to understand something that was complex. Incorporating the complexity into our models may not help us very much. To give a concrete example, in economics, we claim that all purchases which were the realisations of demand for all goods were driven by individual preferences. But in reality, there are also goods which were gifts and because you are buying it for someone, it is not about the preference for the underlying goods but something more complex. Yet if you need to start working out a model where the demand for a good is driven not by preferences for it but cultural perception of the signals around the gifting of a good, it is not so generalisable and would not be useful.
Today, models are getting more complex, and we are building out more and more models in order to perform sophisticated calculations and help support decisions. We create metrics to test our models and they tend to be around how ‘right’ the models are with ‘predicting’ results we already have. The challenge is no longer oversimplification but a problem we call ‘overfitting’, which is where we are trying to create models that is able to fit the data we have so well that it is not generalisable. This is actually the problem above – just because we are capable of building more and more complex models, we think it is better when it isn’t.