Skip to main content

Building anti-fragile technology systems

The Covid-19 outbreak which resulted in the government-mandated shutdown has exposed a lot of weakness in the technological systems we use to manage our day to day life. Apart from the negative effects on the economy and the loss of life and livelihood, Covid-19 is a wake-up call for all the people who build and manage large systems and especially those of us building or thinking of building future AI systems, we have to build anti-fragile systems.

A lot of systems that depend on the most sophisticated forecasting technologies of our times broke down when they faced the unexpected reaction from the pandemic. Some of them are listed below

1. Financial anti-fraud systems broke because consumers changed their behaviour. For example, credit card companies often flag a card as possibly stolen if the purchase pattern associated with it suddenly changes. But this rule of thumb doesn’t work well when huge swaths of society start working from home and stop going to restaurants and malls.
2. Logistics models used to predict supply and demand broke when manufacturers, shippers, and consumers changed their behaviour. Trained on last year's data, a model that predicts 1,000 widgets arriving on time next month can’t be trusted anymore.
3. Online services receiving a new surge or plunge in users are rethinking their demand estimation models since earlier models no longer are accurate. source:
What does this teach us? The natural reaction would be that we build systems that are more robust or disaster resilient. But what does robustness or disaster resiliency really mean? In the world of AI, it could mean getting more and more data to train models that can handle larger swaths of unexpected input, or in traditional systems engineering, it might be adding more redundancy, building systems based on more flexible constraints on input, etc.

While all these traditional approaches to designing robust and disaster-resilient systems are all well and good, we should be expanding the fundamental assumptions about what it means to have intelligent systems from systems that just recognize patterns from input data to systems that modify their own internal designs to cope adaptively with more chaos in the environment.

Nassim Taleb the father of the philosophy of Anti-fragility defines an anti-fragile system as one that benefits from disorder rather than one that abhors it.

As engineers charged with the responsibility of building future AI, our goal should not be to focus on systems that we can only improve incrementally in a gradual iterative fashion by adding more data and then even more because no matter how much data we get it cannot really protect us from black swan-like events except we build insurance against these kinds of events directly into the root of the systems. Our algorithms should not abhor uncertainty but rather embrace uncertainty and radically self-improve from it.

Intelligence must include adaptation to uncertainty besides pattern recognition, and these should be the basis of systems not just in AI but every other critical large human-technology system from supply chains to medicine, etc.


Popular posts from this blog

Next Steps Towards Strong Artificial Intelligence

If you follow current AI Research then it will be apparent to you that AI research, the deep learning type has stalled! This does not mean that new areas of application for existing techniques are not appearing but that the fundamentals have been solved and things have become pretty standardized.

Software of the future

From the appearance of the first programmable computer, the software has evolved in power and complexity. We went from manually toggling electronic switches to loading punch cards and eventually entering C code on the early PDP computers. Video screens came in and empowered the programmer very much, rather than waiting for printouts to debug our programs, we were directly manipulating code on the screen.

At the edge of a cliff - Quantum Computing

Quantum computing reminds me of the early days of computer development (the 50s - 60s) where we had different companies come up with their different computer architectures. You had to learn how to use one architecture then move to another then another as you hoped to explore different systems that were better for one task than they were for another task.