Catching up with some readings on AI and design, I’ve re-read Fabien Girardin’s great When Automation Bites Back.

The post is a thoughtful overview on how automation is sometimes against people’s interest and present some thoughtful and practical guidelines on how to design AI product differently.

But what particularly struck me was Fabien’s image of “stubborn” automation:

stubborn autonomous behaviours are intentionally designed as the core of business models that trades human control for convenience.

Of course I love personification metaphors when talking about designing with AI, in fact our project Solo came from something like that, so I allowed myself to play a bit with it.

One one end we have Fabien’s “Stubborn AI”. This is one that has a very precise knowledge (it’s trained with a lot of data) and can take all decisions autonomously (never requires any guidance from the user) but can be inflexible even on critical cases (and that’s when the issue arises).

But what if we flip that around? What we have, is some kind of “humble AI”. This is an approach to autonomy that needs adapting before being useful (with an initial ad-hoc training by the user), is always open to adapt its behaviour (can be constantly updated with new instructions) and whenever unsure, it requires the user’s intervention (and learns from it). Yes, there will be some tradeoff in term of “seamlessness”, but I feel that this “humble AI” is a better, more human-aware way for our relationship with smart stuff.