What do we lose when we optimize everything?

Reading about AI models for entire oil plants — it's impressive but something about it makes me uneasy. Not the efficiency itself, but the assumption that everything can be modelled. Anyone else feel that?