Agentic AI and the Place for Human Ingenuity

Recently I came across a note on the place of human creativity and ingenuity in mathematics that was written before the LLMs and agentic AI's popularity. Namely I was looking up a concept on the princeton companion to mathemtics (published in 2008) where I came across the follwoing:

Different people draw different morals from these results. In the opinion of some mathematicians, they show that there will always be a place for human creativity in mathematics, however powerful the computers of the future might be.

And I thought wow, this is extremely relevant and contemporary as we are conatemplating our own roles and place in the world as white collar workers. But before I can explain why I think this has anything to do with the LLMs and agentic AI, I need to explain the context for the above mentioned sentence.

The sentence came after a discussion related to Hilbert's tenth problem (10th on the list of unsolved problem's from him), the problem is to determin whether it is possible to find a systematic procedure (algorithm) for solving Diophantine equation. This specific mathemtical problem does not matter for the current discussion, but rather that also Alan Turing and others encountered the same concept from computing prespective. They were asking the question wethere there is a general way to determin if a sequence of strings writtern in a programming language L will halt or run forever for an input I (To determin program P will halt or run forever for an input I). Turns out it is proven that there could not exist such systematic procedure (Algorithm) to do this. Further in the book it explains that this means we can not determin only through systematic procedures the truth or falsity of a mathemtical statement (and also that it is not only the case for the very non practical class of problems) because these statements can be encoded as halting problems and therefore we need still to use creativity and ingenuity to solve them as we can not guarantee that our truth checking program will halt no matter how much time and computing power we throw at it.

So to begin with, we know that agentic AIs are not purely logical algorithms, the agent does stochastic procedures through the LLM, meaning a lot of guessing is happening. So it is not surprising that in the answer to our questions it does halt at some point and come up with an aswer to us (although agents sometimes get stuck in loops, the loops are normally caused by the interfaces to the LLMs more than the language model part). This answer is not always necessarily correct or fully correct, what is known as hallucinations. From here on there are two paths to take to think about the place of human ingenuity in regards to the agent:

1. To assume human creativity and ingenuity goes beyond the stochastic capabilities of the LLM and it's current architecture.
2. To disucss given the multiple answers to a question, or statement who determins the correctness or who's satisfaction with the answer does actually matter.

Regarding 1, there are many prominent figures of the AI research who believe the way humans do heuristics, pattern matching, and are creative is beyond the capabilities of the current LLM architectures, but this also does not prevent companies to claim they have built the tools to turn intelligence to a commodoity and make the knowledge workers obsolete. One can conclude that if 1 is valid then we will eventually discover more of these class of tasks that require the human-specific ingenuity in the comming years as the markets and innovators will try to stay competitive outside the set of tasks discounted by AI agents.

Regarding the second point, we see already that the measure of how good the specific LLM variation is, or how good the coding/research/assistant agent from lab A is versus lab B really depends on the user perception and the expectation on the right answer rather than their performance on the published metrics. And it is probably also the reason these companies spend a lot of money exposing the models to users for free so they can get the feedback and incorporate the feedback with reinforcement learning in the models. In these examples we already see that the human judgement as long as the human stays in the loop of the operations is the measure of the "right answer" or satisfactory answer, be it usage of the agents to create a web application or answering a normativ question about who is the good guy in a certain group picture (a trend on twitter where users kept asking Grok to remove or keep the best person, most moral, etc in photos of multiple people).

Be it point 1 or 2, this means there might still be a place for the humans in the world of automation or that there will never be a full automation without human judgement. What the scale and value of this role will be, is a question answered in due time by the players and stakeholders who can shape the future in this realm with their individual decisions.