Back to LinkedIn posts

LinkedIn post 183

Check out this paper from Salesforce AI Research showing that even leading LLM agen...

Check out this paper from Salesforce AI Research showing that even leading LLM agents achieve only 58% success rate.

And it is not just in CRM agent tasks that LLMs fail often.
Even coding tasks will give you results that look great but are terribly misleading.

EXAMPLE: I was coding a LWC and just for testing gave that same task to the LLM.

It coded a call to a function like this

closeTab( { tabId: aTab.id } );

THE PROBLEM: the actual function parameter is not "tabId" as a property enclosed in an object, but the simple value of the tabId itself. And that function has to be called with async/await.

Running that LLM-generated code would not work and, on top of that, would fail in a way that does not obviously indicate what the issue is.