This time I used Agentforce Vibes to double check roles.
Gave it the task of validating a list of user roles.
It wrote a SOQL query and ran it via SF CLI.
It confirmed existing roles and mapped their parent hierarchy correctly then it identified roles that were not returned in results.
Interestingly, it included a subquery to list one user in each role.
...(SELECT Id, Name FROM Users LIMIT 1)...
Imagine having to check tens of these manually between orgs. This is great to have!
Though I think there is a small risk of the underlying LLM (Sonnet 4.5) hallucinating false positives/negatives or identifying plausible but incomplete results.
In order to make it more deterministic, we would need to split the workflow into two parts:
- use the LLM to build and run the query and return rows into a CSV file
- then use a Python or Shell script as the verifier, listing found vs missing roles (that is, deterministically)
The script could even be customized to do fuzzy matching, dash/whitespace normalization, etc. It may be possible to add it later as a skill ("compare results in CSV file with list of values").
# in Python pseudocode
sf_roles = set( normalize(r) for r in roles_csv )
target_roles = set( normalize(r) for r in expected_list )
matched = sf_roles & target_roles
missing = target_roles - sf_roles
unexpected = sf_roles - target_roles # just in case 🤷🏻
That is to remove the highest-risk part - letting the LLM interpret results - and leave to the LLM the more easily auditable parts: we can inspect the SOQL, the CSV, and the script independently.
