LLMs can generate code and suggest using libraries that have never existed.

Researchers at UTexas-San Antonio released this analysis of fictitious packages referenced by LLM hallucinations in Python and Javascript.

Average percentage of hallucinated packages: 5.2% for commercial models and 21.7% for open-source models.
The highest hallucination rates: Codellama, DeepSeek. The lowest: GPT4

This leads to 2 thoughts:

1) they would probably be useful packages if someone implemented and published them

2) from the security perspective, an actor could publish a package with the same name but with malicious purposes

As mitigation, this paper suggests RAG and self-refinement (questioning the LLM regarding the validity of the packages it referenced).