[ad_1]
OWASP Foundation has released the 0.9.0 version of Critical Vulnerabilities in LLMs (Large Language Models).
A groundbreaking initiative has emerged to address the pressing need for educating developers, designers, architects, and other professionals involved in AI models.
AI-based technologies are currently being developed across various industries with the goal of revolutionizing long-standing traditional methods that have been in use for over three decades.
The scope of these projects is not just to ease the work but also to learn the potential capabilities of these AI-based models.
Organizations working on AI-based projects must understand the potential risks they can create and work on preventing the loopholes in the near future.
Threat actors leverage every piece of information they collect to conduct cybercriminal activities.
OWASP Top-10 for LLMs
As per the recent publishing of the OWASP 0.9.0 version, the top 10 critical vulnerabilities are as follows,
LLM01: Prompt Injection
This vulnerability arises if an attacker manipulates an LLM’s operation through crafted inputs, resulting in the attacker’s intention to get executed.
There are two types of prompt injections as direct prompt injection and indirect prompt injection.
- Direct Prompt Injection
- Indirect Prompt Injection
Direct Prompt Injection which is otherwise called as “jailbreaking” arises if an attacker overwrites or reveals the underlying system prompt resulting in the attacker interacting with insecure functions and data stores that are accessible by the LLM.
Indirect Prompt Injection occurs if the LLM accepts external source inputs that are controlled by the attacker resulting in the conversation being hijacked by the attacker. This can give the attacker the ability to ask the LLM for sensitive information and can get severe like manipulating the decision-making process.
LLM02: Insecure Output Handling
This vulnerability arises if an application blindly accepts LLM output without sanitization, which can provide additional functionalities to the user if the user provides a complex prompt to the LLM.
LLM03: Training Data Poisoning
This vulnerability occurs if an attacker or unaware client poisons the training data, which can result in providing backdoors, and vulnerabilities or even compromise the LLM’s security, effectiveness or ethical behavior.
LLM04: Model Denial of Service
An attacker with potential skills or a method can interact with the LLM model to make it consume a high amount of resources resulting in exceptionally high resource costs. It can also result in the decline of quality of service of the LLM.
LLM05: Supply Chain Vulnerabilities
This vulnerability arises if the supply-chain vulnerabilities in LLM applications affects the entire application lifecycle including third-party libraries, docker containers, base images and service suppliers.
LLM06: Sensitive Information Disclosure
This vulnerability arises if the LLM reveals sensitive information, proprietary algorithms or other confidential details by accident, resulting in unauthorised access to Intellectual Property, piracy violations and other security breaches.
LLM07: Insecure Plugin Design
LLM plugins have less application control as they are called by the LLMs and are automatically invoked in-context and chained. Insecure plugin Design is characterised by insecure inputs and insufficient access control.
LLM08: Excessive Agency
This vulnerability arises when the LLMs are capable of performing damaging actions due to unexpected outputs from the LLMs. The root cause of this vulnerability is excessive permission, functionalities or autonomy.
LLM09: Overreliance
This vulnerability arises when the LLMs are relied on for decision-making or content generation without proper oversight. Though LLMs can be creative and informative, they are still under developmental phase and provide false or inaccurate information. If used without background check, this can result in reputational damage, legal issues or miscommunication.
LLM10: Model Theft
This refers to unauthorised access and exfiltration of LLMs when threat actors compromise, physically steal, or perform theft of intellectual property. This can result in economic losses, unauthorised usage of the model or unauthorised access to sensitive information.
OWASP has released a complete report about these vulnerabilities which must be given as high priority for organisations that are developing or using LLMs. It is recommended for all the organisations to take security as a consideration when building application development lifecycles.
[ad_2]
Source link