Organizations already struggle to keep employees from risky behaviors that could lead to a data breach. Now, generative AI presents an entirely new threat: employees accidentally entering sensitive business or consumer data into ChatGPT.
As more organizations adopt generative AI in the workplace, 15% of employees regularly post data to the tool, according to LayerX research published last year. Of those who share information on ChatGPT, 6% admit to having shared sensitive data.
Now security teams have a new concern: how to prevent employees from entering personally identifiable information and proprietary corporate information into generative AI tools.
Sharing personal data puts the organization at risk of breaching many data compliance laws. For organizations looking to add generative AI to their toolbox, they must create security protocols designed to prevent data leaks of sensitive information.
Placement of railings
The truth about AI, especially generative AI, is that while it presents a risk to businesses, it also offers many benefits. It is up to the organization to recognize how the good side of AI can become a risk.
There’s a need to put in place guardrails that allow organizations to do business safely as they embrace AI, said Max Shier, vice president and CISO at Optiv.
“Everybody is trying to find that balance between enabling and mitigating risk, especially when it comes to privacy laws and protecting confidential company information,” Shier said.
Generative AI used in any organization needs policies and controls designed to protect data.
The best case scenario is that a company doesn’t incorporate ChatGPT and similar tools unless the company already has a mature security program with data loss prevention tools and specific AI user awareness training, Shier said.
CISOs and CIOs will need to balance the need to restrict sensitive data from generative AI tools with the need for businesses to use these tools to improve processes and increase productivity.
They must do all this while complying with the alphabet soup of rules and regulations.
The “easy” answer is to make sure sensitive data doesn’t find its way into LLMs, and that doesn’t just mean training data, John Allen, Darktrace’s vice president of cyber risk and compliance, said in an email interview.
“Many popular LLM offerings specifically state that any data you provide through prompts and/or feedback will be used to tune and improve their models,” Allen said. “However, enforcing this limitation on sensitive data is easier said than done.”
Data Protection
There are two areas of emphasis when it comes to ensuring data privacy in the generative use of AI, according to Craig Jones, vice president of security operations at Ontinue, in an email interview.
Compliance Maintenance:
Organizations must rigorously assess and monitor how LLMs handle data, ensuring alignment with the General Data Protection Regulation, federal law restricting the disclosure of medical information, and the Privacy Act of California consumer.
This involves employing strong encryption, consent mechanisms and data anonymization techniques, along with regular audits and updates of data handling practices.
Protection of sensitive data:
Ensuring the security of sensitive data involves using a multi-layered security approach, including encryption at rest and in transit, strict access controls and continuous anomaly monitoring.
In the event of non-compliance, rapid response and remediation measures must be established, along with clear communication to affected stakeholders following legal and regulatory requirements.
Lessons learned from these incidents should be integrated into improving the data security framework to better address future scenarios.
Safeguards ahead
Generative AI and other security tools are adding subscription levels with enhanced privacy protections or creating APIs that will restrict sensitive data from leaving the company’s system. The data is not used to develop other AI models.
“Indeed, many vendors will also enter into data processing agreements and business partner agreements to meet specific compliance requirements for handling sensitive data,” Allen said.
In addition to generative AI usage policies designed to protect sensitive data, AI companies are also stepping up to better protect sensitive data, adding security controls like encryption and obtaining security certifications like SOC2.
But this is still new territory, and security teams are trying to learn what happens when sensitive data is in a model, how to find it, and how to delete it, especially for PII under strict data compliance regulations.
“The use of generative AI tools is still in its infancy and there are still many questions that need to be addressed to ensure that data privacy is respected and that organizations can remain compliant,” he said Allen.
