Artificial intelligence (AI) has quickly become one of the most talked-about technologies in the insurance sector. From automating first notice of loss (FNOL) intake, to streamlining underwriting submissions, to accelerating claims indexing, AI promises speed, accuracy, and efficiency at scale. Carriers, brokers, and MGAs alike see AI as a lever to unlock growth while addressing workforce shortages and increasing operational complexity.
But as with any powerful tool, AI introduces new risks, particularly in the realm of cybersecurity. During Cybersecurity Awareness Month, it is essential to pause and examine the hidden security vulnerabilities that come with deploying AI in insurance. Insurers do not just manage any data. They handle some of the most sensitive financial, health, and personally identifiable information (PII) in existence.
If AI solutions are rushed into production without appropriate guardrails, the consequences can be severe: regulatory fines, reputational damage, data breaches, and erosion of customer trust. This blog explores the hidden risks, their implications for insurance operations, and strategies to mitigate them.
Why AI Security Matters More in Insurance
Most industries deploying AI are concerned about model accuracy, uptime, and scalability. Insurance faces all those concerns, but with additional layers:
Regulatory scrutiny: State and federal regulators (for example, the New York Department of Financial Services’ cybersecurity regulation 23 NYCRR 500, the National Association of Insurance Commissioners’ model laws, HIPAA for health data, CCPA and GDPR for consumer privacy, and new state-level AI guidelines) are increasing oversight. Failing to comply can result in costly penalties.
Data sensitivity: Insurance workflows handle health records, financial disclosures, driver’s license information, and social security numbers. A breach here is not just embarrassing, it can be catastrophic.
Trust at stake: Policyholders place enormous trust in carriers to safeguard their information. Even one incident can cause lasting brand damage and customer churn.
These factors elevate the importance of robust AI security measures. Yet hidden risks are often overlooked in the rush to deploy.
Hidden Security Risks of AI in Insurance
Workplace AI risks can arise when tools are misused – either through negligence or convenience, rather than malicious intent. Understanding these vulnerabilities is critical for protecting sensitive insurance data from unintended exposure.
1. “Shadow AI” and model training risks
Not all risks come from external actors. Employees might misuse AI, intentionally or accidentally, to extract sensitive data or bypass security rules.
Example in insurance: An underwriter uses an AI tool to process customer applications but pastes in unrelated sensitive data for convenience, creating a data sprawl problem.
Mitigation: Establish clear AI usage policies, train staff on compliance, and implement logging and audit trails to track every AI-driven interaction. Engage with and involve IT leadership in oversight regarding use of third-party AI during every step of this process.
2. Prompt injection and adversarial manipulation
AI models, particularly large language models (LLMs), can be manipulated by malicious prompts or adversarial inputs. Attackers might embed hidden instructions into documents, forms, or emails to alter model outputs.
Example in insurance: A malicious party could embed instructions in a PDF loss run, tricking an AI coworker into routing a claim incorrectly or exfiltrating data.
Mitigation: Implement content filtering, clear escalation protocols, and human-in-the-loop (HITL) checkpoints for sensitive workflows. Also conduct red-team testing, a cybersecurity practice where internal or external experts play the role of attackers, deliberately trying to break or trick the system so vulnerabilities can be found and fixed before criminals exploit them.
3. Supply chain vulnerabilities
Most insurers partner with third-party vendors for AI capabilities. This expands the attack surface, as vendors themselves may lack strong security practices. A single weak link in the supply chain can expose the entire organization.
Example in insurance: An AI vendor without SOC 2 Type 2 certification (a rigorous standard that proves data security and privacy controls are in place and tested) or ISO 27001 certification (an international benchmark for information security management) stores unencrypted claim documents in the cloud. A breach here could implicate the carrier.
Mitigation: Rigorously evaluate vendors for certifications, penetration testing results, and ongoing monitoring. Make vendor risk management part of AI governance.
4. Model drift leading to security gaps
AI models change over time as data evolves, a phenomenon known as “model drift.” If monitoring is lax, models may begin making incorrect or biased decisions, opening new vulnerabilities.
Example in insurance: A model trained to detect fraudulent claims becomes less effective as fraudsters adapt. The system starts approving high-risk claims without human review.
Mitigation: Continuous model monitoring, retraining with fresh datasets, and layered oversight (AI plus human review) help close gaps.
5. Data leakage through generative AI
Generative AI systems risk leaking sensitive training data through outputs. Even if unintended, this can expose policyholder information.
Example in insurance: An employee testing a chatbot asks about prior claims history. If the model was trained on sensitive documents, fragments of that history could resurface in the response.
Mitigation: Use retrieval-augmented generation (RAG), an approach where the model retrieves information from a secure, curated database rather than relying on raw training data. Pair this with de-identified datasets, differential privacy techniques (mathematical methods that prevent individuals from being identified in data), and strict access controls.
6. Overconfidence in automation
One of the most hidden risks is cultural: assuming AI “just works.” Overconfidence can lead insurers to remove human oversight too quickly.
Example in insurance: An AI agent classifies medical bills with 98% accuracy. Leadership removes manual review, only to discover later that the remaining 2% included high-severity misclassifications that regulators flagged.
Mitigation: Use retrieval-augmented generation (RAG) – an approach where the model retrieves information from a secure, curated database – rather than relying on raw training data. Pair this with de-identified datasets, differential privacy techniques (mathematical methods that prevent individuals from being identified in data), and strict access controls.
Exclude sensitive data: Never train or fine-tune models directly on raw policyholder data or claims history.
Embedding Cybersecurity within Your AI Framework in Insurance
To harness AI safely, insurers need a layered defense strategy aligned with both industry regulations and cyber best practices. Key pillars include:
1. Governance & Oversight
- Form AI governance committees spanning IT, compliance, underwriting, claims, and legal.
- Require audit trails showing where AI handled tasks and where human intervention occurred.
- Align governance with NAIC, NYDFS, and state AI model guidelines.
2. Vendor Risk Management
- Obtain proof of certification (SOC 2, ISO 27001, HIPAA, CCPA, GDPR readiness).
- Require ongoing monitoring and breach notification clauses.
- Include explainability and bias audit provisions in contracts.
3. Technical Safeguards
- Encrypt all training and inference data in transit and at rest.
- Use red-teaming, penetration testing, and adversarial input testing.
- Apply least-privilege access policies and robust logging.
4. Human-in-the-Loop Design
- Tier oversight by risk level: light-touch for routine tasks and full review for high-value or regulated workflows.
- Train employees to understand AI outputs, spot anomalies, and escalate when necessary.
5. Culture & Training
- Communicate AI as a tool to support employees, not replace them.
- Provide training on responsible AI use and cyber hygiene.
- Celebrate wins but emphasize accountability and vigilance.
AI and security cannot be separated. For insurers, the stakes are too high to take shortcuts. AI can unlock enormous efficiency, accuracy, and customer satisfaction gains, but only when deployed with security at the center. The hidden risks outlined above remind us that trust is earned every day, not just through claims payments or underwriting decisions, but in how insurers safeguard customer data.
The winners in the next decade of insurance will not just be those who deploy AI fastest, but those who deploy it most securely.
As the industry embraces AI, the hidden security risks must be managed head-on. Cybersecurity Awareness Month is the perfect moment to reaffirm this commitment, not just to technology but to the policyholders whose trust underpins everything insurers do.
Want to turn risk awareness into action? Download our guide on AI Governance Committees: Building a Foundation for Trust to learn how to build a governance committee that aligns AI initiatives with compliance, best practices, and company values.