Which LLMs Are HIPAA Compliant? (w/Examples) + FAQs

No single large language model (LLM) is inherently HIPAA compliant. HIPAA compliance is a property of how a covered entity or business associate deploys the model, not the model itself. A model becomes usable with Protected Health Information (PHI) only when the vendor signs a Business Associate Agreement, configures enterprise-grade security controls, and the user follows strict administrative, physical, and technical safeguards under the HIPAA Security Rule.

The core problem is that most consumer LLM products, including the free tier of ChatGPT and Google’s consumer Gemini, log prompts, train on user input, and refuse to sign BAAs. Under 45 CFR 164.308, any disclosure of PHI to a vendor without a signed BAA is a reportable breach. The consequence can be severe: the HHS Office for Civil Rights (OCR) collected over $144 million in HIPAA settlements between 2020 and 2025, and single-incident penalties can reach $2.134 million per violation category per year under the HITECH tiered penalty structure.

According to a 2025 American Medical Association physician survey, 66% of U.S. physicians now use AI tools in their practice, up from 38% in 2023, yet fewer than half have confirmed a BAA is in place with the AI vendor.

Here is what you will take away from this guide:

• 🧩 How HIPAA actually applies to LLMs and who counts as a “business associate”
• 🔐 Which LLM vendors sign BAAs today, and which explicitly refuse
• 🏥 Real clinical, billing, and mental-health deployment examples that work
• ⚖️ The exact statutes, rules, and OCR enforcement actions that set the rules
• ✅ A practical checklist for choosing and configuring a compliant LLM stack

What HIPAA Requires of Any LLM Deployment

HIPAA is a federal law built from three operative rules that every LLM deployment must satisfy. The Privacy Rule governs use and disclosure of PHI. The Security Rule governs electronic PHI (ePHI) through administrative, physical, and technical safeguards. The Breach Notification Rule forces disclosure of unauthorized access within 60 days.

When a hospital or clinic sends PHI into an LLM API, the LLM vendor becomes a business associate under 45 CFR 160.103. That vendor must sign a BAA before any PHI moves. The consequence of skipping this step is direct: OCR treats the first prompt as an impermissible disclosure, and every subsequent prompt counts as a separate violation.

A common misconception is that “de-identified” prompts solve the problem. De-identification under the Safe Harbor method in 45 CFR 164.514(b)(2) requires removal of 18 specific identifiers, and free-text clinical notes almost never meet that bar without expert review.

The Three-Part Compliance Test

Every LLM use case must clear three hurdles before PHI touches the model. First, there must be a signed, current BAA covering the exact service endpoint used. Second, the deployment must meet the technical safeguards in 45 CFR 164.312, including access control, audit logs, integrity checks, and transmission encryption. Third, the covered entity must complete a documented risk analysis under 45 CFR 164.308(a)(1)(ii)(A) specific to the AI workflow.

Missing any one of these elements creates liability. A 2024 OCR settlement with Green Ridge Behavioral Health for $40,000 turned on the absence of an enterprise-wide risk analysis, a failure the same agency now flags in AI-specific guidance.

A real scenario: Dr. Alvarez, a family physician in Austin, pastes a patient note into the free ChatGPT web app to draft a referral letter. Because OpenAI’s consumer product does not cover that endpoint under a BAA, the paste is a reportable breach, triggering written notice to the patient and, if more than 500 records are involved, to HHS and the media.

The 2024 HHS Proposed Security Rule Update

In December 2024, HHS issued a Notice of Proposed Rulemaking to modernize the Security Rule for the first time since 2013. The proposal removes the distinction between “required” and “addressable” safeguards, making all controls mandatory. It also adds explicit requirements for encryption at rest, multifactor authentication, and annual penetration testing.

If finalized in 2026 as expected, the rule will directly affect LLM deployments. Vendors that today rely on “addressable” language to skip encryption on certain logs will lose that flexibility. The consequence for covered entities is that any LLM integration will need a fresh compliance review, and contracts signed before the effective date may need to be amended.

A common misconception is that the NPRM only affects hospitals. In practice, the cascading BAA obligations under 45 CFR 164.504(e) push every downstream subcontractor, including LLM API resellers, into the same requirements.

Which LLM Providers Sign BAAs

The list of LLM providers that will sign a BAA is small but growing. As of April 2026, the major enterprise routes are OpenAI’s Enterprise, Team, and API business tiers, Microsoft Azure OpenAI Service, AWS Bedrock, Google Cloud Vertex AI, and Anthropic’s commercial API and Claude for Enterprise. Each requires specific configuration to stay within the BAA’s scope.

No consumer product is in scope. The free tiers of ChatGPT, Claude.ai, Gemini, Copilot consumer, Perplexity consumer, and Meta AI are all excluded from BAA coverage. The consequence of mixing consumer and enterprise accounts on the same device is that an accidental paste into the wrong window is an instant breach.

A named example: Nurse Patel at a Boston clinic uses Claude for Enterprise to summarize admission notes. The identical prompt in the free Claude.ai web app would violate HIPAA because Anthropic’s consumer terms explicitly prohibit PHI.

OpenAI (ChatGPT Enterprise, Team, and API)

OpenAI will sign a BAA for ChatGPT Enterprise, ChatGPT Team, and the OpenAI API when requested through its sales channel. Under those agreements, prompts and completions are not used for training, and data is encrypted in transit and at rest. Retention defaults to 30 days for abuse monitoring but can be reduced to zero for eligible customers under the Zero Data Retention program.

The consequence of using the same OpenAI account on the consumer ChatGPT plan is that the BAA does not apply. A named example: Dr. Chen, a cardiologist, subscribes personally to ChatGPT Plus and uses it at work to draft patient emails. Her employer has no BAA with OpenAI for that endpoint, so every email is a reportable disclosure.

A common misconception is that “API means compliant.” The API is BAA-eligible only when OpenAI has countersigned the agreement in writing and the organization uses the business endpoint, not a personal key tied to a consumer account.

Microsoft Azure OpenAI Service

Azure OpenAI Service runs GPT-4o, GPT-4.1, o3, and o4 models inside Microsoft’s HIPAA-eligible Azure boundary. Microsoft’s enterprise BAA automatically covers Azure OpenAI when the tenant has accepted it. Abuse monitoring can be disabled through a formal application, yielding a true zero-retention deployment.

The consequence of leaving default content filtering without the abuse-monitoring opt-out is that Microsoft staff may, in rare cases, review flagged prompts. For covered entities, that review is not automatically a breach, but it is a use that must be documented in the risk analysis.

A real mini-scenario: Mercy Regional Hospital deploys Azure OpenAI for clinical summarization with the abuse-monitoring opt-out approved. Because the service sits in a dedicated Azure subscription and the BAA is active, the deployment satisfies 45 CFR 164.312(e)(1) transmission security requirements.

Amazon Web Services (Bedrock)

AWS Bedrock is listed as a HIPAA-eligible service and is covered by the AWS BAA. Bedrock routes to Anthropic Claude, Meta Llama, Mistral, Cohere, Amazon Titan, and Amazon Nova. AWS states that customer prompts are never shared with model providers and are not used for training.

The consequence of using Bedrock outside the HIPAA-eligible Region set or without the BAA activated is loss of coverage. Bedrock is HIPAA-eligible in most U.S. commercial Regions, but AWS GovCloud and Bedrock Agents with third-party action groups require separate review.

A named example: Riverside Billing LLC, a claims processor, uses Bedrock with Claude 3.5 Sonnet to auto-code superbills. Because AWS signed the BAA and Riverside encrypts all S3 inputs with KMS customer-managed keys, the workflow meets the Security Rule’s encryption standard.

Google Cloud Vertex AI and MedLM

Google Cloud’s Vertex AI platform, including Gemini 2.5 and the healthcare-tuned MedLM family, is covered by Google’s Cloud BAA. Vertex AI does not use customer data to train foundation models, and customer-managed encryption keys are available through Cloud KMS.

The consequence of using consumer Gemini at gemini.google.com is that no BAA applies, even if the user signs in with a Google Workspace account unless that tenant has HIPAA coverage activated. A named example: Dr. Okafor, a pediatrician, uses his practice’s Google Workspace Gemini to paraphrase parent notes. If the Workspace admin has enabled HIPAA coverage through the BAA, the use is compliant; otherwise it is a breach.

A common misconception is that MedLM, because it is marketed for healthcare, is automatically HIPAA compliant. The medical tuning does not change the legal analysis: a BAA, a risk assessment, and technical safeguards are still required.

Anthropic Claude

Anthropic signs BAAs for Claude for Enterprise and the commercial API directly and also provides Claude through AWS Bedrock and Google Vertex AI, both of which route under the cloud provider’s BAA. Claude’s default policy is not to train on customer data for commercial customers.

The consequence of using the free claude.ai consumer site with PHI is identical to other consumer products: no BAA, no coverage, reportable breach. A real example: Hillcrest Therapy uses Claude for Enterprise to draft intake summaries with a 0-day retention setting and audit logging piped to its SIEM.

A common misconception is that “safer” models, like Claude’s constitutional AI training, reduce HIPAA risk. HIPAA does not measure model safety; it measures contracts, safeguards, and documentation.

Healthcare-Specific LLMs and Wrappers

A second tier of vendors wraps a foundation model with healthcare features and offers a BAA directly to clinicians. Examples include Abridge, Suki AI, Nabla Copilot, DeepScribe, Hathr AI, BastionGPT, and CompliantGPT. Each signs a BAA and markets HIPAA-aligned deployments.

The consequence of choosing a wrapper is that the compliance surface shrinks, but vendor diligence grows. Covered entities must confirm the wrapper’s subprocessor list under 45 CFR 164.308(b)(1), because a wrapper that silently sends prompts to a non-BAA model breaks the chain.

A named example: Summit Orthopedics pilots Suki to draft SOAP notes. Suki signs the BAA and discloses that it uses Azure OpenAI under Microsoft’s BAA as a subprocessor, creating a complete, documented chain.

Three Real-World Deployment Scenarios

Scenario tables help translate the rules into daily decisions. Each of the following reflects the most common patterns OCR sees in 2025–2026 guidance and enforcement letters.

Scenario 1 — Ambient Clinical Scribing

The ambient scribe market grew to an estimated $1.1 billion in 2025, and OCR has signaled it will treat scribe vendors as business associates without exception. A named example: Dr. Ramirez uses Nabla Copilot on an iPad inside her clinic; the BAA is signed, the audio is deleted in 24 hours, and the output is reviewed before it enters the EHR.

Scenario 2 — Medical Billing and Coding

The HHS 2024 breach portal shows that billing vendors accounted for 19% of reported incidents, up from 11% in 2022. A named example: Pinnacle Revenue Cycle uses Bedrock Claude with VPC endpoints, private DNS, and CloudTrail logs, producing a fully auditable trail that satisfies 45 CFR 164.312(b) audit controls.

Scenario 3 — Mental Health Chatbots

The 2023 FTC action against BetterHelp and the 2023 action against Cerebral show that even when HIPAA does not squarely apply, the FTC Health Breach Notification Rule can. A named example: MindBridge Health licenses a MedLM screener, signs the BAA, and runs a third-party SOC 2 Type II audit every year.

Real Enforcement Actions That Shape the Rules

OCR enforcement in 2024–2025 focused on three themes relevant to LLMs: risk analysis, access control, and timely breach reporting. The 2024 Montefiore Medical Center settlement for $4.75 million turned on missing audit controls, a direct parallel to LLM logging gaps.

The Anthem settlement of $16 million remains the largest HIPAA resolution and still frames how OCR assesses insufficient access controls. The consequence for AI deployments is that shared API keys, unrotated tokens, and overly broad IAM roles are likely early targets.

The 2023 New York-Presbyterian case taught a separate lesson: recording PHI for a use the patient did not authorize is itself a violation. Any LLM workflow that captures ambient audio must have explicit patient consent documented in the chart.

Mistakes to Avoid

• Pasting PHI into consumer ChatGPT, Gemini, Claude.ai, or Copilot — every paste is an impermissible disclosure with no BAA backing.
• Assuming “de-identification” is automatic — free-text notes almost always retain identifiers that fail the Safe Harbor test.
• Using a personal API key for work — the BAA must be signed by the employer, not the individual developer.
• Skipping the AI-specific risk analysis — a generic IT risk analysis does not satisfy 45 CFR 164.308 for a novel AI workflow.
• Ignoring subprocessors — a wrapper that silently pipes prompts to a non-BAA model breaks the compliance chain.
• Turning off audit logs to “save money” — missing logs are a direct Security Rule violation and a red flag in any OCR review.
• Forgetting patient authorization for ambient recording — recording the encounter for LLM processing without consent violates the Privacy Rule.
• Assuming BAAs are evergreen — vendors update terms; a BAA signed in 2023 may not cover 2026 model endpoints.
• Mixing PHI and non-PHI tenants — a single shared Azure subscription for both creates auditability nightmares.
• Failing to train staff — the Security Awareness standard requires documented AI-specific training.

Do’s and Don’ts

Do’s:

• Do confirm the BAA covers the exact endpoint, because coverage is service-specific, not vendor-wide.
• Do require zero data retention in writing, because defaults still include 30-day logs at most providers.
• Do isolate PHI workloads in a dedicated cloud subscription, because blast-radius limitation is a core Security Rule control.
• Do maintain a living subprocessor registry, because downstream changes must trigger re-review under 164.308(b).
• Do run quarterly access reviews of API keys and roles, because stale credentials are OCR’s favorite finding.

Don’ts:

• Don’t allow BYOD access to enterprise LLMs without MDM, because lost devices become reportable breaches.
• Don’t rely on vendor marketing claims of “HIPAA compliant”, because only a signed BAA and your own safeguards provide coverage.
• Don’t store prompts in general-purpose logging systems, because those systems rarely meet the encryption and access-control standards.
• Don’t use consumer accounts on work devices, because one errant paste is a breach.
• Don’t skip penetration testing of the AI integration, because the 2024 NPRM will soon make annual tests mandatory.

Pros and Cons of LLMs in Healthcare

Pros:

• Time savings of 1–2 hours per clinician per day, according to the 2024 Permanente ambient scribe study.
• Reduced burnout scores, because clinicians spend less time in the EHR after hours.
• Improved documentation completeness, because LLMs surface missed review-of-systems items.
• Faster prior-authorization drafting, because structured letters can be produced in seconds.
• Better patient communication, because reading-level adjustments improve comprehension.

Cons:

• New breach vectors, because every prompt is a potential disclosure if misrouted.
• Ongoing vendor diligence burden, because subprocessor chains shift frequently.
• Hallucination risk, because fabricated facts in a note are a clinical safety issue.
• Cost of enterprise tiers, because per-seat pricing can reach $60+ per user per month.
• Training overhead, because staff must learn when PHI is allowed and when it is not.

Step-by-Step Process for Onboarding an LLM

The onboarding path mirrors any other business associate onboarding but adds AI-specific checks. Each step has a decision point and a consequence for skipping it.

1. Scope the use case in writing. Identify whether PHI will enter the prompt, the retrieval context, or only the output. Skipping this step leads to scope creep and undocumented disclosures.

2. Request and sign the BAA. Confirm the BAA names the specific product, such as “Azure OpenAI Service” rather than “Microsoft.” An incomplete BAA leaves the endpoint uncovered.

3. Complete a focused risk analysis. Use the HHS Security Risk Assessment Tool and add AI-specific threats such as prompt injection, model inversion, and training-data leakage.

4. Configure technical safeguards. Turn on encryption at rest with customer-managed keys, MFA on all admin accounts, and SIEM forwarding of all prompt and response logs.

5. Write the workforce policy. Define who may submit PHI, through which interface, and with what review. The Sanction Policy standard requires documented consequences for violations.

6. Train and test. Run a tabletop exercise that simulates a PHI paste into a consumer tool and tests the breach-response playbook.

7. Monitor and reassess. Re-run the risk analysis whenever the vendor changes models, adds subprocessors, or expands Regions.

State Law Nuances

HIPAA sets a federal floor, but state laws can add stricter duties. California’s Confidentiality of Medical Information Act extends protection to any business that “maintains medical information,” which captures many AI vendors that HIPAA might not.

Texas HB 300 defines “covered entity” more broadly and requires specific training within 90 days of hire. The consequence for a Texas clinic that uses an LLM without HB 300-specific training is state enforcement on top of federal HIPAA exposure.

New York’s SHIELD Act and Washington’s My Health My Data Act also reach AI vendors, with the Washington law explicitly covering consumer health data outside HIPAA’s scope. A named example: Evergreen Wellness, a Seattle wellness app, must comply with My Health My Data even though it is not a HIPAA-covered entity.

Comparison of Major LLM Providers

Key Entities You Should Know

The regulatory landscape is populated by a handful of critical entities that covered entities must recognize. The HHS Office for Civil Rights enforces HIPAA through investigations and settlements. The Federal Trade Commission enforces the Health Breach Notification Rule against non-HIPAA health apps.

The National Institute of Standards and Technology publishes the Cybersecurity Framework and the AI Risk Management Framework that OCR increasingly cites. The ONC HTI-1 Final Rule adds transparency requirements for certified AI in EHRs.

On the vendor side, the cloud hyperscalers (Microsoft, AWS, Google) host most enterprise LLM traffic, while foundation labs (OpenAI, Anthropic, Meta, Mistral, Cohere) provide the models. Healthcare-specific wrappers (Abridge, Suki, Nabla, DeepScribe, Hathr, BastionGPT) connect the two for clinicians.

FAQs

Is ChatGPT HIPAA compliant?

No. Consumer ChatGPT is not HIPAA compliant. ChatGPT Enterprise, Team, and the OpenAI API can be used with PHI after OpenAI signs a BAA and the organization configures required safeguards.

Is Microsoft Copilot HIPAA compliant?

Yes, Microsoft 365 Copilot is covered by the Microsoft BAA when deployed in a qualifying tenant. The free consumer Copilot at copilot.microsoft.com is not covered and must not receive PHI.

Will Google sign a BAA for Gemini?

Yes, Google Cloud signs a BAA that covers Vertex AI, including Gemini and MedLM. The consumer Gemini app is not covered and should not be used with patient information.

Does Anthropic sign a BAA for Claude?

Yes, Anthropic signs BAAs for Claude for Enterprise and the commercial API directly, and Claude is also available under AWS and Google Cloud BAAs through Bedrock and Vertex AI.

Is open-source Llama HIPAA compliant on my own servers?

Yes, a self-hosted Llama model can support HIPAA workflows if the hosting environment meets the Security Rule. No external BAA is required because no external business associate is involved.

Can I use de-identified data in any LLM?

Yes, data that meets the Safe Harbor or Expert Determination method under 45 CFR 164.514 is no longer PHI and can be used in any tool, though contractual and ethical duties may still apply.

Is prompt injection a HIPAA issue?

Yes, prompt injection that causes unauthorized disclosure of PHI is a Security Rule incident and must be evaluated under the Breach Notification Rule within 60 days.

Do I need a BAA with an ambient scribe vendor?

Yes, ambient scribe vendors are business associates because they create, receive, maintain, or transmit PHI on behalf of the clinician, triggering 45 CFR 160.103.

Are OCR penalties really enforced against small practices?

Yes, OCR has settled cases against solo dentists, small clinics, and single-physician practices, with penalties ranging from $25,000 to $250,000 for documentation and risk-analysis failures.

Does the FTC also regulate health AI?

Yes, the FTC enforces Section 5 of the FTC Act and the Health Breach Notification Rule against health apps and AI tools that fall outside HIPAA, as seen in the BetterHelp and Cerebral orders.

Is a SOC 2 report enough to prove HIPAA compliance?

No. SOC 2 is a useful signal but does not substitute for a signed BAA, a HIPAA-specific risk analysis, and the Security Rule safeguards required by 45 CFR 164.308 through 164.312.

Can I use Perplexity or other search LLMs with PHI?

No, consumer Perplexity and similar retrieval-augmented tools do not offer a BAA for consumer endpoints. Enterprise versions must be reviewed individually for BAA coverage and retention settings.