Before a practice lets AI tools and patient information come together, it needs a clear picture of what data flows through the service, who can handle it, and what happens after processing. A vendor’s product description or encryption claim alone does not answer those questions.
Use this checklist to structure a conversation with the vendor, your privacy or compliance lead, and the people who manage your systems. The goal is not to assume every AI tool is unsuitable. It is to understand the workflow and assess it before patient information is involved.
Map the data before you evaluate the tool
Start with a specific use case, such as drafting visit summaries, routing patient messages, or organizing documents. Then trace the information from the moment staff enter it until the service no longer retains it.
- What information goes in? Ask whether the tool receives names, contact details, appointment information, clinical notes, images, audio, records, or other identifiers. Include information pasted into prompts or uploaded as attachments.
- What does the tool generate? Identify whether it creates summaries, recommendations, classifications, transcripts, or other outputs that could include or reveal patient information.
- What else is captured? Ask about prompts, usage logs, diagnostic records, support tickets, and metadata. These may contain sensitive details even when staff do not intentionally enter them.
- Where does data travel? Ask which systems receive, store, or transmit it, including integrations with the practice’s existing systems.
- Who can access it? Clarify access by the vendor’s employees, technical support staff, and other service providers, and how access is controlled and recorded.
A useful way to keep this review grounded is to document the workflow, the data involved, and the people responsible for each step. An AI readiness assessment can help organize questions about existing systems and operational needs before a tool is selected.
Ask how the vendor uses patient data
Do not assume that data is used only to provide the feature your practice requested. Ask the vendor to explain its data-use terms in plain language and point to the contract or policy that governs them.
- Does the vendor use prompts, uploaded information, or generated outputs to train or improve models?
- Can the vendor or another provider review submitted information for support, quality, safety, or product development?
- Can the practice disable these uses, and does the restriction apply to every component of the service?
- Does the vendor share information with affiliates, analytics providers, or other third parties?
- Are data-use commitments documented in contract terms, rather than only in general marketing materials?
If a setting or contract term is unclear, ask for a written explanation of what it changes and what it does not change. A setting that limits model training, for example, does not by itself explain retention, support access, or subprocessors.
Review the vendor and its subcontractors
Ask the direct vendor to identify every subcontractor that may create, receive, maintain, or transmit patient information. That can include cloud hosting, model services, support, or other technical providers. For each party, ask what information it handles, why it needs access, where it processes data, and what contractual protections apply.
Also ask how the vendor evaluates subcontractors, controls their access, and responds if a subcontractor changes or a service provider is added. Contractual safeguards should extend through the service chain where required, not stop at the company your practice pays.
HHS explains that a cloud provider maintaining electronic protected health information may be a business associate even if it cannot decrypt the information, and that subcontractors handling ePHI can also have business associate obligations; review the HHS guidance on cloud computing and ePHI when assessing the arrangement.
Decide whether a business associate agreement is needed
Ask whether the vendor handles protected health information on the practice’s behalf as part of the proposed workflow. The answer depends on the relationship and what the service actually does, not just the product label, a claim that data is encrypted, or the vendor’s general statement about HIPAA.
Where a business associate relationship applies, confirm that the required agreement is in place before the vendor handles the information. Ask whether relevant subcontractors are covered by appropriate agreements as well. Have your privacy or compliance lead, and qualified counsel when appropriate, review the workflow and documents. HIPAA does not automatically cover every organization or every health-related data flow, so the practice should establish which rules apply to its specific context rather than assuming one answer fits all.
Check retention, deletion, and the end of service
Get clear answers about how long the vendor keeps each kind of data. Ask separately about prompts, outputs, logs, backups, and data held by subcontractors. Find out whether retention differs by product feature, support process, or contract setting.
- Can the practice set or limit retention?
- What happens when the practice requests deletion, and how is completion confirmed?
- Are copies kept in backups or logs after active records are removed, and when are they cleared?
- Can the practice export records it needs before ending the service?
- What happens to patient information and AI-generated outputs when the contract ends?
Make sure the answers are reflected in the agreement or applicable service terms. Do not rely on an informal assurance if the contract describes a different process.
Assess risk and start with a limited pilot
Before deployment, document the intended use, the data involved, likely risks, safeguards, and who will review the tool’s outputs. Consider what could happen if the system produces an inaccurate result, exposes information to an unintended party, or becomes unavailable. The NIST Generative AI Profile offers a risk-management reference that practices can use to structure this review.
Begin with a narrow pilot and the least sensitive, limited data that can reasonably test the workflow. Confirm that access is restricted, staff know what not to enter, and a responsible person checks outputs before they influence patient care or operations. Track issues and revise the process before expanding use.
For implementation planning, connect the privacy review to the everyday work the tool is meant to support. AI implementation should account for existing systems and staff workflows, with the aim of minimizing disruption. This guide to AI process optimization can help frame the operational side of that discussion.
Make the decision with the full workflow in view
A healthcare practice should be able to explain what patient information an AI tool handles, how the vendor and its subcontractors use it, whether a BAA is needed, and what happens to the data throughout and after the service. If important answers are missing, pause before using patient information and resolve the gaps with the appropriate privacy, security, and legal reviewers.
AI Services Near Me serves businesses throughout Florida and Georgia and offers a free AI consultation. If your practice is assessing an AI workflow, contact the team to discuss the use case and the questions your review should address.