AI in Psychiatry: What APA Members Need to Know

Output from AI can be misleading or incorrect. It can draw conclusions that may lead to bias-related harm.

Knowing tech sources, algorithm features, and training methods may provide some insight into the accuracy of output and what biases may exist, but this information is often not disclosed by tech companies.

New evaluation metrics and benchmarks are needed to assess generative AI performance and utility of specific models in psychiatry.

We need to educate patients on the risks of using LLMs to answer personal health questions and share that LLMs do not maintain confidentiality.

If AI is used to make clinical decisions, patients must be informed.

APA urges caution in the application of untested technologies in clinical settings. Clinicians should approach AI technologies with caution, being aware of potential biases or inaccuracies and ensure that they are continuing to comply with HIPAA in all uses of AI.

Physicians remain responsible for the care they provide and can be liable for treatment decisions they make relying on AI that result in patient harm. As such, physicians should always carefully review any output guided by AI before implementing it into a treatment plan.

Physicians should ensure that they are transparent with patients about how AI is being used in their practice, particularly if AI is acting in a “human” capacity.

Regulatory guardrails and best practices exist to protect patient privacy (that is, HIPAA best practices), including informed consent, data minimization, data security, and accountability. To utilize LLMs or generative artificial intelligence, health care entities generally need to enter into business associate agreements with technology companies to safeguard protected health information.

Prompts entered into LLMs are stored on company servers and subject to the company’s privacy policy. Prompts containing private health information could be leaked or sold to third parties, compromising patient privacy.

Q  What are some available tools?

A  There are many LLMs available to the public. The most popular are ChatGPT, Google Bard, and Bing Chat powered by GPT-4. GPT-4All is an open-source ecosystem of chatbots that include uncensored models that can run locally and offline. Some LLMs focus on medical applications and include BioBERT, Clinical BERT, Med-BERT, and Google’s Med-PaLM2. There are generative AI models that can create images, video, and audio as well. A multitude of apps and services utilize generative AI technology to offer specific functionalities such as editing photos, creating presentation slides, summarizing journal articles, and more. Regardless of which model you try or use, keep the privacy considerations in mind to avoid HIPAA violations. References provided by LLMs are often false and generated, so make sure to double check output for accuracy.

Q  How can we use AI to our advantage especially regarding documentation without violating HIPAA or patient trust?

A  While publicly available models have the capability to minimally assist with documentation, the risks of HIPAA violations and inaccurate output are too great. Entering into a business associate’s agreement with a business focused on developing generative AI for clinical use may offer a HIPAA-compliant way to harness the technology as it continues to improve. ■