Dr. Kellie Owens: Ethical and Trustworthy AI in Health Care

Dr. Kellie Owens, is an assistant professor in the section of medical ethics at NYU Grossman School of Medicine. She's a sociologist and empirical bioethicist studying how digital technologies are woven into the fabric of healthcare and society and how governance structures can better support those affected by technological change. She also leads the responsible AI review process for the Division of Applied AI Technologies and with the IRB. PART 2 In medicine, we do know that examples of bias and harm from that bias have happened. The most famous case was brought to our attention from Ziad Obermeyer and his colleagues in Science in 2019. It was examining racial discrimination resulting from a commercial algorithm that was widely used by health systems to identify patients with complex health needs that might need extra support. Instead, we found that the algorithm was prioritizing white patients over black patients and was reducing the number of black patients who should have been enrolled in those care management programs by more than half. In some ways, there are pretty easy ways to fix these kinds of biases and problems if we're attuned to them. So once we see them, we can usually fix them. In this case, training the model using active chronic conditions rather than total costs would have significantly reduced bias.  The point is that understanding the social context in which an algorithm is developed and is operating in is really important. Otherwise, it can be pretty easy to make mistakes like this if you're not paying attention to disparate impacts in the ways that our choices in algorithm development can have impacts on fairness and outcomes for different groups. Another major ethics consideration touched upon in her presentation is mostly about data privacy.

Dr. Kellie Owens, is an assistant professor in the section of medical ethics at NYU Grossman School of Medicine. She's a sociologist and empirical bioethicist studying how digital technologies are woven into the fabric of healthcare and society and how governance structures can better support those affected by technological change. She also leads the responsible AI review process for the Division of Applied AI Technologies and with the IRB. Part 1 Dr. Owens began by indicating that by AI, she means any computer system that's designed to perform tasks that typically require human intelligence, which could be things like recognizing patterns, interpreting data, making predictions, or even making decisions. In the health care context, that means AI systems that analyze complex. medical data that assist with diagnosis or risk prediction that can help automate some routine processes.  She wants to talk mostly about the less fun side, the sort of limitations and potential harms that we have to be aware of to make sure that we're using this technology responsibly. Her focus today is on the people and process side of AI integration and not the technical side. Examples raised today focus specifically on health care implementation. She indicated that there are increasing calls from AI developers and users and regulators and folks in ethics to increase evidence standards for implementing new AI systems. The majority fall outside of FDA regulation, which means they really haven't undergone any formal review for safety and efficacy. The stated goals of ambient documentation, ambient scribes, are to give doctors more time with their patients or more time with their families. We are going to have to vigorously defend that time savings. Otherwise, that free time will be used in plenty of ways that don't touch the problem of physician burnout at all. She wants to be really intentional about how we think about time savings and efficiency regarding AI.

Dr. Capozzi is a board-certified orthopedic surgeon with specialty training in joint replacement surgery. After receiving his bachelor's degree from Columbia University, he obtained his medical degree from Mount Sinai School of Medicine. He completed his orthopedic training at Mount Sinai and the Otto Alfrank Adult Joint Reconstructive Fellowship at the New England Baptist Hospital in Boston. In addition to performing hip and knee replacements, Dr. Capozzi specializes in difficult revision surgery, utilizing the newest techniques in joint reconstructive surgery.   Part 2   Figuring out what happened took them less than a day because the cause of the crash was fuel exhaustion. The plane ran out of gas. Why it happened took about 15 months to figure out all that took place, how a plane runs out of fuel on its approach. So, what transpired? The crash analysis in the NTSB report indicates that there were significant communication errors. Keep this in mind when dealing with matters in the hospital. When there's really an urgent situation, you're very mindful of how you're conveying that urgency. To air traffic control, saying that we're running out of fuel doesn't mean anything significant. It's like if I'm in the operating room and I say that a patient's blood pressure is a little low. Well, pressure is always a little low. There's blood loss, there's anesthesia, there's pain medication, we're operating. The pressure is expected to be low. But if the patient's crashing, I would assume there'd be more sense of urgency in that statement. It is called mitigated speech when it downplays or sugarcoats or when wanting to be polite or the speaker is ashamed, embarrassed or trying to be deferential to authority. So doing away with mitigated speech, doing away with ambiguous terminology, and really being clear when there's a patient safety concern that needs to be brought up. Again, this is classic for an operating room.

Dr. Capozzi is a board-certified orthopedic surgeon with specialty training in joint replacement surgery. After receiving his bachelor's degree from Columbia University, he obtained his medical degree from Mount Sinai School of Medicine. He completed his orthopedic training at Mount Sinai and the Otto Alfrank Adult Joint Reconstructive Fellowship at the New England Baptist Hospital in Boston. In addition to performing hip and knee replacements, Dr. Capozzi specializes in difficult revision surgery, utilizing the newest techniques in joint reconstructive surgery.   Part 1 This presentation is about tenets of high reliability organizations involving modes of communication errors and communication training. When looking at studies of large organizations that perform complex and dangerous tasks, a couple of principles stand out that have some common themes. One is that the work tends to be highly technical and inherently dangerous. Very often there are high time constraints and time pressures that entail rushing to complete work in very tight environments, which sounds like any of our operating rooms or ICUs. At some point, most of these entities are going to fail spectacularly. The government defines a high-reliability organization as one that can operate in complex, high-hazard domains for extended periods of time without serious accidents or catastrophic failures, which pretty much defines what we do every day in the hospital. These organizations tend to have five key principles that define all of them the same way. One is a deference to expertise. Second is a reluctance to simplify. Third is a sensitivity to operations. Fourth is a commitment to resilience. Fifth is a preoccupation with failure. Two not-too-distant past failures are interesting. One is the Chernobyl nuclear disaster and the second is the Avianca plane crash on Long Island.

Dr. Mahya Beheshti is a physician-scientist at Rusk Rehabilitation, NYU Langone Health, where she works at the intersection of medicine and engineering to advance neurorehabilitation and assistive technologies. Her research focuses on neurorehabilitation, human–machine interfaces, EEG-based pattern recognition, and wearable technologies. She is particularly interested in how eye–hand coordination and multimodal neurophysiological signals can inform the development of intelligent rehabilitation systems for individuals with stroke, multiple sclerosis, and vision impairment. Fascinated by the powerful synergy between medicine and engineering, she earned her medical degree from Gulf Medical University and she is pursuing a part-time Ph.D. in mechanical and aerospace engineering while continuing her clinical and scientific work at Rusk. The discussion included the following topics: reason for choosing to do research in the area of visual impairment rehabilitation; research involving centers on sensory–motor coordination; key research findings; what is missing when doing traditional motor control assessments; how subtle timing disruptions between the eyes and hands affect daily functioning; and new investigations that may be undertaken in the next 12 months.

Dr. Sofiya Prilik is Clinical Director of Cardiopulmonary Rehabilitation and Transplant Rehabilitation and Clinical Assistant Professor of Physical Medicine and Rehabilitation at the NYU Rusk Rehabilitation department. Dr. Greg Sweeney is a Clinical Instructor in the Department of Rehabilitation Medicine at the NYU Grossman School of Medicine. Part 2 The discussion included the following topics: phases of cardiac rehabilitation; impact of mobile apps and wearables; common barriers that patients face in obtaining care; challenges aligning cardiac care with other comorbidities; provision of virtual services; research that led to modifying health care delivery models; evidence that patients who participate in rehabilitation have better health outcomes; role of government in expanding access to rehabilitation; and possible ways of making cardiac care more accessible.

Dr. Sofiya Prilik is Clinical Director of Cardiopulmonary Rehabilitation and Transplant Rehabilitation and Clinical Assistant Professor of Physical Medicine and Rehabilitation at the NYU Rusk Rehabilitation department. Dr. Greg Sweeney is a Clinical Instructor in the Department of Rehabilitation Medicine at the NYU Grossman School of Medicine. Part 1 The discussion included the following topics: nature of a webinar to be offered on June 12  at NYU on the topic of cardiac rehabilitation; measuring whether the webinar achieves its objectives; outcomes or activities attendees can be expected to undertake afterward; kinds of health conditions and problems that qualify patients to be eligible to participate in cardiac rehabilitation; role played by disparities in obtaining health care; and kinds of patients that benefit most from cardiac rehabilitation.

Dr. Steven Flanagan is Howard A. Rusk Professor of Rehabilitation Medicine and Chairperson of the Department of Rehabilitation Medicine at Rusk. He provides care for patients with physical and cognitive disabilities. He specializes in treating those who are recuperating from a stroke or brain injury. He is accompanied in this interview by Dr. Jonathan Whiteson who holds the rank of professor in both the Department of Medicine and the Department of Rehabilitation at Rusk Rehabilitation. Dr. Whiteson's skills and expertise focus on patients recovering from coronary and lung conditions. Part 3 The discussion included the following topics: measures used to monitor patient safety, quality of care, and patient satisfaction during the transition period; approaches to dealing with staff burnout; challenges that continue to exist after 18 years when positive outcomes proved highly difficult to achieve; and the likelihood that any new clinical divisions will be created at Rusk or the number of residency slots and accredited fellowships will be increased.

Dr. Steven Flanagan is Howard A. Rusk Professor of Rehabilitation Medicine and Chairperson of the Department of Rehabilitation Medicine at Rusk. He provides care for patients with physical and cognitive disabilities. He specializes in treating those who are recuperating from a stroke or brain injury. He is accompanied in this interview by Dr. Jonathan Whiteson who holds the rank of professor in both the Department of Medicine and the Department of Rehabilitation at Rusk Rehabilitation. Dr. Whiteson's skills and expertise focus on patients recovering from coronary and lung conditions. Part 2 The discussion included the following topics: Dr. Whiteson was asked if there is any kind of new knowledge he may have to acquire to be successful in his upcoming role and how he is obtaining it; he also was asked how he intends to preserve Dr. Flanagan's  accomplishments while balancing them with the development of new priorities; and short-term and long-term challenges involved in incorporating AI in the work at Rusk.

Dr. Steven Flanagan is Howard A. Rusk Professor of Rehabilitation Medicine and Chairperson of the Department of Rehabilitation Medicine at Rusk. He provides care for patients with physical and cognitive disabilities. He specializes in treating those who are recuperating from a stroke or brain injury. He is accompanied in this interview by Dr. Jonathan Whiteson who holds the rank of professor in both the Department of Medicine and the Department of Rehabilitation at Rusk Rehabilitation. Dr. Whiteson's skills and expertise focus on patients recovering from coronary and lung conditions. Part 1 The discussion included the following topics: when they both began working at Rusk; how Rusk looked 18 years ago (e.g., amount of external research funding); potential upsides and downsides of selecting an insider to take over leadership responsibilities versus bringing in an outsider; when planning for the transition was started; and the kinds of engagement with groups that could be affected by this transition, such as teaching faculty, clinicians, patients, and labor unions.