▶ 0:33:34The subcommittee will come to order and without objection, the chair may declare a recess at any time. Like many who have worn the uniform and received VA healthcare, I know the frustration when the system is slow, the paperwork stacks up, or the technology fails or doesn't lead us in the direction we're trying to go. And uh that's why this subcommittee's work is so critical and why it's important that we have the folks here joining us today.
▶ 0:33:58It's our duty to ensure VA's technology is efficient and reliable, helping veterans rather than standing in the way of their care. And that brings us to the focus of today's hearing. artificial intelligence or AI as it's of course commonly referred to right now. For some, AI sounds like a science fiction movie. Uh we've all seen many of them. Something only computer scientists worry about or even something scary because it's unknown and not well understood.
▶ 0:34:23It feels like today everything is about the
▶ 0:39:44Department of Veterans Affairs and how we can support the VA in closing those gaps through technology. However, today's review of artificial intelligence use cases at the Veterans Health Administration feels like a distraction. VA is struggling with the basics. We are here discussing the newest technologies while VA is still working with a crumbling IT infrastructure and still grapples to modernize systems and workflows.
▶ 0:40:12As the ranking member on the technology modernization subcommittee, I am certainly excited by the potential of both AI and innovation. AI could improve some of VA's challenges through large language models and higher processing speeds. We have seen promising studies of providers using AI to identify cancers more easily, improve patient outcomes, and ease clinician burnout by taking on more administrative tasks.
▶ 0:40:42VA has certainly been a leader in the research, development, and widespread usage of a number of significant and groundbreaking technologies, and it stands to do so again with AI. However, success in these efforts requires adequate resources and investments in its budgets, its processes, and its people. Veterans chose VA choose VA for the community it provides, for the people it employs, and for the fact that it is not driven by profit.
▶ 0:41:12What VA does best is make veterans feel seen and understood. As we have seen, AI can be a tool to provide decision support, ease provider burdens, and help with note-taking. so doctors can be more present with the patient. But we should also acknowledge that it is not the answer to every challenge the VA faces. Also, we as a committee and as Congress need to have a real conversation about AI policy and how to implement it safely.
▶ 0:41:41I am excited about the opportunities that AI presents, but I'm not convinced that VA is prepared to deploy this technology just yet. I have a number of concerns that I hope to address today like the lack of regulation and governance structures and the need for better transparency around what data is involved in training such models.
▶ 0:42:02Further like all technology modernization efforts implementing AI successfully requires a highly skilled adequately staffed workforce. Almost two weeks ago, the acting head of the Department on Government Efficiency stressed the need to quote hire and empower great tech talent in government end quote. I couldn't agree more with that.
▶ 0:42:25However, I think we should all note the irony of that statement considering OIT is proposing a massive reorganization and intends to cut at least 20% of its workforce. Success is also reliant on strong IT leadership. If OIT is in fact undergoing significant changes to its organizational structure, priorities list and workforce makeup, we need a confir confirmed chief information officer at VA.
▶ 0:42:55This position is particularly critical as we see the acceleration and progression of modernization efforts at the department. It seems the VA still lacks a coherent enterprise IT strategy, leaving projects like AI integration to happen in silos. Without stable and competent leadership, veterans and VA employees will continue to be stuck with cobbled together systems and workflows that don't meet their needs.
▶ 0:43:22rather than a solid strategy for technology usage to guide its decisionmaking. I hope that we can get some clarity into the administration's plan to propose a nominee for the CIO position and that one and that one can be confirmed before many of these sub substantial changes occur. Lastly, I understand the subcommittee held a similar hearing in January of 2024, though neither I nor the chairman were on this subcommittee at that point.
▶ 0:43:53In that hearing, data privacy was an intrinsic part of the discussion. I hope that it still is the case today. As we become more interconnected through technology advancements like artificial intelligence, we must become increasingly aware of the concerns about the privacy of users data, especially in healthcare.
▶ 0:44:14Since la since this last hearing, the department has been entangled in multiple cyber security incidents which have potentially place veterans data at risk. Though many of these BR breaches have been targeted at VA contract have been targeted at VA contractors, veterans data has still been implicated and VA maintains some responsibility for its safety.
▶ 0:44:39Though I do feel that this hearing is perhaps too early considering VA has yet to develop and release some of its policies and plans to align its efforts with the administrations. I hope to hear from our VA witnesses today about how data privacy and security as well as the views of both VA employees and patients will be integrated into such plans. Thank you and I yield back, Mr.
▶ 0:45:04Thank you uh Ranking Member Bazinski and I u I join you in um making sure that we have adequate ethics guardrails around this and certainly privacy is is paramount in that uh as well. Um, and now I want to introduce our witnesses. And again, thank you for joining us today. From the Department of Veteran Affairs, we have Mr. Charles Worthington, the chief technology officer and chief artificial intelligence officer. Thank you for being here. Accompanying Mr. Worthington is Dr.
▶ 0:45:31Evan Kerry, the acting director over the National Artificial Intelligence Institute at the VA. We also have uh Mr. Sid Gatek. Did I say that correctly? Thank you. the chief technical adviser from the National Artificial Intelligence Association and Dr. Muhammad GMI, an assistant professor at Michigan State University. Go Green. Thank you for being here today as well.
▶ 0:45:58Um and finally from the Government Accountability Office, we have Miss Carol Harris, a familiar face to all of us on this committee. Thank you again for being here and joining us. Um and she's also the director of IT and cyber security at the GAO. And uh again, thank you all for being here. At this time, I ask the witnesses to please stand and raise your right hand. Do you solemnly swear under penalty of perjury that the testimony you're about to provide is the truth, the whole truth, and nothing but the truth?
▶ 0:46:27Thank you. And let the record reflect that all witnesses have answered in the affirmative. Mr. Charles Worthington, you are now recognized for five minutes to deliver your opening statement on behalf of EA. Chairman Barrett, Ranking Member Bazinski, and distinguished members of the subcommittee, thank you for the opportunity to discuss the Department of Veterans Affairs use of artificial intelligence to enhance healthcare and services for veterans. Your steadfast support of the veterans and their families is invaluable. Uh, I'm joined today by Dr.
▶ 0:46:56Evan Kerry, acting director of the National AI Institute in the Digital Health Office of the Veterans Health Administration. While AI is not new to VA, recent advancements in AI systems present a tremendous opportunity to improve VA services. When used effectively, AI can improve the efficiency and accuracy of many time consuming and errorprone tasks that create burdens for VA staff and veterans alike. That's why VA is rapidly working to capitalize on this technology.
▶ 0:47:22Our strategic vision is to make VA a leader in AI, providing faster services, higher quality care, and more cost-effective operations. We will aggressively deploy this new technology while remaining committed to strong controls that ensure security, privacy, and effectiveness of our technology We have distilled this vision into five key priorities. First, we are aggressively expanding AI access across our workforce. Second, we are reimagining high impact workflows through AI and automation.
▶ 0:47:52Third, we're prioritizing investment in data and infrastructure that supports those high potential use cases. Fourth, we are cultivating an AI ready workforce. And finally, we're ex executing transparent and effective governance, an essential requirement to maintain veterans trust. VA is already bringing this strategy to life, making significant investments in AIdriven tools. In 2024, our AI inventory had 227 use cases in it, which was nearly 100 more than the previous year.
▶ 0:48:20And we expect this growth to continue in 2025 as we prepare for our December update to this inventory. These investments are delivering tangible results. I'm pleased to report that all VA employees now have access to secure generative AI tool to assist them with their work. And in surveys, users of this tool are reporting that it's saving them over two hours per week. Additionally, over 2,000 VA staff and contract software developers are using an AI software development co-pilot tool, enabling faster delivery of features that help veterans.
▶ 0:48:49AI is also revolutionizing clinical care. In fact, 82% of VA's AI use cases come from the Veterans Health Administration. VA's stratification tool for opioid risk mitigation uses machine learning to identify veterans at high risk of overdose and suicide, enabling healthcare teams to review and intervene effectively. Since 2017, the Reachvet program, as you mentioned, has used AI algorithms to identify over 130,000 veterans at elevated risk, improving outpatient care and reducing suicide attempts.
▶ 0:49:18AI assisted colonoscopy devices have increased adenoma detection rates by 21% reducing late stage cancer incidents and mortality. And thanks to groundbreaking groundbreaking research by folks like Dr. Rafie Haggopian and Dr. Evan Kerry, VA is exploring how AI could help providers detect heart disease uh earlier by reviewing the millions of CT scans that are not currently evaluated for cardiovascular disease risk at all. As we advance our AI deployments, protecting veterans data remains paramount.
▶ 0:49:48All AI systems approved for use at VA must meet VA's rigorous security and privacy standards before receiving an authority to operate. Additionally, consistent with OMB's policy, we conduct a thorough agency level review of each AI use case to ensure that it meets the government standards. We will publish the results of this review in our annual AI inventory, positioning us as one of the most transparent healthcare systems in the country with regards to our use of artificial intelligence. Despite our progress, adopting AI tools does present challenges.
▶ 0:50:18As you mentioned, integrating new AI solutions with a complex system architecture and balancing innovation with stringent security compliance is crucial. Recruiting and retaining AI talent remains difficult and scaling commercial AI tools incurs additional costs. This underscores the importance of full congressional funding for VA to continue this critical work. In conclusion, the Department of Veterans Affairs is committed to harnessing AI to improve the lives of veterans.
▶ 0:50:44Through strategic investments in AI tools and workforce capabilities, we strive to deliver faster, higher quality, and more cost-effective services. Your continued support is vital for VA to lead in AI innovation and set a benchmark for responsible AI use in government. Thank you for the opportunity to discuss our strategy and we look forward to your
▶ 0:51:05Thank you, Mr. Worthington. The written statement, Mr. Warthingington will be entered into the hearing record. Mr. Gota, you are now recognized for five minutes to deliver your opening
▶ 0:51:17Thank you very much.
▶ 0:51:22Is your uh microphone on, sir?
▶ 0:51:26There you go. My name is Sid Gate and for almost three decades I've designed and deployed artificial intelligence and forecasting systems across finance, healthcare, pharmaceuticals, media and government. I currently serve as the chief technology officer, chief technology adviser for the national artificial intelligence association, the premier organization representing 1500 businesses in the advancement of AI and I'm also the founder and chief executive officer of increase alpha where we use artificial intelligence to predict stock prices and we license these predictions to hedge funds in the federal government.
▶ 0:51:56and I served in the general services administration for four years where I was the a director of the data and analytics center of excellence. In that role, I co-authored the federal AI maturity model uh three years before AI took the world by storm. I also contributed previous executive orders on uh the critical issues of data privacy and data security. At increase alpha, I increased a predict I architected a predictive AI model that generates alpha once thought impossible. A deep learning system exceptionally accurate at predicting equity prices.
▶ 0:52:25increase alpha far exceeds multiple industry benchmarks including accuracy, sharp ratio and alpha generation. The solution itself is not based on large language large language models at all but is purpose-built designed for this specific need. I want to emphasize that this company and our solution is uh completely unrelated to the Department of Veterans Affairs and has no bearing on today's testimony. I mention it only as an example of how AI when carefully designed with a clear purpose can achieve exceptional effectiveness.
▶ 0:52:55Taken together, this diverse background spanning academia, government, and industry has given me the rare opportunity to actually build AI systems that work well in the real world. Because I've spent my career outside the orthodox worlds of academia, venture capital, and big tech, I'm also not beholden to herd mentality. Instead, I bring an expert independent perspective, which is especially valuable now when much of the world is caught up in the art of the possible with AI.
▶ 0:53:17When what is most urgently needed is a sober understanding of what is safe, practical, and ready to serve the LLMs like chat GPT, Claude and Gemini are a powerful subset of AI but they come with their own set of problems specifically in healthcare where hallucinations and sickopanty on the part of chat bots can lead to susceptible users down psychological rabbit holes which is why it's important to clarify that AI is bigger than just chat GPT and its competitors to use an analogy the steam
▶ 0:53:48engine transformed society fueling the industrial revolution. While steam power still exists today, it gave way to other forms of power over time. Until steam engines were used to create the first railroads, no human had ever traveled faster than a horse. This new form of transportation opened the world's eyes to what is possible. Just as Chacht has shown the world the art of the possible with artificial intelligence. But early train travel was dangerously unreliable.
▶ 0:54:11Accidents were frequent, derailments common, and thousands of lives were lost before rail systems matured into safe networks we know today. The lesson is clear. Revolutionary technologies will evolve and improve over time when the private sector and government work in collaboration. The same applies to artificial intelligence. As the committee gathers information on how to modernize in technology at the VA, I'd like to offer a few pieces of from my many decades of front lines in building and implementing advanced analytical solutions.
▶ 0:54:44As I mentioned, for the last several years, the world has been consumed with LLMs to the point where AI has become synonymous with it. However, that is not the case. Many other types of AI may have similarities to these models, but function very differently. Technologies that specialize in interpreting and understanding images, video, and audio, for example, or technologies that are better suited to working with numbers and symbols instead of words, a new technology that has yet to be invented. There is an old adage that when you are a hammer, everything looks like a nail.
▶ 0:55:13The world has become so enamored with LLMs and rightfully so. Interacting with them can feel magical, giving you the sense that they are real people, but they are not. This may be why little to no investment is being made in these other areas. At increase alpha, we demonstrate clearly what can be done with other forms of artificial intelligence. I began building our models at the same time as a research underlying chat GPT was was published. I'd also encountered the same compute cost, energy, and reliance on NVIDIA we still we see today.
▶ 0:55:41But I took a different approach to conserve resources and focus on simplification using predictive intelligence which led to lean AI models that use a minuscule amount of data compared to LLM and which are small enough to run on a cell phone. What does all this mean for the VA and the well-being and care of our veterans? I can't claim to know. No one really does. But I want to leave you with a prediction. I believe that we truly are on the verge of a scale of a revol revolution on the scale of the industrial revolution.
▶ 0:56:10So if if I could leave you with one idea today, it would be this. AI is much bigger than today's LMS. And it is these technologies, many of which have yet to be invented, that will enable the VA to execute on its
▶ 0:56:24Thank you.
▶ 0:56:25Thank you, uh, Mr. Gatic. The written statement of Mr. Gatic will be entered into the hearing record and, uh, appreciate your remarks. I think if we all use chat GPT for cat memes, it will not be meeting its full potential and leaving a lot of things behind. So, thank you. Um, Dr. uh Gasemi, you are now recognized for five minutes over your opening statement.
▶ 0:56:54Chairman, ranking member and members of the subcommittee, thank you for the opportunity to speak I am a scientist and an entrepreneur that's focused on artificial intelligence, but especially its applications to healthcare.
▶ 0:57:10The views I'm going to share today are my own, but they're informed by roles I've played as a professor at Michigan State University, where I direct a research laboratory on AI and its applications to the health sciences.
▶ 0:57:27I'm also going to bring a perspective as the founder of an AI consultancy, Gamut Corporation, which has helped large pharmaceutical companies, insurance companies, as well as health systems plan and execute their AI strategy. I want to be clear, I'm not a veteran health specialist.
▶ 0:57:50My perspective is on how artificial intelligence can broadly advance care in ways directly relevant to the needs of patients and this very critically includes our veterans. The subcommittee has identified in their invitation letter three priorities for AI and health. These were transforming healthcare delivery, streamlining services, and improving outcomes.
▶ 0:58:17So, I'm going to frame my remarks around three roles that AI can play to help with these three priorities. The three roles are automation, which is reducing low value work through the use of machines. Augmentation, which is having a machine assist a human in a task.
▶ 0:58:43to strengthen clinical decisionm as an example and insights which is allowing us to extract complex patterns from data patterns far too complex for us to discern just with our human intuitions alone so let's talk about these three first AI can h can transform what happens during care itself clinicians today spend hours on paperwork but AI scribes can generate notes automatically so they
▶ 0:59:13can focus more fully on patients. We've heard that from more than one person in the conversation today. In emergency rooms, decision tools powered by AI can help identify the sickest patients sooner and get them treated faster. And continuous monitoring systems can pick up on the early signs of decline, like sepsis, long before they would be obvious to our human eyes. These tools make the encounter safer, timelier, and more patient centered.
▶ 0:59:47AI can not only streamline what happens during care, it can streamline the plumbing of health care itself. Missed appointments waste scarce clinician time. Automated reminder systems which don't have to use a large language model or a sophisticated tool like chat GPT can reduce these no-shows and save that time.
▶ 1:00:17Patients also too often fall through the cracks between primary care and specialist visits. AI can flag missing referral information, track follow-ups, and prevent all these gaps. And when imaging or labs reveal unexpected findings like god forbid a lung nodule discovered by chance, AI tracking systems can ensure these findings are followed up on so that the treatable conditions don't get overlooked.
▶ 1:00:48This is how we reduce wasted effort and ensure smoother, more reliable care. In conclusion, artificial intelligence is not a silver bullet. I say this as a person who's been working on developing the methods for several years, but it can already help with the subcommittee's three priorities.
▶ 1:01:11It works best when it reduces low-value work, strengthens rather than replaces clinical judgment, and turns complex data into actionable insights. To succeed, we need disciplined pilots, clear metrics, and safeguards for safety, equity, and privacy.
▶ 1:01:30If deployed with CLA with care, AI can return time from paperwork to patients, ensure that critical findings are not missed, and support clinicians in their hardest I look forward to our conversation, and I'm grateful for the invitation to be here with you today.
▶ 1:01:49Thank you, Doc. Uh the uh written statement of Dr. Gasemi will be entered into the hearing record and Miss Harris, you're now recognized for five minutes to deliver your opening statement on behalf of GAO.
▶ 1:01:59Chairman Barrett, Ranking Member Budinski, and members of the subcommittee. Thank you for inviting us to testify today on the use of artificial intelligence at VA. Develops in generative AI, which is a subset of AI, which can create text, images, video, and other content when prompted by a user, have revolutionized how the technology can be used in many industries, including healthcare and at VA and other federal agencies. AI holds substantial promise for improving the operations of government agencies.
▶ 1:02:29However, it can increase risk for agencies and poses unique oversight challenges because the source of information used by AI is not always clear or accurate. Given the fast pace at which AI is evolving, the government must be proactive in understanding its complexities, risks, and societal It should also be noted that VA has experienced long-standing challenges in managing its IT projects and programs, raising questions about the efficiency and effectiveness of its operations
▶ 1:02:59and its ability to deliver intended As requested, I'll briefly summarize our prior work on the department's AI use and challenges, as well as principles and key practices for federal agencies, including VA, that are considering and implementing AI systems. In July 2025, we reported that VA's AI use cases increased from 40 in 2023 to 229 in 2024.
▶ 1:03:26For example, VA is developing a generative AI use to automate various medical imaging processes. This use may enhance VA's ability to analyze medical images, integrate existing and new data workflows, and create summary diagnostic In the health and medical sector, agencies have adopted generative AI to advance medical research and improve public health outcomes, including at VA.
▶ 1:03:51It's also worth noting that of the 229 use cases, 64% were considered to be high impact AI, meaning that their capabilities impact the rights and/or safety of individuals or entities. And looking at just VHA, that percentage increases to 72%. The department also reported to us a number of challenges they face in using and managing generative AI. The full list is noted in my written statement, so I'll only highlight a few here.
▶ 1:04:20Challenge one, complying with existing federal policies and guidance. VA officials shared that the existing federal AI policy could present obstacles to the adoption of generative AI, including in the areas of cyber security, data privacy, and IT Challenge number two, having sufficient technical resources and budget.
▶ 1:04:40Genai can require infrastructure with significant computational and technical resources and VA noted challenges in obtaining or accessing the needed technical resources and also in having the funding necessary to establish those resources and support desired AI And the last challenge, hiring and developing an AI workforce.
▶ 1:05:02Among other things, VA reported difficulties in establishing and providing ongoing education and technical skills development for their current workforce. VA officials told us they are working towards implementing the new AI requirements in OM's April 2025 memorandum and doing so will provide opportunities to develop and publicly release AI strategies for identifying and removing barriers and addressing the challenges I noted.
▶ 1:05:29Additionally, GAO has identified a framework of key practices to help ensure accountability and responsible AI use in the design, development, deployment, and continuous monitoring of AI systems. Our framework is organized around four complimentary principles that address governance, data, performance, and monitoring. Consideration of the key practices in this framework can help VA as it considers, selects, and implements AI systems.
▶ 1:05:57Lastly, I'll mention that we have 26 open recommendations to VA concerning the management of its IT resources. If the department implements these recommendations effectively, it will be better positioned to overcome its long-standing challenges in managing its IT resources and will improve its ability to address the rapidly changing AI landscape. That concludes my statement and I look forward to addressing your questions.
▶ 1:06:22Thank you, Miss Harris. and the uh written statement of Miss Harris will be entered into the hearing record. And again, thank you to all of our witnesses. And we'll now uh proceed to um questioning. Um I'll recognize myself for five minutes to uh begin questioning. Um I'm going to start with Mr. Worthington.
▶ 1:06:41Um, the VA, uh, I know we've got a lot of concerns obviously about data security, data privacy, what can be used, what can be modeled off of veteran information, but the VA requires vendors to sign contracts directly stipulating that it will prevent secondary use of veteran data and number one, can you kind of walk us through how that works and how are you making sure that companies actually follow that rule?
▶ 1:07:06Thank you for the question, Chairman Barrett. I think it's extremely important that everyone understands that there's there's not a second set of rules for AI systems. Uh in the VA, we have a a very uh clear and stringent set of rules around both security and privacy for any technology system.
▶ 1:07:23And so before we bring a system into production, we have to review that system for its compliance with those requirements and ensure that the partners that are working with us on those systems uh attest to and agree with those requirements. And so AI systems receive an authority to operate just like any other system would before we would put uh veteran data into the
▶ 1:07:45Okay. So um I appreciate that. Um, for example though, I know um the large language model kind of most stereotypical use of AI. We're going to be looking at, you know, the the millions of record that the VA has and then modeling patient outcomes from that and then looking kind of retrospectively to see where people are at on that spectrum today and say, well, if if we know this condition led to 10 years later a a worse condition over here,
▶ 1:08:16how can we stem that off uh earlier? um if we allow a a AI vendor to have access to that to cultivate that knowledge, is that something that could be then used as an outgrowth in another way for like another um like as a research tool for other things?
▶ 1:08:35For example, if a person has a a predisposition to kidney disease or diabetes or something like that and we can look retrospectively at their health record to show that they had certain indicators ahead of time, wouldn't we want that to be to the benefit of all medicine and not just within the VA?
▶ 1:08:55Uh yes, I I think that as you're mentioning in the training phase of models, which VA does occasionally do that uh if we work with a vendor, we make sure that the agreements say that any protected health information can only be used for that specific purpose that we have uh contracted with. And often that's taking place in environments that VA already runs and controls.
▶ 1:09:15Now, when we're talking about using a large language model, uh those are provided typically via one of the big cloud service providers and those environments are uh set aside in a VA boundary that basically the vendor has to attest that they already meet VA security requirements. So, when we're sending information to a large language model to get feedback back from that model, we're using a version of that model that has been made secure to meet government standards.
▶ 1:09:42Okay. And um I will fully confess that I'm not an expert on this, but a would a large large language model allow a practitioner to say, "I have a veteran presenting with these conditions. What are the risk factors that I ought to look for to maybe run tests that wouldn't ordinarily be otherwise um top of mind?
▶ 1:10:02There could be a variety of AI approaches for a use case like that. And uh Dr. Dr. Kerry may just quickly provide a couple of examples of those sorts of uh decision support type use
▶ 1:10:15Absolutely. Thank you. And it's a fantastic question. I think there's two versions of that. As you note, there are tools where a provider can get general advice and they might specifically articulate the needs of the veteran and sort of the conditions that they're looking for to point out to make sure they follow the different procedures that are recommended and identify the guidelines. those tools are available within the VA.
▶ 1:10:36Okay. So, um after the passage of the PAC Act, you know, we have this burn pit registry and everything and they're supposed to track veterans and conditions that arose from that. Um obviously the specific information about a particular veteran we want to have protected and and not revealed. But if there are um outcomes of that that could be useful to um to you know human medicine in total is there a way for that to be um revealed?
▶ 1:11:05Yeah, thank you for the question and VA does have as you note a very large amount of health data and we have a robust more than anybody in the world I
▶ 1:11:12That's right. Um we have a robust tradition of research to advance not just VA healthcare but healthcare overall and we are seeing an increasing interest in using that data uh for AIdriven research papers like the one that Dr. Kerry re recently wrote.
▶ 1:11:27Okay. And that's the cons like the benefit but also the concern is we obviously have a a large repository of medical data but if that is being used or to the benefit of a a curator of artificial intelligence should the VA be you know should that be brought into account for the the cost of services and other things like that what I don't want is a provider to come in and leech that information out solely for their benefit.
▶ 1:11:57um while not providing a benefit to the VA and to the veterans as well.
▶ 1:12:01We agree.
▶ 1:12:02Okay. Thank you. Uh ranking member
▶ 1:12:06Thank you, Mr. Chairman. Um uh Dr. Kerry and Mr. Worththington, thank you so much for both being here. Um, I understand that several of VA's AI use cases like the ambient uh dictation pilot intend to use an opt-in practice for consent uh for systems that are perhaps less directly veteranfacing like the use of AI and benefits determinations or medical assessments.
▶ 1:12:31How is the department educating veterans on these use cases uh to ensure for their
▶ 1:12:38at a very high level? And thank you for the question. uh we are using our AI use case inventory as the way to catalog all of the uses of AI and make sure that that's publicly available and so when there is not as you mentioned a like a onetoone interaction that provides the opportunity to explain directly what's happening as there is in many healthcare settings uh what we're relying on is our publishing of the overall AI strategy and use case to explain how the department is using AI in various uh products and services
▶ 1:13:08okay so other than just that general awareness is For veterans, is there any way to kind of draw their attention to this so that they know that you know what their situation might be using to inform an AI model?
▶ 1:13:20We're always listening for veterans feedback through a variety of mechanisms and reacting to that. And that's true of AI situations and nonAI situations, but we certainly want to monitor this for AI in particular because I think maintaining veterans trust in the VA as we introduce these new technologies is going to be critical.
▶ 1:13:36Okay. Um and then Mr. Mr. Worththington, I'm glad um that you and your teams are committed to transparency and AI use cases at the department. Um that is commendable. However, there have been reports that certain employees had access to certain data sets and systems within VA's enclave um which may have been used for AI related operations. So, I have some specific employees I want to mention by name and then I have some questions for you. Um I'm going to ask about these employees.
▶ 1:14:06Justin Fulture Sahil Levin Levvenia um Christopher Russos Payton Railings Carrie Vulpert or John Kovville. So I'm just looking for like a yes or no to these questions. Did you ever work with any of those um
▶ 1:14:26Uh yes, I've come across several of
▶ 1:14:28Okay. Are or were these individuals affiliated with the Department of Government Efficiency? I am not exactly clear on the relationship. I believe they're VA employees and um at points they were introduced as also being part of the uh doge movement.
▶ 1:14:46Okay. Um did any of these employees access data sets that included VA patient medical records or other personally identifiable information?
▶ 1:14:55I am not aware.
▶ 1:14:57Okay. Um, were you or anyone you know ever asked to duplicate data sets by these employees?
▶ 1:15:05Uh, no, I was not.
▶ 1:15:07Okay. And can you commit to me that no veterans data was removed from the Department of Veterans Affairs?
▶ 1:15:15As far as I understand, all the VA employees follow all the VA IT security processes and procedures and that that was a key priority for all of us uh and always is a key priority.
▶ 1:15:26Okay. Okay. Um, Mr. Worthington, um, almost two weeks ago, the acting director of the US Digital Service noted that the federal government needs more tech employees to and to hire and empower great talent. Do you believe that VA shares that sentiment?
▶ 1:15:42Uh, yeah, I do. I think having technologists in government is critically important, as is having great researchers and doctors.
▶ 1:15:48Okay. Uh, Secretary Collins has often noted the importance of VA employees in direct care roles, disregarding the importance of what he might call support employees in the provision of this work. Do you believe that this type of rhetoric has helped the department to recruit and retain tech talent? I think the the good thing about working at the VA is our mission is so clear and the mission of serving veterans is the the most important one that I've worked on in my tech career and I think there's many technologists across the country that are willing to sign up for that mission.
▶ 1:16:18Uh and I I love trying to recruit those people to my team.
▶ 1:16:21Okay. Um Miss Harris, real quick on a followup. Uh GAO's artificial intelligence accountability framework notes the workforce as a key component to ensuring effective AI application. How does a highly skilled technical workforce ensure adequate scalability of AI applications and protection of veteran data?
▶ 1:16:39Well, while there is great excitement around AI because of the potential to improve VA operations, there's also significant concerns, the ones that I articulated earlier about cyber security, intellectual property as well, built-in bias in the AI system, um, as well as environmental and other concerns. So we want to make sure that we have a workforce that understands both the potential of these systems but also understands the risks in in AI as well. So having those two are are vital.
▶ 1:17:08Okay. Thank you and I yield back.
▶ 1:17:10Thank you uh Mr. Latrell.
▶ 1:17:12Thank you Mr. Chairman.
▶ 1:17:15Mr. Cassim, you you laid out a very well articulated plan of attack on how the VA could tackle this healthc care artificial intelligence kind of combining of forces. the problem is you've sounds like you've never worked with the United States government because that is what kills this effort is the United States government.
▶ 1:17:41And Miss Harris, you your opening statement was very well articulated and you hit every single And then the problem is we have such an issue with the VA because it's a big machine and we're trying to compound or we're trying to bring in artificial intelligence to streamline the process. And you have 172 different VA facilities plus satellite campuses.
▶ 1:18:08And that's that's 172 different silos and they don't work together. They don't communicate very well with each other. We've spent almost $16 billion trying to push electronic healthcare records across multiple uh facilities. And now we're going to try to tackle artificial intelligence as well. And in 2024, we had 229 AI actions. Correct, Mr. Warthingington?
▶ 1:18:35Yes. Approximately. What site if did that come from? Because I would dare say that that didn't come from all or every single VA installation. I that sounds like that sounds to me that's collected from like a few. Is that correct?
▶ 1:18:50We did attempt to have a pretty comprehensive review process to gather all of the uses of AI across the country. So we
▶ 1:18:58I didn't get anything out of that.
▶ 1:19:00It was almost a yes or no question, but go ahead again. Try.
▶ 1:19:03Yes. I I believe that the AI is being used at facilities across the country and this inventory covers those uses
▶ 1:19:09because the conversations I have with multiple sites is they don't have artificial intelligence capabilities because their sites aren't ready or they don't have the infrastructure in place to do that because we keep compounding software on top of software and some sites can't function at all with new software they're trying to implement. That's a pretty fair statement. Correct.
▶ 1:19:31I I would agree that having standardized uh systems is a challenge at the VA and so there is a a bit of a difference in different facilities although I do think many of them are starting to use AI assisted medical devices for example and a number of those are covered in this inventory.
▶ 1:19:45So how do we fix the problem? Again I'm going to ask you sir because I usually ask everyone that's sitting in front of me from the VA is how would we fix this problem? Sean Mr. Gatic and Mr. Cassimi have probably thought about this quite a bit before they showed up in front of us, but again they haven't actually I don't know this for certain. I may be throwing this at you can of course correct me if you like but I don't think they've had to deal with the United States government I and also the VA. Now how long have you been in this position sir?
▶ 1:20:10Uh I've been at the VA uh nearly 10 years in this position for about uh two years as chief AI officer. So what comes first the communication between the sites and the ability to ask that information questions which we don't do that or we don't have the ability to do that.
▶ 1:20:30Do we run the implementation of artificial intelligence in parallel with that or do we have to do one before the other in my personal opinion we can't wait because AI is here whether we're ready or not and increasingly every solution we buy from our partners in the private sector is going to have it embedded inside of it. So I think our challenge is we need to come up with very good standard templates that every site can use and allow those standard tools to be deployed.
▶ 1:20:58Things like the the VA GPT tool that I mentioned which is now available to every VA employee uh in a standard way
▶ 1:21:04since the Department of Veterans Affairs houses the most important data set on the planet arguably and everyone wants to touch it including Dr. Dr. GMIA at Michigan State, I would had to guess, especially when you were at MIT and Cambridge, I'm sure. Pretty impressive resume, sir. Everybody's trying to touch it. Everybody wants to be a part of it.
▶ 1:21:27And you have to deal with every single subject matter expert that walks through your door that says, "I'm the best." And I can assure you, every one of those corporations and companies walks into our office as well. So, the question is, who is it? Who do you vet? And who's going to touch it? Because it can't be everybody. We don't have an in my personal opinion that I'm not aware of.
▶ 1:21:48We don't have an enclave that can house all of our information where everybody can get in there and not steal So implementation of implementation of artificial intelligence which we don't have the ability to regulate. So the question is who will do that or do you have the AI system itself regulate itself? I think it's a great observation and concern. That's one we share.
▶ 1:22:11And the reason why we are putting every every AI use case through that review process is to ensure that if it's being used with real veteran data that it meets VA's stringent security requirements.
▶ 1:22:24Thank you, Mr. Chairman. Thank you, sir. I yield back.
▶ 1:22:28All right. Thank you. Uh Mr. Sherless
▶ 1:22:32thank you so much. Thank you so much. Um I wanted to kind of piggyback on off of some of um Representative Latrell's questions. You mentioned standardization and we know now from doing this for years that standardization in the VA has not been our strong suit. Are there any um things that you've learned from our lack of standardization for all of our uh electronic medical records? We've been consistently having an issue there with standardization. So I have two questions for you.
▶ 1:22:57First, are you confident that you can actually have a standardization mech mechanism that will be able to have a smooth transition and Thank you for the question and it is a a critical uh topic for us and I do think that the investments this committee has helped make over the past years have helped with that.
▶ 1:23:17We do have uh for example in the space of decision support we have an investment that allows AI assisted decision support tools to be uh purchased or built uh and then deployed to every Vista site and also to every
▶ 1:23:30So I guess my question really is because you know like I said we've been trying to be successful here and it hasn't been. So, how confident are you now? And what are the missing links for standardization when it comes to AI? Because AI has some complexities that I think we can all acknowledge, especially when it comes to biases.
▶ 1:23:45And if we're going to implement AI into our system, we want to make sure that we have precise implementation and we also take into consideration responsible implementation of AI, which actually addresses the biases immediately, that deals with security immediately. So I was going to go into those questions first, but I said I can't even go there if we don't deal with standardization. So what have we learned? How confident are you?
▶ 1:24:10Or should we really be taking some time to step back and look at standardization again but through a mag a magnifying glass to make sure we get it right?
▶ 1:24:18I do feel confident that we are approaching this in a enterprise approach. And so that's why partnerships with the VHA and our our colleagues like Dr. Kerry is so critical. So AI is both a new area. It's one we need to be able to experiment in before we commit to that enterprise solution. But then once we commit, we don't want to have uh you know every medical center buying its own its own version of the same product. And so we've got a pretty careful balance of that innovation. But we are doing structured pilots to help us decide what to purchase and what to deploy to the
▶ 1:24:48So I wanted to talk more about the implementation and development because we know that most of the biases will be during the development phase and also the implementation phase. What are you doing specifically to make sure that these biases aren't being inherently put into the system to make sure that all of our veterans actually have access to equitable care?
▶ 1:25:06Yeah, that is a a great question and it's a concern that is uh of critical importance for us as we adopt AI. Uh the office of management and budget in their policy has determined uh defined high impact use cases. So those would be things involved in healthcare or benefits. And they provided a set of requirements that any AI needs to meet before they're used. And so some of the the highlights of those are uh pre-eployment testing to make sure that the model performs well across different different demographic groups.
▶ 1:25:35But not just pre-eployment testing, also ongoing monitoring so that we can make sure that the the models perform well over time. So
▶ 1:25:41could you tell me how you're doing that? Because we've been reading um I've been loving this AI conversation. and I've been looking at it through all spectrums and one of the articles that I'm going to actually ask to put into the record it talks about the clinical decision making for in the implementations and I also want to hear from Miss Harris about um are we matching the need right now to identify bi bias
▶ 1:26:03I do believe that we through the AI use control process and the governments that we put in place with our partners in VHA that we do have uh commitment from all the use case owners to meet those standards in the the OM require
▶ 1:26:16And Miss Harris, what would you like to see when it comes to actually um being vigilant on making sure that we're not uh utilizing a system that is has inherent biases in it?
▶ 1:26:26Yeah, for sure. So, one thing to note um you know, Mr. Worththington talked about these high impact systems. VHA has 72% of their AI use cases as being high impact. So, meaning that they um they affect people and entities and their rights. And so that's that's a quite a number a high number of systems um that have that implica implication.
▶ 1:26:49And so yes, you have to go through additional hoops as he had mentioned with pre-eployment and during monitoring to make sure that you know rights aren't compromised. Um but there VA has told us that there is a need for more privacy officers to handle increased data security demands. So we would like to see more of those positions being filled to ensure that privacy is really taken care of as it relates to these high impact use cases.
▶ 1:27:13And I only have a few um seconds left, but I did want to ask um Dr. Cassimi, are there any cases that you've seen in public usage or private usage where they've done an excellent job in actually removing the biases, identifying them um immediately? there's a really active domain of researchers who are trying to solve exactly that problem.
▶ 1:27:36A lot of the studies are happening with uh for example the mimic database which is based out of the Boston area something that I actually contributed to. Um to summarize I think the the broader domain of that research activity in a few words is it's possible to do it but it requires a thoughtful approach and each data set is different.
▶ 1:27:59So what you have in the VA and debiasing that will be different than if you're doing it in the context of the data set in Boston and somewhere else.
▶ 1:28:07Thank you. I yield back. Thank you for the time.
▶ 1:28:09Thank you. And I'll recognize myself for five minutes again. And um Dr. Dr. GMI, I wanted to come back to you and um you've listened to some of the back and forth testimony and and some of the responses both from the VA from members here. Um you're outside of the the VA, so you you you have the the benefit of being removed from some of this internal stuff. And I'm curious, you know, kind of what your thoughts are to me. And to Mr.
▶ 1:28:35for Latrell's point is we're trying to upgrade this legacy health record system on a I guess parallel track to to use the term you used. We're trying to modernize some of the easy lift items that can be that can be done through assisted technology or or augmented I think somebody said in their tech in their testimony as well. Um, do you think that's achievable?
▶ 1:29:00Number one, and you know, how do you think that the VA can do this responsibly um to make sure that it's done in the appropriate way? So, the short answer is I think it's Um, how can it be done responsibly? It has to start first and foremost with unification of the data. I I heard in
▶ 1:29:30conversations that
▶ 1:29:32in unification of data, are you talking about having a singular system or are you talking about the the the data itself not being fragmented across all these different VA facilities? What I mean is that you need a singular way to represent the data so that an AI system that operates in one system can move and operate in another.
▶ 1:29:55Now actually the good news is that artificial intelligence can be used to help with that unification process itself. So I'll speak about some of my external experiences here and say why I think there's room to be hopeful. Um it's it's a common problem in industry for corporations to deal with. They have they have a a large database of customers or health systems have a large database of patients and they want to enrich that with some data from outside of their ecosystem.
▶ 1:30:26That's a common problem. And so there's reconciliation of two complex data sets where column names in these data sets don't match, representations of values inside these data sets don't match. There's so many things that are misaligned here.
▶ 1:30:41But the same instead of thinking of AI's role as coming in after you've done a very heavyduty and costly and inglorious task of aligning that data, you can use the AI tools to perform alignment of that data, right? to ask how you do the combination of the information, the debiasing considerations that were brought up earlier and so on.
▶ 1:31:10Thank you. I appreciate that. And um how do you think um balancing you know the access to this and the benefit that comes from that with keeping the paramount interest of you know veterans consent and you know privacy and all of those um all those things that we can't miss the mark on uh as well. Um and yeah be interested in your thoughts on that.
▶ 1:31:34Yeah I think I think disclosure is really important transparency. So, you know, when we go to a a supermarket and we turn around an item that's on the shelf, on the back is disclosed to us through the nutrition label, what are what are the contents inside of the food that we purchase.
▶ 1:31:52And a similar way, if you think of if you think of care that we receive as an item, then we need a similar way to inspect what components what which parts of the ingredients in that care came from which sources. Did they come from a a model that Oracle trained on their Cerner ecosystem? Did they come from an academic paper? Did they come from a clinician's judgment?
▶ 1:32:19So the traceability of that decision and making it transparent back to the end consumer of the care which is the veteran that's really important because they have a right to know how care is being derived prior to consenting to receive it. So I think that transparency sits at the beating heart of doing this correctly.
▶ 1:32:44And the reason there's trepidation as far as I understand it behind the use of AI not just in healthcare by the way but in a large number of industries is because the transparency is an issue right it could tell you um hallucinations right I think maybe some of you have heard of this concept if you haven't I'll quickly define it is when a model basically confidently tells you the wrong answer there are ways to overcome this They're
▶ 1:33:14they require some expertise but it is
▶ 1:33:19Thank you. Appreciate it. I am uh out of time. Uh ranking member Bazinski, I recognize you for five minutes.
▶ 1:33:26Thank you very much. Um so September is suicide prevention month and um our full committee hasn't had um a hearing for many years on suicide prevention which I think is something that is a very big missed opportunity and something I'm hoping we can be getting to.
▶ 1:33:44But I can use this opportunity at this subcommittee hearing to ask the VA some questions around suicide prevention and then the connection with AI and how AI might be a useful tool um in suicide prevention like the reach vet algorithm uh model in particular. So my question is for actually Dr. Kerry. Can you speak to how VA is planning to use its AI inventory to build on this success?
▶ 1:34:16Absolutely. Thank you so much for the question. So, as you note, it's incredibly important that we take care of our veterans, especially in this context of mental health needs. Uh, so we have been operating the reach vet mile uh model for a number of years as Mr. Worththington noted since 2017 successfully. We have updated that model recently to ensure it has ongoing high performance of identifying identification of veterans at the highest risk cortiles.
▶ 1:34:42And then we implement that model as part of a multi-pronged effect uh strategy to ensure veterans get the care they need. So their receipt of the care they need does not depend only on identification of an AI tool or being flagged as being at high risk. It's just one of many strategies we use to ensure that veterans are regularly screened. And as you noted in the opening statement, if anybody falls through the cracks that they have an opportunity to still receive the care they need.
▶ 1:35:07And one of my concerns is just we don't want to prevent human involvement from being a part of suicide prevention. We can use AI as a tool, but how does the VA look at, you know, working to ensure that human involvement isn't um eliminated as a part of the critical nature of the of the care that we want to be able to provide to a veteran uh with suicide prevention efforts?
▶ 1:35:33Thank you. That's a fantastic question and we completely agree. I want to make it absolutely clear that VA clinicians deliver care to veterans. Uh VA clinicians are in control of the care that veterans receive. So while we do use EI tools to surface risks and ensure that all veterans are flagged to get the care they need, what happens next is that a human at the VA reaches out to that veteran or first reviews the information and decides if outreach is Um, and so can you could you commit for me that the VA
▶ 1:36:03will never use AI and including chatpots as a substitute for frontline staff responders for mental health crisis intervention?
▶ 1:36:12We do not currently have any plans that I'm aware of to use AI as a treatment device instead of providers and I've personally been a part of many conversations where we ensure that continues to be the case.
▶ 1:36:22Okay. Thank you. Um, Miss Harris, could I ask what risks are posed by utilizing AI tools for use cases other than their intended purpose like the use of chat bots that were developed for programs like VR& or home loans and crisis intervention support?
▶ 1:36:40Well, I think that there would be significant risk in a tool that's not being performed as as intended. So, for example, if you're using an AI chatbot for one program um and but you know, obviously if you use that same bot for another program, it's going to produce poor results. And so, that's because the data that um was used to teach that tool uh would not be relevant to the expected role for that other program. So, we would certainly think that there's significant risk in in doing what what you've asked.
▶ 1:37:08Okay. Um, and then I just VA's Office of Inspector General reported in April that VBA's automated decision support tool was ineffective in helping claims processes assign the correct effective date for PACTA claims. This resulted in at least $7 billion in improper payments. I worry that VHA's rush to expand automation will lead to similar errors that could put patients at risk. Um, so shifting gears, Mr.
▶ 1:37:36Worthington, how do you plan to measure accuracy of implemented and piloted AI tools?
▶ 1:37:42Well, it's a great question and I think by having all of the use cases documented along with the owner of each AI use case, we'll have the consistency plans available to us so that then our colleagues in VHA can be regularly following up to see what they found because we agree that continuous monitoring of AI in production is very important. I do think our health care system is particularly well designed to monitor for those sorts of things because that's part of what they do in a non-AI context as well.
▶ 1:38:08Okay. Just a quick followup. Um at the hearing on this topic, Mr. Worththington, last year, you mentioned that the key to understanding how any particular AI may introduce biases is to understand the data that it was trained on and the outputs it provides. Considering the efforts of this administration to limit limit what kind of data may be available in research data sets or in a veterans medical file, do you believe that this will impact the efficacy of VA's AI tools?
▶ 1:38:37I would have to get into the specifics of any given case. I think at a high level it's very un important to understand what data went into the training and do the pre-eployment testing before we use something in
▶ 1:38:48Okay. Okay. I yield back.
▶ 1:38:51Thank you. Um I will now recognize Mr.
▶ 1:38:54Thank you Mr. Chairman, Mr. Gimmia fascinated but with your previous statement. Uh so clean data, dirty data, retrospective, prospective data, the transfer of information is very challenging. I'm not going to say impossible. We'll never say that. Um currently the VA doesn't house all of veterans data. It sits in the different silos of of the of the different hospitals. I think the death records lives in one spot but everyone else is dissim that that's correct. Correct.
▶ 1:39:24There's definitely siloed systems although our health data is pretty It it makes sense to me and I don't know the price tag on this if this is even possible that if all the data lived in one enclave the entire veteran space lived under say just the VA data center which I don't even know what that would look like but then the VA can control access to anybody including all the sites plus every single university and research student whomever that wants to touch it and they could prevent the ability for
▶ 1:39:54data theft. Is that a fair
▶ 1:40:00uh I do think that consolidating data into secure platforms can be a good enabler of this sort of technology for sure. Are
▶ 1:40:05we even having that discussion inside the VA
▶ 1:40:08you can say no?
▶ 1:40:10Yes, we are we are actively uh working and in fact have done a number of data consolidations to make that possible.
▶ 1:40:16I've been here for about three years now and the word actively working it doesn't really resonate in this place. are we really wanting to do this or is this just something that you're that's just something you're throwing at me?
▶ 1:40:29Uh no, I I think like an example like the reachvet model that we just described is a model that was created based on that consolidated data set uh that draws on data from all the different medical centers as well as other data into one central uh data
▶ 1:40:42so everybody can touch it. So if somebody in Conro, Texas, a VA facility that I have, says, "Hey, look, I have a veteran in here that has this." They can reach out to that data center, populate from tens of trillions of data points and send back, "Hey, most likely this is what we're looking at."
▶ 1:40:57Well, when you're using it, uh it gets complicated quickly as you know. So, uh
▶ 1:41:02different use cases have different degrees of connectedness, but in terms of building places where we can create those models that we just went through like Reachvat, we do already have investments that help with that.
▶ 1:41:12Okay. So, if we do have the willingness to do this, it's somebody's going to have to have the software in place to do it. Mr. Gatic, I'm not going to let you out of here without getting something saying something. Okay. Who can handle something like this companywise, industry, whomever? And don't say Michigan State because he's sitting in a room with me.
▶ 1:41:32No, sir. I would say someone from the University of Michigan where I went to school, they could probably take
▶ 1:41:35They're pretty good, too. Okay, sir.
▶ 1:41:39Yeah. Straight D. Um sir I uh I spent four years in the federal government. I worked in uh the general services administration under TTS and um had the opportunity to work with a lot of different agencies in that capacity. Um what I saw there was what I had seen throughout my commercial career which is that or as I as I put in my written statement um organizations have way more data than they realize and that data exists in more locations than they're aware of and that data means different things in different places at the fundamental root level in terms of where the data exists.
▶ 1:42:09And the number one reasons that projects fail, if it's an AI project or if it's any other technology project, it's because of the data. If the data isn't there, then the then no matter what position, what solution you have, it'll never really work. It's sort of like what we call putting uh lipstick on a pig, in other words. So, you have to solve that problem. Now, who solves that problem? That's an enterprisewide problem. That's an enterprisewide acknowledgement that the problem exists and an enterprisewide effort to make the investment in solving that problem. um from
▶ 1:42:38so multiple agencies are going to have to come in on top of this.
▶ 1:42:40I would say multiple departments within an agency have to be have would report up through a business leader chief data officer reporting up um at the highest level to make that investment and to solve that problem at the fundamental level because not if it's not solved fundamentally then the underlying structure of any solution will not work.
▶ 1:42:57I'm I'm going to make the assumption which I probably should that this is what's going to have to happen. Yes. I I think we need to find ways to get the exact right piece of data from everything that VA and DoD have access to to the person that needs it at the right time. And I actually think that search and summarization capability is actually one of the things that we are excited about AI maybe being able to help with.
▶ 1:43:17This is what AI will do for us.
▶ 1:43:18I think it could help with those sorts of things to sift through all those.
▶ 1:43:21I don't think the human brain can process that many data sets.
▶ 1:43:24That's right. So this is one of the areas we're actively investing.
▶ 1:43:27I shouldn't say that. The human brain can absolutely do anything. A human being cannot. it I think it gives uh an opportunity to empower people to act on more information than they would be able to do manually.
▶ 1:43:38That's something that kind of downstream I'd like to to h you know I'd like to see the p how we're laying this out um because at the end of the day as appropriators in Congress we're going to have to put a dollar sign on that and since the EHR is really giving us a great time. Can I see where I'm going with this? Thank you Mr. Chairman. I yield back.
▶ 1:43:59Yeah. Thank you Mr. El and I'll recognize myself for 5 minutes. Um Mr. uh Gatkock um you um mentioned in your testimony you kind of compared AI to the early days of the railroad, right? And you know this was a great advancement but it was fraught with all these problems and challenges and um uh you know over time was perfected and and I guess never truly perfected but certainly perfected to the degree that we can reasonably get to.
▶ 1:44:29Um I think we when it comes to artificial intelligence there's a greater risk than the occupants of a a train rolling down a railroad track. this could have catastrophic outcome if left you know unguarded or breach of information or you know who knows what it could be truly problematic what are the guardrails that you think are appropriate and necessary right now to make sure that that doesn't happen with AI like how are we going to look over the horizon at what could happen and then prevent it from happening on
▶ 1:45:00the front end
▶ 1:45:01thank you um it's it's a great question and I think it's a very uh it's sort of a fundamental question in terms of uh of AI and what it is and what it is not as I said in my statement it when you interact with AI tools today it feels like you're talking to a human being but it is not a human being it has no moral conscience it doesn't really understand the words that's actually being given to it or the words that are it's producing so there are a number of ways to to really address this issue one of those um is really understanding the difference between correlation and
▶ 1:45:31causation without getting into great statistical detail um there's nearly a perfect correlation um as I uh put in my testimony in terms of the number of Google searches for the word Nintendo and the number of librarians in the state of Michigan. Most statistical models will rely on this relation this this using correlation to identify patterns then reproduce those patterns in its output.
▶ 1:45:55What is really needed is an emphasis on causation understanding the inputs that a model uses how those inputs relate to each other and how those relate to the outputs. there is very little effort being placed on that type of technology and that type of investment because the dollars are already chasing correlation. Correlation is a lot easier to do than causation. Um so that's where a lot of the investment goes.
▶ 1:46:16So I would say one of the fundamental areas is and I don't know if it can be mandated but I would think I would hope that the scientific and research community would realize that's the power um of AI is to to unlock the true potential of it is to really mimic how a human mind works which is it sees something it reacts to it and then produces something else. So to mimic that with other technology would be great. The other thing that I did want to want to say is that is going back to the data itself.
▶ 1:46:43um a model is trained uh I think the earlier question was around bias right the the data that the model is given if it's not inherently debiased if it's not if it if a lot of thought isn't given to the data itself that the model is model receives then the output will be inherently biased it could be biased because of the way it's engineered it could be bei biased because of the data that it's given but because these models are so complex and so little work has been done to understand how they work they'll never know if it's the model that's biased
▶ 1:47:13or the data that was biased. So again, a principle, a development principle, a research principle, a standardization that's uh adopted by industry to address all of those would be would be very
▶ 1:47:23Yeah, thank you. And that correlation causation thing is really important. I I would I would bet or guess that a lot of information at the beginning is correlation information. Then over time maybe it can be perfected or improved into the causitive and non-causitive you know parts of that.
▶ 1:47:44But at the beginning it's if this then that correlation we may not know why or how but these things especially when you're deal with medical information over a long period of time and you know if if enough people come in with a correlating condition enough times we begin to believe it's positive for a risk factor for something else. Um, so I guess how do we like how do we make good decisions based on that?
▶ 1:48:12Um, you know, because we may not even understand the causitive nature of it, but if it's enough correlation data there, maybe it it does tell us
▶ 1:48:21Absolutely. I think correlation has a has a purpose and in terms of identifying patterns and identifying things that are outside of the norm. Absolutely. It's it's a wonderful tool and it's it's a critical tool. Uh my position would be that it just can't be used in a vacuum. That coupled with caus understanding causation and investing more in those types of tools to help understand the true relation between two things and why one is causing the other. Um as I mentioned there's no obviously relationship between Google searches and the number of librarians.
▶ 1:48:50But the problem is correlation models don't know that they just use that number and run with it. So um there are lots of crazy examples that I could give but that's a that's a good one and relevant. But the that emphasis on causation I think is really one that is not been invested in as much as it should be. Um it's something that we found is really helpful and powerful that helps us understand our models and why they behave that way. Also when they fail we understand why they fail. Um it's likely because something broke in that relationship or did not work in that
▶ 1:49:20Thank you. I am uh out of time. I'm going to yield to Ranking Member Bazinski for five minutes.
▶ 1:49:24Thank you. Thank you, Mr. Chairman. Um I wanted to ask Miss Harris um some follow-up questions. Um, just as ranking member, I have spent now a lot of time um, asking the VA how it plans to juggle all of these different numerous modernization efforts the department is pursuing like EHRM, of course, supply chain, HR modernization, and now AI.
▶ 1:49:46Um, I was wondering if you could speak to the types of resources that the VA will need um, considering to to consider having at its disposal as it deploys these systems.
▶ 1:49:57Uh, yes, thank you for the question. I mean first and foremost I think it's hugely problematic that VA does not have a permanent CIO in place. I know you mentioned it in your opening statement. Um that's because under his or her leadership that's where these you know various IT modernizations get prioritized you know um and plus our work has shown that you know when you have that steady leadership over you know three to four year time period that's essential for any successful major IT initiative including all the AI initiatives zero trust
▶ 1:50:27EHRM all those things the the second point um OIT is obviously going through a major restructuring right now you know they they've requested almost $300 million less um in fiscal year 26 than the previous year. They've also reduced staff by 931 staff. Um now more than ever VA needs to fully understand um have a comprehensive grasp on the the skills and inventories that they have in their IT workforce.
▶ 1:50:56And at this time they don't know that. So they're not in a position to know effectively assess what they need if they don't know what they have. and that's an open recommendation that we have. So that's first and foremost something that they need to do in order to answer your
▶ 1:51:10Okay. Thank you. Um Mr. Worththington, in GAO's review from July, VA noted that it faced challenges with implementing generative AI use cases due to a lack of sufficient technical resources and budget. Your testimony highlights this issue of cost as well. um as it is currently funded and staffed, do you believe the VA is capable of implementing additional AI use cases on top of these other modernization efforts that I've mentioned?
▶ 1:51:41Thank you for the question, Ricky member, and I I do think that we have the resources to implement high impact AI, but it is a tough environment. Everything is competing for resources with each other. Um and so it's a matter of prioritizing those things that are going to have the most amount of veteran impact with the resources that we have. Okay. I guess I just go back to what Miss Harris's recommendation, getting a CIO, I think is really critical to helping to prioritize all of these different really important initiatives. Um, and Mr.
▶ 1:52:08Worthington, do you believe the VA's challenges with reta challenges with retaining AI experts and other technical employees may impact VA's ability to scale AI tools and other modernization efforts? I definitely think having AI experts on the VA side uh will help make us a better purchaser of these solutions and it's a it's an important thing for us to do.
▶ 1:52:30We we've invested a lot in trying to build this team especially through partnerships with things like the uh United States Digital CPS and the Presidential Innovation Fellows program. We want to lean into those sorts of partnerships to help us bring AI experts in in addition to those that we can recruit ourselves.
▶ 1:52:46Okay, great. Um Mr. Worthington, we're hearing reports that VA's ambient listening pilot will be rolled out across 10 facilities by the end of this year. Um, what is the department determining um the as success for this
▶ 1:53:02Thank you for the question. I'll have Dr. Kerry uh give you some details on
▶ 1:53:08thank you for the question. Um, we have established a series of criteria and evaluation as we roll this out that's focused on user acceptance testing, veterans perceptions of the tool as it's used and their ongoing trust in the care they receive and just overall performance of the tool. We'll continue to monitor that during the pilot.
▶ 1:53:27Okay. And are you measuring clinician burden and and what are your targets?
▶ 1:53:33We are um I can take that for the record to get back to you with the specifics. In general, we are measuring clinician burden and getting clinician feedback both synchronously and through survey mechanisms to understand the impacts.
▶ 1:53:44One thing I would love to add is the users of our generative AI tool that's deployed to the workforce as a whole. In a survey, 73% of the users of that tool reported that they were able to spend more time fully using their professional skill skills and 68% reported increased job satisfaction. So, I do think that these tools are going to be uh value ads to our workforce to help them do more to serve veterans.
▶ 1:54:07Um, well, it seems to me that we are placing a massive burden on providers. That's a concern from being an ambassador to the tool for veterans, ensuring the tools accuracy and then reporting and mediating issues as they arise. Um, how is the department working to pre to be proactive about receiving feedback from providers on issues with this tool? Thank you. That's a great question.
▶ 1:54:34And just briefly want to recognize it's so important to balance that survey response burden and burden on the clinicians that are also providing care. We've been partnering with clinicians on day one designing this as they are the end users. Uh and so we just have ongoing conversations with them about the best way to balance those competing
▶ 1:54:51Okay. Thank you. I yield back.
▶ 1:54:53Thank you. Um I will um we're going to close here momentarily. I just have one uh quick question on that um listening and automation uh transcribing. Is that file of that recording is that deleted after it is transcribed? Is there some protection there to make sure that it is not archived or or held
▶ 1:55:16We do have procedures on that and be happy to get that back to you for the record. I don't have the details in front of me, but yes, we've got that accounted for.
▶ 1:55:22Thank you. Uh I will now yield to Rank Member Bazinski for her closing
▶ 1:55:26Okay. Thank you. I just want to thank the panelists for being here today to have this conversation. I do very much appreciate it. I do want to go back though, Mr. Worthington, to a conversation we had earlier about um the six VA employees that had been working with Doge and a letter that ranking member Takano had written to the VA back in June. We haven't gotten a response. We just want some more transparency around access to the data that those six employees had. Um that's veterans data.
▶ 1:55:55just want transparency and some additional information on that. So, anything you can do to help us get a response back for ranking member Takano would be very appreciated. Thank you.
▶ 1:56:09Thank you, uh, Ranking Member Bazinski. Appreciate it. Um, and I want to thank our panelists and the members today for, uh, joining us for this, uh, important hearing. Um, this hearing has made clear that VA has both made a tremendous uh, we have both a tremendous opportunity as well as a serious responsibility when it comes to using artificial intelligence within the VA. VA has access to some of the best data and research assets in the world. And I know Mr. Latrell pointed that out some of his questioning too.
▶ 1:56:36If used the right way, AI could help doctors detect cancer earlier, prevent heart disease, cut down on paperwork, and most importantly, save veterans lives, and hopefully prevent veteran suicides in the process. Programs like Reachvet show us it's possible when technology is focused on the mission, and we can improve outcomes. Let's be clear, AI is a tool, not a replacement for doctors, nurses, and care teams.
▶ 1:57:00And I appreciate the the VA stipulating that we're not trying to replace practitioners with AI tools. Can help identify risks earlier and provide clinical uh clinical pathways, but it cannot and must not replace uh treatment or human judgment. There's a reason we send doctors to college, right? Because we want them to be experts on what they're doing. Veterans deserve both cutting edge technology and a strong medical team working together on their behalf.
▶ 1:57:27That means vigilance and self- responsibility are still and a sense of responsibility are still required. If VA fails to safeguard veterans data or or to maintain trans uh transparency, trust will be lost and progress is going to This subcommittee will continue to hold the VA accountable to ensure that AI enhances care, reduces red tape, and strengthens not substitutes the human touch needed in medicine.
▶ 1:57:55I ask unanimous consent that all members have five legislative days to revise and extend the remarks to include extraneous material. Without objection, that's awarded and this hearing is adjourned. Thanks a lot. All right, I got to run. I think