This week, Democratic Senator Adam Schiff of California unveiled the Human Authority in Lethal Operations (HALO) Act, a bill that would require a human commander to make the final decision on any actions taken by autonomous weapons systems. The bill would also mandate detailed records of how military decisions were made and how targets were selected for later review, establish protections for whistleblowers, and prohibit the use of AI in certain cases involving nuclear weapons and mass surveillance.
“The past few months have shown us that there is an urgent need for common sense guardrails to ensure the Department of Defense’s use of AI is consistent with Americans’ national security and privacy priorities,” Senator Schiff said in the press release. “My legislation will protect Americans from unlawful domestic surveillance, ensure that humans in the chain of command will hold accountability for the use of any lethal technology, and maintain strong ethical protections in the deployment of autonomous and semi-autonomous weapons.”
Artificial intelligence has been a part of warfare for some time now. Armies around the world take the help of AI systems for target selection in attacks and mass surveillance, a prime example of this is the use of AI by the Israeli army against Palestinians. The United States has also long deployed artificial intelligence in military operations, including its latest war against Iran.
But earlier this year, the use of AI in the military rose to the top of the public discussion when an existing deal between the Pentagon and Anthropic fell apart, and in an unprecedented move, the AI giant was designated as a supply chain risk. Anthropic reportedly refused to get rid of guardrails in its AI systems, intended to prevent the DOD from using its models for mass domestic surveillance and fully autonomous weapons, meaning there was no human involvement.
Following the outcome of the deal, the Pentagon signed contracts with all the other major AI companies, including OpenAI, Google, Nvidia, SpaceX, Microsoft, and Amazon Web Services. Meanwhile, Anthropic has challenged the designation in court, although reports say the Trump administration is warming to the company following the release of Mythos, with its latest model touted as a nightmare for the cybersecurity industry.
Following the Trump administration’s public parting with Anthropic, many Democrats came out in support of the AI company and its stance. That list included Sen. Schiff, author of the HALO Act.
“I wish we had more voices like Anthropic,” Senator Schiff said at a Punchbowl news conference in March.
Schiff has introduced a series of AI-related bills over the past few months, including requiring large data centers to pay for their electricity, ordering AI companies to disclose copyrighted work used to train models, and proposals to bring AI literacy classes to schools. Now, he is reportedly aiming to tag this bill with the annual military spending package, aka NDAA, which should pass by the end of the year.
He’s not the only Democrat with such a plan. New York Senator Kirsten Gillibrand introduced a similar bill earlier this month, banning the use of AI in nuclear weapons deployment, domestic surveillance and fully autonomous weapons. All of these and more “high-consequence actions,” as the bill defines them, would require approval from a high-ranking DoD official to proceed. Senator Gillibrand also reportedly plans to introduce the proposal as an amendment to the NDAA.
Then, there’s the AI Guardrails Act, introduced by Sen. Elissa Slotkin of Michigan in March. Aiming for the same guardrails as Senators Schiff and Gillibrand’s bills, Senator Slotkin is reportedly preparing to introduce it as an amendment to the NDAA as well.
Although all three of these bills aim to ensure security by bringing human oversight to any decisions taken by AI systems in military settings, the dangers do not end there.
Many AI users suffer from what experts call automation bias, that is, the belief that an AI system can make more accurate decisions than you because it has access to more information or perhaps reasons in a more efficient way. This, obviously, is not true: the technology is far from perfect, and LLMs are prone to hallucinations or biased thinking. Combine this with the “black box” nature of AI, where users don’t have full knowledge of how or why the system reasons, and you have a military AI plan that could lead to potentially fatal mistakes even with human oversight.
<a href