Caution: AI Ahead

Caution: AI Ahead

In addition to their own research explorations of the uses and misuses of artificial intelligence strategies, CCS faculty and staff often serve as informed commentators on issues that continue to emerge with the deployment of these technologies. Between late 2025 and the first half of 2026, CCS researchers were asked to weigh in on a number of AI-related initiatives, including the technology’s potential to insert back doors in computer chips, and threaten financial systems, Lastly, we highlight an editorial in IEEE Spectrum that points out how the drastic reduction in the cost of hacking a target makes it critical to consider employing more resilient defenses.

Brooklyn Researchers Warn AI Can be Weaponized to Plant Flaws in Computer Chips”

The Brooklyn Daily Eagle, October 26, 2025

AI tools can simplify many tasks, including the introduction of vulnerabilities into computer chips. This story reports on a study, led by NYU Tandon Ph.D. candidate Jason Blocklove, that found AI systems like ChatGPT can create undetectable “hardware Trojans,” or hidden flaws in computer chips capable of leaking data or granting attackers remote access. “AI tools definitely simplify the process of adding these vulnerabilities,” Blocklove explained to staff writers of The Brooklyn Eagle, a daily newspaper from Brooklyn, NY. Professor Ramesh Karri of NYU Tandon’s Electrical and Computer Engineering Department, who co-authored the paper with Blocklove and Dr. Hammond Pearce, Senior Lecturer at the University of New South Wales, also told The Daily Eagle that “once manufactured, hardware flaws can not be patched,” adding “if such an attack did happen, the consequences would be catastrophic.” 

The results of the study can be found in the paper “Lowering the Bar: How Large Language Models Can be Used as a Copilot by Hardware Hackers,” which was published in IEEE Security and Privacy. 

“ICE’s Use of AI Will Lead to Big Mistakes. Maybe That’s the Point”
Rolling Stone Magazine, January 31, 2026

Though the U.S. Customs and Enforcement Service, better known as ICE, has come under fire for many of their tactics, an article that appeared in Rolling Stone on January 31, 2026, suggests its pattern of wrongful arrests may be due in part to its injudicious use of a number of AI-powered surveillance and data analysis tools. Professor Damon McCoy, who is co-director of the NYU Center for Cybersecurity and a Professor of Computer Science and Engineering at NYU Tandon, points to a number of concerns with this strategy, starting with privacy breaches against individuals who are unaware of their exposure to AI-powered surveillance. “There’s a pretty rich stream of location data that’s coming off of your phone,” he told the article’s author, Miles Klee, noting that even things as “presumably innocuous as your flashlight app are collecting fine-grained GPS data on you.” In addition, McCoy notes that ICE has also begun to explore acquisitions of data from other sources that could offer potential insight into a person’s health or financial history. “That data is very powerful. There’s a lot of location data, there’s a lot of browser history data, there’s a lot of purchasing data. It’s a very rich, powerful data source that’s been built out. Instead of being used to figure out what pair of sneakers you might buy, it might be used to figure out what protest you attended.”

Perhaps most concerning, the article’s author, Klee worries that “open availability of this type of information poses a threat not only to migrants and activists but anyone with a digital footprint, because depending on AI to sort through it will inevitably lead to mistakes.” McCoy concurred, “If they’re not vetting things well, and they’re just heavily relying on the AI, there’s more than likely a lot of issues that they’re going to encounter.” 

“The New AI Model that Could Steal your Life Savings”
The Indicator from Planet Money, NPR, May 11, 2026

This 10-minute audio post looks at Claude Mythos, an AI model from Anthropic powerful enough that its creators deliberately limited its initial release to just a few institutions because there was concern over its potential to open new vulnerabilities. Among the experts asked to comment in program was CCS faculty member Professor Rachel Greenstadt of NYU Tandon’s Computer Science and Engineering Department. She noted that tools like Mythos in the hands of less experienced developers do pose a security challenge. “People are able to write much more complex things, much quicker, that they don’t understand, that AI doesn’t understand and this is an opportunity for a whole new class of bugs,” she said. However, she tempered her remarks by adding, “the vulnerabilities they are finding are real. But, we’ve seen these things before. It’s neither the case that the angels have appeared, nor the case that the sky is falling.”

With $1 Cyberattacks on the Rise, Durable Defenses Pay Off
IEEE Spectrum, April 30, 2026

“Writing memory-safe code beats patching your way to safety,” is the subtitle for this editorial, which was cowritten by Professor Justin Cappos of NYU Tandon’s Computer Science and Engineering Department, and the department’s newest faculty member Assistant Professor Evan Johnson, The two researchers acknowledge AI’s growing capacity to effectively identify and patch vulnerabilities, but cautions, “It is not yet clear whether AI-driven bug finding will ultimately favor attackers or defenders.” With this in mind, after discussing why AI guardrails and automated patching may not be a complete solution, the authors ask readers to consider “more robust defenses,” including a layered prevention strategy that employs, where possible, memory-safe languages, the use of software sandboxing techniques to contain “the blast radius of vulnerabilities that do exist,” and the adoption of formal verification which “proves, mathematically, that certain bugs cannot exist.”

The authors conclude by saying the latest wave of smarter AI bug scanners “can still be useful for cyberdefense—not just as another overhyped AI threat. But AI bug scanners treat the symptom, not the cause. The lasting solution is software that doesn’t produce vulnerabilities in the first place.”