Sunday , July 20 2025

AI for security. Now we need security for AI

After the release of ChatGPT, artificial intelligence (AI), machine learning (ML) and large language models (LLMs) have become the number one topic of discussion for cybersecurity practitioners, vendors and investors alike. This is no surprise; as Marc Andreessen noted a decade ago, software is eating the world, and AI is starting to eat software.

Despite all the attention AI received in the industry, the vast majority of the discussions have been focused on how advances in AI are going to impact defensive and offensive security capabilities. What is not being discussed as much is how we secure the AI workloads themselves.

HPE alerts of hardcoded passwords in Aruba access points

Hewlett-Packard Enterprise (HPE) warns that Aruba Instant On Access Points have hardcoded credentials, enabling attackers to skip normal authentication and...
Read More
HPE alerts of hardcoded passwords in Aruba access points

Akira Ransomware Allegedly Compromise 12 Companies in 72 Hours

The Akira ransomware group increased its attacks, adding 12 new victims to its dark web portal from July 15 to...
Read More
Akira Ransomware Allegedly Compromise 12 Companies in 72 Hours

Singapore urgently engage military force to tackle ‘serious’ cyberattack

Defence Minister Chan Chun Sing said these select units will work with the Cyber Security Agency (CSA) in a united...
Read More
Singapore urgently engage military force to tackle ‘serious’ cyberattack

Hackers infect 10M Androids with BADBOX 2.0

Google is suing 25 unidentified cybercriminals thought to be from China for running BADBOX 2.0, a major global botnet with...
Read More
Hackers infect 10M Androids with BADBOX 2.0

Oracle Patched 200 Vulns With July 2025 CPU

Oracle's July 2025 Critical Patch Update includes 309 new security patches, with 127 addressing remotely exploitable vulnerabilities. SecurityWeek found about...
Read More
Oracle Patched 200 Vulns With July 2025 CPU

Ivanti Zero-Days Exploited to Drop MDifyLoader

Cybersecurity researchers have revealed a new malware named MDifyLoader, linked to cyber attacks using security vulnerabilities in Ivanti Connect Secure...
Read More
Ivanti Zero-Days Exploited to Drop MDifyLoader

CISA added Fortinet FortiWeb vul to KEV catalog

U.S. Cybersecurity and Infrastructure Security Agency (CISA) has added a crucial vulnerability in Fortinet FortiWeb in its Known Exploited Vulnerabilities...
Read More
CISA added Fortinet FortiWeb vul  to KEV catalog

Adoption Agency Exposes One Million+ Records

Security researcher Jeremiah Fowler discovered an online database exposing sensitive information from an adoption agency. Jeremiah Fowler Jeremiah specializes in...
Read More
Adoption Agency Exposes One Million+ Records

CVE-2025-20337
Patch Now! Cisco ISE bug allows pre-auth command execution

A critical vulnerability in Cisco Identity Services Engine (ISE) and Cisco ISE-PIC, identified as CVE-2025-20337, has a CVSS score of...
Read More
CVE-2025-20337  Patch Now! Cisco ISE bug allows pre-auth command execution

BD Bank Honours PABC Officials for Foiling $20 Million Cyber Fraud Attempt

On Tuesday, Bangladesh Bank organized a special award ceremony at its headquarters in Dhaka to formally recognize and honor a...
Read More
BD Bank Honours PABC Officials for Foiling $20 Million Cyber Fraud Attempt

Over the past several months, we have seen many cybersecurity vendors launch products powered by AI, such as Microsoft Security Copilot, infuse ChatGPT into existing offerings or even change the positioning altogether, such as how ShiftLeft became Qwiet AI. I anticipate that we will continue to see a flood of press releases from tens and even hundreds of security vendors launching new AI products. It is obvious that AI for security is here.

A brief look at attack vectors of AI systems

Securing AI and ML systems is difficult, as they have two types of vulnerabilities: Those that are common in other kinds of software applications and those unique to AI/ML.

First, let’s get the obvious out of the way: The code that powers AI and ML is as likely to have vulnerabilities as code that runs any other software. For several decades, we have seen that attackers are perfectly capable of finding and exploiting the gaps in code to achieve their goals. This brings up a broad topic of code security, which encapsulates all the discussions about software security testing, shift left, supply chain security and the like.

Because AI and ML systems are designed to produce outputs after ingesting and analyzing large amounts of data, several unique challenges in securing them are not seen in other types of systems. MIT Sloan summarized these challenges by organizing relevant vulnerabilities across five categories: data risks, software risks, communications risks, human factor risks and system risks.

Some of the risks worth highlighting include:

  • Data poisoning and manipulation attacks. Data poisoning happens when attackers tamper with raw data used by the AI/ML model. One of the most critical issues with data manipulation is that AI/ML models cannot be easily changed once erroneous inputs have been identified.
  • Model disclosure attacks happen when an attacker provides carefully designed inputs and observes the resulting outputs the algorithm produces.
  • Stealing models after they have been trained. Doing this can enable attackers to obtain sensitive data that was used for training the model, use the model itself for financial gain, or to impact its decisions. For example, if a bad actor knows what factors are considered when something is flagged as malicious behavior, they can find a way to avoid these markers and circumvent a security tool that uses the model.
  • Model poisoning attacks. Tampering with the underlying algorithms can make it possible for attackers to impact the decisions of the algorithm.

In a world where decisions are made and executed in real time, the impact of attacks on the algorithm can lead to catastrophic consequences. A case in point is the story of Knight Capital which lost $460 million in 45 minutes due to a bug in the company’s high-frequency trading algorithm. The firm was put on the verge of bankruptcy and ended up getting acquired by its rival shortly thereafter. Although in this specific case, the issue was not related to any adversarial behaviors, it is a great illustration of the potential impact an error in an algorithm may have.

AI security landscape

There is a lot about the problem of AI security online, although it looks significantly less compared to the topic of using AI for cyber defense and offense. Many might argue that AI security can be tackled by getting people and tools from several disciplines including data, software and cloud security to work together, but there is a strong case to be made for a distinct specialization.

When it comes to the vendor landscape, I would categorize AI/ML security as an emerging field. The summary that follows provides a brief overview of vendors in this space. Note that:

  • The chart only includes vendors in AI/ML model security. It does not include other critical players in fields that contribute to the security of AI such as encryption, data or cloud security.
  • The chart plots companies across two axes: capital raised and LinkedIn followers. It is understood that LinkedIn followers are not the best metric to compare against, but any other metric isn’t ideal either.

Although there are most definitely more founders tackling this problem in stealth mode, it is also apparent that AI/ML model security space is far from saturation. As these innovative technologies gain widespread adoption, we will inevitably see attacks and, with that, a growing number of entrepreneurs looking to tackle this hard-to-solve challenge.

Closing notes

In a world powered by AI, any unexpected behavior of the algorithm compromised of the underlying data or the systems on which they run will have real-life consequences. The real-world impact of compromised AI systems can be catastrophic: misdiagnosed illnesses leading to medical decisions which cannot be undone, crashes of financial markets and car accidents, to name a few.

Although many of us have great imaginations, we cannot yet fully comprehend the whole range of ways in which we can be affected. As of today, it does not appear possible to find any news about AI/ML hacks; it may be because there aren’t any, or more likely because they have not yet been detected. That will change soon.

Today, we are in a very different place. Although there is not enough security talent, there is a solid understanding that security is critical and a decent idea of what the fundamentals of security look like. That, combined with the fact that many of the brightest industry innovators are working to secure AI, gives us a chance to not repeat the mistakes of the past and build this new technology on a solid and secure foundation.

Will we use this chance? Only time will tell. For now, I am curious about what new types of security problems AI and ML will bring and what new types of solutions will emerge in the industry as a result.

Ross Haleliuk is a cybersecurity product leader, head of product at LimaCharlie and author of Venture in Security.

Check Also

FortiWeb

CVE-2025-25257
Fortinet Addresses Major SQL Injection Flaw in FortiWeb

Fortinet has issued a critical patch for a critical vulnerability in its FortiWeb product, a …

Leave a Reply

Your email address will not be published. Required fields are marked *