Cybersecurity in the Age of AI

 /  Security, Ai-Agents
Cybersecurity in the Age of AI
Contents

Last year, a finance professional joined a video call with colleagues they believed they knew. The meeting felt ordinary. The instructions to move money felt plausible. It was not until later that the organisation discovered something chilling: the voices and faces on the call were not human in the way everyone assumed. This is not science fiction. It is the kind of scenario regulators and the World Economic Forum have discussed in connection with the Arup case, where reporting describes a deepfake-enabled fraud on the order of tens of millions of dollars. The details matter less for day-to-day work than the lesson: trust signals we relied on for decades—voice, face, urgency—are now forgeable at scale.

That is the emotional core of cybersecurity in the AI era. The technology that helps people write, code, and analyse faster is the same technology helping attackers craft more convincing lures, automate busywork across the attack lifecycle, and probe new paths into our systems. The goal of this article is not fear-mongering. It is clarity: what is changing, what defenders are doing about it, and what you can do this quarter—as an individual, a team lead, or an organisation.

How AI is reshaping the attack surface

Phishing and social engineering have always been about believable stories. Large language models lower the cost of drafting polished, context-aware messages. Some widely circulated percentage claims about “AI phishing” come from non-primary blogs; treat those headlines skeptically. What security teams consistently observe is more qualitative: more variation, faster iteration, and better localisation of malicious content. In practice, that means your training scenarios need to evolve beyond obvious grammar mistakes.

Deepfakes and voice cloning turn a second problem into a first-order risk for finance, HR, and executives. When a short audio clip can seed a plausible voice imitation, phone-based verification and “I heard the boss approve it” workflows need stronger out-of-band checks—not drama, just process.

Malware is not the whole story. Industry reporting—including CrowdStrike’s global threat reporting—has repeatedly emphasised that many modern intrusions lean on valid credentials and living-off-the-land techniques rather than dropping traditional malicious binaries. The exact percentages shift with definitions and report editions, so the actionable takeaway is simpler: credential hygiene, session security, and identity governance are not “nice to haves”; they are the main line.

Enterprise copilots and AI assistants introduce a different class of risk: indirect prompt injection. A well-publicised line of research and disclosure—often discussed alongside CVE-2025-32711 and the EchoLeak narrative—shows how content such as email or documents can carry hidden instructions that an automated assistant may follow, potentially exfiltrating data through trusted channels. The lesson for organisations is not “turn off AI,” but architect with least privilege, monitor autonomous tool use, and patch assistants and platforms on the same urgency curve as core infrastructure.

Finally, agentic AI—systems that plan, call tools, and act—expands the identity surface. Gartner and major vendors have warned that machine identities tied to agents can outpace governance if security teams treat them like exotic edge cases. They are rapidly becoming central to how work gets done.

How defenders are fighting back

Security operations centres are adopting AI for triage, summarisation, correlation, and reporting. That does not replace analysts; it removes toil so experienced people spend time on judgement calls. Surveys and vendor roadmaps differ on maturity, but the direction is consistent: humans plus automation, with careful rollout and measurement.

On the detection side, machine learning has long helped find anomalous behaviour that static rules miss. The newer pitch—sometimes framed as “agentic defence”—is that response workflows can be accelerated when trusted automation sits alongside human oversight. When you read vendor statements from companies such as Google Cloud at events like RSAC, treat them as strategic positioning worth evaluating against your own risk model, not as universal truth.

The honest through-line is speed. Attackers automate; defenders must compress detection and response time without abandoning accountability. That is less about buzzwords and more about playbooks, logging, identity, and backups finally getting funded because the cost of delay is visible.

What the data is really telling us

Large annual datasets such as the Verizon Data Breach Investigations Report remain useful for patterns: ransomware’s persistence, the role of partners and software supply chains, and the continued importance of basics like patching and phishing-resistant authentication. Any single statistic in a blog post should be checked against the original report wording—especially when a percentage could mean growth, share, or incidence depending on context.

Independent catalogues such as the AI Incident Database illustrate breadth: reporting in 2025 pointed to on the order of three hundred documented AI-related incidents, spanning deepfakes, fraud, and unsafe outcomes. That number is method-bound—real-world harm almost certainly exceeds what gets catalogued—but it is still a sobering map of failure modes and a useful prompt for red-teaming your own products and processes.

For breach costs, long-running industry studies (historically associated with sources such as IBM’s Cost of a Data Breach research) are the safer anchor when you need a single “average cost” talking point—again, with the caveat that your risk depends on sector, region, and controls.

Practical advice: individuals

  1. Treat voice and video approvals as untrusted by default for anything financial. Use a second channel you initiate yourself.
  2. Use phishing-resistant MFA where available (passkeys, FIDO2) rather than SMS-only, especially for email and identity providers.
  3. Be slower, not paranoid, when someone pressures you to bypass procedure “just this once.” That pattern predates AI; AI just makes the story smoother.

Practical advice: organisations

  1. Identity first: MFA everywhere, least privilege, lifecycle management for human and service accounts—and a plan for agent identities before you have dozens of them.

  2. Secure AI like software: data classification, prompt-injection testing for internal copilots, logging of tool calls, and patch cadence tied to disclosed vulnerabilities (including assistant-related CVEs).

  3. Supply-chain honesty: Major outages at healthcare clearinghouses, automotive SaaS platforms, and logistics providers remind us that concentration risk can dwarf any one malware sample. Redundancy, segmentation, and exit plans matter as much as AI headlines.

  4. Exercise the new scenarios: tabletop a deepfake CFO call, a poisoned document that manipulates a copilot, and a stolen refresh token—then fix what breaks.

  5. Measure what you ship: if you deploy an internal assistant, define allowed data scopes, retention, and audit expectations up front. The fastest way to lose trust is an impressive demo with invisible data flows.

Closing

AI did not invent cyber risk. It scaled skills that used to sit with specialists, and it compressed the time from idea to exploit. The organisations that adapt will combine skeptical optimism: adopt assistants where they help, govern them like production systems, and keep humans in the loop where stakes are high.

Security in the AI era is still, mostly, security: identity, patching, logging, backups, and a culture that rewards reporting over heroics. The difference is that the stories attackers tell—and the speed at which they iterate—just got more convincing. Our processes need to be better, not louder.