Blog
/
Email
/
September 30, 2024

Business Email Compromise (BEC) in the Age of AI

Generative AI tools have increased the risk of BEC, and traditional cybersecurity defenses struggle to stay ahead of the growing speed, scale, and sophistication of attacks. Only multilayered, defense-in-depth strategies can counter the AI-powered BEC threat.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Carlos Gray
Senior Product Marketing Manager, Email
Default blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog image
30
Sep 2024

As people continue to be the weak link in most organizations’ cybersecurity practices, the growing use of generative AI tools in cyber-attacks makes email, their primary communications channel, a more compelling target than ever. The risk associated with Business Email Compromise (BEC) in particular continues to rise as generative AI tools equip attackers to build and launch social engineering and phishing campaigns with greater speed, scale, and sophistication.

What is BEC?

BEC is defined in different ways, but generally refers to cyber-attacks in which attackers abuse email — and users’ trust — to trick employees into transferring funds or divulging sensitive company data.

Unlike generic phishing emails, most BEC attacks do not rely on “spray and pray” dissemination or on users’ clicking bogus links or downloading malicious attachments. Instead, modern BEC campaigns use a technique called “pretexting.”

What is pretexting?

Pretexting is a more specific form of phishing that describes an urgent but false situation — the pretext — that requires the transfer of funds or revelation of confidential data.  

This type of attack, and therefore BEC, is dominating the email threat landscape. As reported in Verizon’s 2024 Data Breach Investigation Report, recently there has been a “clear overtaking of pretexting as a more likely social action than phishing.” The data shows pretexting, “continues to be the leading cause of cybersecurity incidents (accounting for 73% of breaches)” and one of “the most successful ways of monetizing a breach.”

Pretexting and BEC work so well because they exploit humans’ natural inclination to trust the people and companies they know. AI compounds the risk by making it easier for attackers to mimic known entities and harder for security tools and teams – let alone unsuspecting recipients of routine emails – to tell the difference.

BEC attacks now incorporate AI

With the growing use of AI by threat actors, trends point to BEC gaining momentum as a threat vector and becoming harder to detect. By adding ingenuity, machine speed, and scale, generative AI tools like OpenAI’s ChatGPT give threat actors the ability to create more personalized, targeted, and convincing emails at scale.

In 2023, Darktrace researchers observed a 135% rise in ‘novel social engineering attacks’ across Darktrace / EMAIL customers, corresponding with the widespread adoption of ChatGPT.

Large Language Models (LLMs) like ChatGPT can draft believable messages that feel like emails that target recipients expect to receive. For example, generative AI tools can be used to send fake invoices from vendors known to be involved with well-publicized construction projects. These messages also prove harder to detect as AI automatically:

  • Avoids misspellings and grammatical errors
  • Creates multiple variations of email text  
  • Translates messages that read well in multiple languages
  • And accomplishes additional, more targeted tactics

AI creates a force multiplier that allows primitive mass-mail campaigns to evolve into sophisticated automated attacks. Instead of spending weeks studying the target to craft an effective email, cybercriminals might only spend an hour or two and achieve a better result.  

Challenges of detecting AI-powered BEC attacks

Rules-based detections miss unknown attacks

One major challenge comes from the fact that rules based on known attacks have no basis to deny new threats. While native email security tools defend against known attacks, many modern BEC attacks use entirely novel language and can omit payloads altogether. Instead, they rely on pure social engineering or bide their time until security tools recognize the new sender as a legitimate contact.  

Most defensive AI can’t keep pace with attacker innovation

Security tools might focus on the meaning of an email’s text in trying to recognize a BEC attack, but defenders still end up in a rules and signature rat race. Some newer Integrated Cloud Email Security (ICES) vendors attempt to use AI defensively to improve the flawed approach of only looking for exact matches. Employing data augmentation to identify similar-looking emails helps to a point but not enough to outpace novel attacks built with generative AI.

What tools can stop BEC?

A modern defense-in-depth strategy must use AI to counter the impact of AI in the hands of attackers. As found in our 2024 State of AI Cybersecurity Report, 96% of survey participants believe AI-driven security solutions are a must have for countering AI-powered threats.

However, not all AI tools are the same. Since BEC attacks continue to change, defensive AI-powered tools should focus less on learning what attacks look like, and more on learning normal behavior for the business. By understanding expected behavior on the company’s side, the security solution will be able to recognize anomalous and therefore suspicious activity, regardless of the word choice or payload type.  

To combat the speed and scale of new attacks, an AI-led BEC defense should spot novel threats.

Darktrace / EMAIL™ can do that.  

Self-Learning AI builds profiles for every email user, including their relationships, tone and sentiment, content, and link sharing patterns. Rich context helps in understanding how people communicate and identifying deviations from the normal routine to determine what does and does not belong in an individual’s inbox and outbox.  

Other email security vendors may claim to use behavioral AI and unsupervised machine learning in their products, but their AI are still pre-trained with historical data or signatures to recognize malicious activity, rather than demonstrating a true learning process. Darktrace’s Self Learning-AI truly learns from the organization in which it is installed, allowing it to detect unknown and novel vectors that other security tools are not yet trained on.

Because Darktrace understands the human behind email communications rather than knowledge of past attacks, Darktrace / EMAIL can stop the most sophisticated and evolving email security risks. It enhances your native email security by leveraging business-centric behavioral anomaly detection across inbound, outbound, and lateral messages in both email and Teams.

This unique approach quickly identifies sophisticated threats like BEC, ransomware, phishing, and supply chain attacks without duplicating existing capabilities or relying on traditional rules, signatures, and payload analysis.  

The power of Darktrace’s AI can be seen in its speed and adaptability: Darktrace / EMAIL blocks the most novel threats up to 13 days faster than traditional security tools.

Learn more about AI-led BEC threats, how these threats extend beyond the inbox, and how organizations can adopt defensive AI to outpace attacker innovation in the white paper “Beyond the Inbox: A Guide to Preventing Business Email Compromise.”

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Carlos Gray
Senior Product Marketing Manager, Email

More in this series

No items found.

Blog

/

/

May 8, 2025

Anomaly-based threat hunting: Darktrace's approach in action

person working on laptopDefault blog imageDefault blog image

What is threat hunting?

Threat hunting in cybersecurity involves proactively and iteratively searching through networks and datasets to detect threats that evade existing automated security solutions. It is an important component of a strong cybersecurity posture.

There are several frameworks that Darktrace analysts use to guide how threat hunting is carried out, some of which are:

  • MITRE Attack
  • Tactics, Techniques, Procedures (TTPs)
  • Diamond Model for Intrusion Analysis
  • Adversary, Infrastructure, Victims, Capabilities
  • Threat Hunt Model – Six Steps
  • Purpose, Scope, Equip, Plan, Execute, Feedback
  • Pyramid of Pain

These frameworks are important in baselining how to run a threat hunt. There are also a combination of different methods that allow defenders diversity– regardless of whether it is a proactive or reactive threat hunt. Some of these are:

  • Hypothesis-based threat hunting
  • Analytics-driven threat hunting
  • Automated/machine learning hunting
  • Indicator of Compromise (IoC) hunting
  • Victim-based threat hunting

Threat hunting with Darktrace

At its core, Darktrace relies on anomaly-based detection methods. It combines various machine learning types that allows it to characterize what constitutes ‘normal’, based on the analysis of many different measures of a device or actor’s behavior. Those types of learning are then curated into what are called models.

Darktrace models leverage anomaly detection and integrate outputs from Darktrace Deep Packet Inspection, telemetry inputs, and additional modules, creating tailored activity detection.

This dynamic understanding allows Darktrace to identify, with a high degree of precision, events or behaviors that are both anomalous and unlikely to be benign.  On top of machine learning models for detection, there is also the ability to change and create models showcasing the tool’s diversity. The Model Editor allows security teams to specify values, priorities, thresholds, and actions they want to detect. That means a team can create custom detection models based on specific use cases or business requirements. Teams can also increase the priority of existing detections based on their own risk assessments to their environment.

This level of dexterity is particularly useful when conducting a threat hunt. As described above, and in previous ‘Inside the SOC’ blogs such a threat hunt can be on a specific threat actor, specific sector, or a  hypothesis-based threat hunt combined with ‘experimenting’ with some of Darktrace’s models.

Conducting a threat hunt in the energy sector with experimental models

In Darktrace’s recent Threat Research report “AI & Cybersecurity: The state of cyber in UK and US energy sectors” Darktrace’s Threat Research team crafted hypothesis-driven threat hunts, building experimental models and investigating existing models to test them and detect malicious activity across Darktrace customers in the energy sector.

For one of the hunts, which hypothesised utilization of PerfectData software and multi-factor authentication (MFA) bypass to compromise user accounts and destruct data, an experimental model was created to detect a Software-as-a-Service (SaaS) user performing activity relating to 'PerfectData Software’, known to allow a threat actor to exfiltrate whole mailboxes as a PST file. Experimental model alerts caused by this anomalous activity were analyzed, in conjunction with existing SaaS and email-related models that would indicate a multi-stage attack in line with the hypothesis.

Whilst hunting, Darktrace researchers found multiple model alerts for this experimental model associated with PerfectData software usage, within energy sector customers, including an oil and gas investment company, as well as other sectors. Upon further investigation, it was also found that in June 2024, a malicious actor had targeted a renewable energy infrastructure provider via a PerfectData Software attack and demonstrated intent to conduct an Operational Technology (OT) attack.

The actor logged into Azure AD from a rare US IP address. They then granted Consent to ‘eM Client’ from the same IP. Shortly after, the actor granted ‘AddServicePrincipal’ via Azure to PerfectData Software. Two days later, the actor created a  new email rule from a London IP to move emails to an RSS Feed Folder, stop processing rules, and mark emails as read. They then accessed mail items in the “\Sent” folder from a malicious IP belonging to anonymization network,  Private Internet Access Virtual Private Network (PIA VPN) [1]. The actor then conducted mass email deletions, deleting multiple instances of emails with subject “[Name] shared "[Company Name] Proposal" With You” from the  “\Sent folder”. The emails’ subject suggests the email likely contains a link to file storage for phishing purposes. The mass deletion likely represented an attempt to obfuscate a potential outbound phishing email campaign.

The Darktrace Model Alert that triggered for the mass deletes of the likely phishing email containing a file storage link.
Figure 1: The Darktrace Model Alert that triggered for the mass deletes of the likely phishing email containing a file storage link.

A month later, the same user was observed downloading mass mLog CSV files related to proprietary and Operational Technology information. In September, three months after the initial attack, another mass download of operational files occurred by this actor, pertaining to operating instructions and measurements, The observed patience and specific file downloads seemingly demonstrated an intent to conduct or research possible OT attack vectors. An attack on OT could have significant impacts including operational downtime, reputational damage, and harm to everyday operations. Darktrace alerted the impacted customer once findings were verified, and subsequent actions were taken by the internal security team to prevent further malicious activity.

Conclusion

Harnessing the power of different tools in a security stack is a key element to cyber defense. The above hypothesis-based threat hunt and custom demonstrated intent to conduct an experimental model creation demonstrates different threat hunting approaches, how Darktrace’s approach can be operationalized, and that proactive threat hunting can be a valuable complement to traditional security controls and is essential for organizations facing increasingly complex threat landscapes.

Credit to Nathaniel Jones (VP, Security & AI Strategy, Field CISO at Darktrace) and Zoe Tilsiter (EMEA Consultancy Lead)

References

  1. https://spur.us/context/191.96.106.219

Continue reading
About the author
Nathaniel Jones
VP, Security & AI Strategy, Field CISO

Blog

/

/

May 6, 2025

Combatting the Top Three Sources of Risk in the Cloud

woman working on laptopDefault blog imageDefault blog image

With cloud computing, organizations are storing data like intellectual property, trade secrets, Personally Identifiable Information (PII), proprietary code and statistics, and other sensitive information in the cloud. If this data were to be accessed by malicious actors, it could incur financial loss, reputational damage, legal liabilities, and business disruption.

Last year data breaches in solely public cloud deployments were the most expensive type of data breach, with an average of $5.17 million USD, a 13.1% increase from the year before.

So, as cloud usage continues to grow, the teams in charge of protecting these deployments must understand the associated cybersecurity risks.

What are cloud risks?

Cloud threats come in many forms, with one of the key types consisting of cloud risks. These arise from challenges in implementing and maintaining cloud infrastructure, which can expose the organization to potential damage, loss, and attacks.

There are three major types of cloud risks:

1. Misconfigurations

As organizations struggle with complex cloud environments, misconfiguration is one of the leading causes of cloud security incidents. These risks occur when cloud settings leave gaps between cloud security solutions and expose data and services to unauthorized access. If discovered by a threat actor, a misconfiguration can be exploited to allow infiltration, lateral movement, escalation, and damage.

With the scale and dynamism of cloud infrastructure and the complexity of hybrid and multi-cloud deployments, security teams face a major challenge in exerting the required visibility and control to identify misconfigurations before they are exploited.

Common causes of misconfiguration come from skill shortages, outdated practices, and manual workflows. For example, potential misconfigurations can occur around firewall zones, isolated file systems, and mount systems, which all require specialized skill to set up and diligent monitoring to maintain

2. Identity and Access Management (IAM) failures

IAM has only increased in importance with the rise of cloud computing and remote working. It allows security teams to control which users can and cannot access sensitive data, applications, and other resources.

Cybersecurity professionals ranked IAM skills as the second most important security skill to have, just behind general cloud and application security.

There are four parts to IAM: authentication, authorization, administration, and auditing and reporting. Within these, there are a lot of subcomponents as well, including but not limited to Single Sign-On (SSO), Two-Factor Authentication (2FA), Multi-Factor Authentication (MFA), and Role-Based Access Control (RBAC).

Security teams are faced with the challenge of allowing enough access for employees, contractors, vendors, and partners to complete their jobs while restricting enough to maintain security. They may struggle to track what users are doing across the cloud, apps, and on-premises servers.

When IAM is misconfigured, it increases the attack surface and can leave accounts with access to resources they do not need to perform their intended roles. This type of risk creates the possibility for threat actors or compromised accounts to gain access to sensitive company data and escalate privileges in cloud environments. It can also allow malicious insiders and users who accidentally violate data protection regulations to cause greater damage.

3. Cross-domain threats

The complexity of hybrid and cloud environments can be exploited by attacks that cross multiple domains, such as traditional network environments, identity systems, SaaS platforms, and cloud environments. These attacks are difficult to detect and mitigate, especially when a security posture is siloed or fragmented.  

Some attack types inherently involve multiple domains, like lateral movement and supply chain attacks, which target both on-premises and cloud networks.  

Challenges in securing against cross-domain threats often come from a lack of unified visibility. If a security team does not have unified visibility across the organization’s domains, gaps between various infrastructures and the teams that manage them can leave organizations vulnerable.

Adopting AI cybersecurity tools to reduce cloud risk

For security teams to defend against misconfigurations, IAM failures, and insecure APIs, they require a combination of enhanced visibility into cloud assets and architectures, better automation, and more advanced analytics. These capabilities can be achieved with AI-powered cybersecurity tools.

Such tools use AI and automation to help teams maintain a clear view of all their assets and activities and consistently enforce security policies.

Darktrace / CLOUD is a Cloud Detection and Response (CDR) solution that makes cloud security accessible to all security teams and SOCs by using AI to identify and correct misconfigurations and other cloud risks in public, hybrid, and multi-cloud environments.

It provides real-time, dynamic architectural modeling, which gives SecOps and DevOps teams a unified view of cloud infrastructures to enhance collaboration and reveal possible misconfigurations and other cloud risks. It continuously evaluates architecture changes and monitors real-time activity, providing audit-ready traceability and proactive risk management.

Real-time visibility into cloud assets and architectures built from network, configuration, and identity and access roles. In this unified view, Darktrace / CLOUD reveals possible misconfigurations and risk paths.
Figure 1: Real-time visibility into cloud assets and architectures built from network, configuration, and identity and access roles. In this unified view, Darktrace / CLOUD reveals possible misconfigurations and risk paths.

Darktrace / CLOUD also offers attack path modeling for the cloud. It can identify exposed assets and highlight internal attack paths to get a dynamic view of the riskiest paths across cloud environments, network environments, and between – enabling security teams to prioritize based on unique business risk and address gaps to prevent future attacks.  

Darktrace’s Self-Learning AI ensures continuous cloud resilience, helping teams move from reactive to proactive defense.

[related-resource]

Continue reading
About the author
Pallavi Singh
Product Marketing Manager, OT Security & Compliance
Your data. Our AI.
Elevate your network security with Darktrace AI
OSZAR »