Local models for security operations: sovereignty, latency and cost

Security telemetry is among your most sensitive data

Logs and alerts are not anonymous statistics. They name users and administrators, list hostnames and internal addresses, describe which systems hold which data, record which accounts have privileged access and show where defences are weak. Put together, a SOC’s telemetry is a detailed map of the organisation, and exactly the map an attacker would want.

When an AI add-on sends alerts and log excerpts to a cloud model for analysis, that map leaves the premises. Even with strong contractual protection, the data is now processed under another party’s infrastructure, staff and jurisdiction. For many organisations that raises questions of data sovereignty, regulatory compliance and contractual obligation that are difficult to answer.

The organisations most at risk are the ones left out

Many AI security offerings exist only as a cloud service tied to a cloud model. They cannot be installed on the customer’s servers, in a private cloud or on a network that is isolated from the internet. For banks, government agencies, telecom operators and energy and utility companies, that often ends the evaluation before it starts.

These are the sectors that hold the most sensitive data, run critical infrastructure and face the most persistent attackers. They also tend to operate segmented or air-gapped networks where cloud connectivity is restricted by design. A security approach that requires sending telemetry to the cloud leaves exactly these organisations to manage the alert flood by hand.

Masking or anonymising data before it is sent helps only partly. Investigation depends on the very details masking removes: which account, which host, which internal address. Strip them out and the model cannot reason about the case; keep them in and the data has left.

Why smaller, specialised models fit security work

A frontier model is built to do almost anything: write poetry, answer trivia, generate code. Most SOC tasks are much narrower. Deciding whether an alert belongs with an existing case, recognising a routine backup, judging whether a login is unusual for this user, choosing which log source to check next: each is a focused task with a well-defined input and output.

Focused tasks suit compact models trained for that task. They can run on modest on-premises hardware, answer in a fraction of the time of a remote call and handle very high volume at a predictable cost. Because they run where the data lives, they can be adapted to the customer’s own environment and keep improving as cases are settled, without the data ever leaving.

Large models still have a role. For the rare case that is genuinely novel, broad reasoning can help, and some organisations will choose to allow it. The difference is that it becomes an option used sparingly and only under the customer’s policy, rather than the engine behind every alert, and what it contributes can be turned into local capability so the same question does not need to go out again.

Questions to ask any AI security vendor

• Can the product run fully on our own servers, in a private cloud and on an isolated network?
• Which data, if any, leaves our environment by default, and can we turn that off?
• If an external model is used, who decides when, and is every call recorded?
• How does cost change when our alert volume doubles?
• Does the system learn from our own settled cases, and how is an improvement verified before it is trusted?
• Does it work with the SIEM and EDR we already run, without replacing them?

The approach ApexAI takes

ApexAI ships as an on-premises and private-cloud product. Its default AI runs locally, and a cloud model is an opt-in choice governed by your policy. It works with the tools you already run, with Elastic, CrowdStrike Falcon and Palo Alto Cortex XSIAM today and Microsoft Sentinel next, and it keeps learning from your environment so that more of the work is handled locally over time. The Why ApexAI page puts this in the context of the wider SOC problem.

Questions