» Resources » Eye Security Blog »
Written by Eye Security
To fully unlock the potential of AI systems in cybersecurity, organisations need effective quality management processes. Best-practice guidelines for the responsible use of AI algorithms and LLM data should be accompanied by privacy-friendly IT system design and robust cybersecurity measures. Given the many unresolved legal questions in this area, organisations should address them in an open and structured manner.
AI systems also introduce a range of legal and regulatory risks. Since not all methods for validating the accuracy and performance of AI systems are reliable, companies must carefully assess the regulatory and operational risks associated with these technologies.
Artificial Intelligence (AI) refers to the development of computer systems capable of performing tasks and making decisions that typically require human intelligence. These systems can learn from experience, recognise patterns, and solve problems without being explicitly programmed. AI systems encompass both software and hardware that use AI to act “rationally” in physical or digital environments. They are deployed across a wide range of applications, from automating business processes to enhancing cybersecurity. The importance of AI systems continues to grow as they help organisations operate more efficiently and securely.
Machine Learning (ML) is a widely used subfield of AI. ML algorithms and techniques enable systems to learn from data and make decisions without explicit programming. By analysing large datasets and identifying patterns, ML continuously improves system performance, enabling smarter, more adaptive AI solutions.
Deep Learning (DL) is a specialised subset of machine learning that uses artificial neural networks inspired by the human brain. These networks can tackle complex tasks such as image recognition, natural language processing, and autonomous decision-making. By applying deep learning, organisations gain deeper insights from data and can develop more advanced, AI-driven solutions.
Requirements for data privacy and explainability present a real challenge for many AI users. The growing complexity of machine learning and the increasing use of deep learning algorithms, which often function as black boxes, means that highly accurate results can be achieved without a clear understanding of how those results were generated.
The benefits, however, remain significant. When applied to IT security, AI enables security teams to efficiently process vast amounts of data, detect and analyse security events of varying complexity, maintain real-time oversight of system status, and respond to cyberattacks.
According to IBM’s Total Cost of Data Breach Report 2024, companies that use AI in their cybersecurity operations save an average of $3.05m (approximately €2.83m) and 74 workdays in addressing security incidents.
The following provides an overview of the most common applications of AI for protecting critical enterprise systems against cyberattacks:
AI can be integrated into Intrusion Detection and Prevention Systems (IDPS) to identify attackers and trigger predefined protective measures, such as blocking IP addresses or isolating compromised systems.
Intrusion Detection Systems (IDS) can autonomously detect attacks or assist security teams in identifying them. Supervised machine learning (ML) algorithms use historical attack data as training input to recognise similar events. However, supervised ML struggles to detect novel attack patterns. Unsupervised anomaly detection, on the other hand, identifies behaviors that deviate significantly from the norm but may generate a higher number of false positives.
AI-driven automation can also be applied in threat mitigation, such as Incident Response (IR) automation. AI systems detect threats and respond according to predefined playbooks, helping minimise risk and accelerate reaction times. Typical IR automation steps include attack detection, isolation of compromised systems, blocking malicious activity, and recovery via backups. Regular system reviews additionally ensure AI-driven IR remains efficient and secure.
SIEM solutions aggregate comprehensive system data to detect and respond to incidents in real time. AI enhances SIEM capabilities by analysing both structured data, such as logs, and unstructured data, including app crash reports, employee-submitted alerts, or AI-generated visualisations of threats.
AI can also predict future threats by analysing datasets and identifying patterns. This goes beyond historical cyberattacks; AI systems use threat intelligence to collect data from security logs, the dark web, and threat databases, enabling organisations to anticipate attacks and adapt defences proactively.
Incidents can be detected by analysing data from ongoing attack campaigns or monitoring forums. Natural Language Processing (NLP) is often employed to assess potential communication between threat actors and identify risks.
The imperative of AI and automation in cybersecurity: Without AI and automation, managing modern cyber risks is nearly impossible. At the same time, using AI in cybersecurity requires a robust quality management strategy to ensure the technology is effective, safe, and aligned with organisational security goals.
AI systems can also be manipulated by attackers to produce false or misleading results. This can create a range of issues, including violations of data protection laws and compromises to the security of systems and applications. For example, attackers could alter AI models so that they fail to detect legitimate threats or mistakenly classify harmless activities as dangerous.
Regular reviews and updates to AI systems ensures they remain secure and reliable. Additionally, organisations should educate employees about the risks associated with AI systems and encourage caution when interacting with these technologies. Providing training and clear guidelines can promote the safe and responsible use of AI tools.
In summary, by combining technical safeguards with employee education, companies can minimise AI-related risks and strengthen their overall security posture.
The following outlines key criteria and implementation measures first presented in the Fraunhofer Institute for Secure Information Technology‘s whitepaper “Eberbacher Gespräche über KI, Sicherheit und Privatheit“. These recommendations focus on ensuring responsible, secure, and legally compliant AI use within corporate cybersecurity frameworks.
Machine Learning (ML) systems often demonstrate different performance in controlled lab environments compared to real-world operational settings. It can be difficult to determine the exact basis on which results are generated. Companies should assess AI systems using metrics such as False Acceptance Rate (FAR) and False Positive Rate (FPR), alongside transparency regarding the training data employed.
Key takeaway: Decisions made by AI systems must be verifiable, and outputs should be reproducible to ensure reliable cybersecurity performance.
Once raw data is fed into an AI model, removing it is typically impractical without retraining the model, which is costly and time-consuming. Organisations should therefore implement privacy-preserving mechanisms to safeguard data integrity before processing.
While Big Data and AI can enhance business insights, they pose challenges to data privacy. Analyses can create links between data points, undermining intended anonymity. In some cases, neural network training sets may allow reconstruction of original data, potentially revealing personal information.
Best practice: Develop verifiable protocols for pseudonymisation or anonymisation of personal data before it is used in AI processing, ensuring compliance with privacy regulations.
Unclear liability rules can deter companies from fully leveraging AI for cybersecurity. Without a clear understanding of legal risk, organisations may avoid otherwise plausible AI applications.
Following the adoption of the EU AI Act, the European Commission released guidelines on AI liability, aiming to create a harmonised legal framework and address gaps in accountability for AI-driven outcomes.
Several frameworks exist to guide ethical and safe AI deployment, such as the Hambach Declaration and recommendations from Germany’s Federal Data Ethics Commission. A corporate Code of Conduct for AI should be:
Additional challenges stem from the diverse range of threat actors who increasingly employ AI-driven methods as offensive tools. These actors may include financially motivated cybercriminals as well as state-sponsored groups targeting critical infrastructure. In this regard, comprehensive risk management strategies should encompass the prioritisation of threat actors and the establishment of tiered priority levels for potential target locations.
Key measures include:
Gartner predicts that organisations applying AI TRiSM controls will reduce inaccurate or unauthorised outputs by at least 50% by 2026, minimising errors that could lead to faulty decisions.
Organisations are increasingly using AI-driven systems to enhance their security posture, enabling faster threat detection, more effective attack mitigation, and improved forecasting of emerging risks. At the same time, the adoption of such technologies introduces new challenges that must be addressed with equal rigour, including limited model transparency, data protection concerns, and the absence of clearly defined liability frameworks.
Against this backdrop, organisations are encouraged to establish quality standards for AI systems and implement privacy-friendly mechanisms, such as data anonymisation, from the outset. With a well-structured approach to quality and risk management, AI can not only reduce security risks but also serve as a strategic tool for maintaining a competitive advantage.
AI is used to detect threats faster, prevent attacks, and predict future risks. Machine learning (ML) and deep learning (DL) help organisations identify patterns, automate threat detection, and respond to incidents in real time. AI transforms cybersecurity from a reactive practice into a proactive strategic tool.
AI, including LLMs and deepfake tools, allows attackers to generate highly personalised phishing, vishing, and other social engineering campaigns. It can mimic writing styles, create contextually relevant content, and scale attacks rapidly, making them harder to detect.
Yes. AI can assist in:
When paired with robust quality and risk management practices, AI can:
Explainable AI ensures that organisations can understand how decisions are made. This is critical for compliance, incident investigation, and validating threat detection accuracy. Without explainability, high-performing models may be mistrusted or underutilised.
AI adoption is accelerating. Organisations that proactively integrate AI with structured quality standards, privacy measures, and risk management frameworks will reduce cybersecurity threats and leverage AI as a strategic business advantage.
Research and Technical
A lot can change in 12-months. Your business may have added new systems, suppliers, or employees. The amount of data you hold may have increased. Your security controls may have changed. And your current insurance policy may no longer offer the protection you think it does.
Cyber budgets have climbed every year for a decade. Attacks keep succeeding anyway. This gap tells us that the size of the budget was rarely the issue. September is when European leadership teams set the budget allocation for 2027. Every department competes for the same investment, and cybersecurity has to earn its place alongside the […]
If a threat actor logged into one of your employees’ Microsoft 365 accounts right now, how long would it take you to notice?
