Hybrid AI: A Better Model for Leveraging Machine Learning and Human Expertise
Artificial intelligence (AI) – i.e., algorithms and machine learning processes – are drivers of many of today’s major innovations in software and hardware, from social media chatbots to highly automated customer relationship management applications. What’s next for AI?
One of the key differences to look at in the years ahead will be traditional AI versus hybrid AI, which we can understand by comparing two AI-driven technologies: IBM Watson and the Tesla Model S.
Watson versus Tesla
These two categories were put forward at the start of a September 2017 SC 20/20 Webcast discussion of the future of AI, to contrast traditional and hybrid AI. Hybrid AI goes further than the former category by integrating human expertise. Could it be the secret to making bots, machines and applications more intelligent than the ones that rely on the limited AI of today?
IBM Watson, which was identified in the webinar as an example of conventional AI, has become an important exploratory tool in fields such as health care. However, the CIO of the Mayo Clinic called it “still pretty dumb” in terms of how it arrives at its insights.
It needs to plow through massive amounts of data to “learn” anything, putting it in a similar position to a human toddler who can only determine if a stove is hot by actually touching it, whereas an adult could infer it was unsafe to touch by evaluating various cues and evidence. Machine learning needs these advanced human insights to make its own inferences and deductions about datasets and also place its conclusions in context.
“What all this AI is lacking is an ontological model where you can describe a structure abstractly,” observed Cris Ross of the Mayo Clinic, according to Med City News. “Watson had no idea what a patient was, what a hospital is, what a doctor is, what a drug is, what the effect is on a patient, what’s the relationship between a doctor, drug, a patient and an outcome.”
In contrast to Watson, there are the Tesla automobiles. Tesla is best known for making electric vehicles, but also for its integration of cellular connectivity, powerful software and features supported by such infrastructure, including full self-driving technology within all Tesla models. This feature is a form of AI, albeit one that can be overridden by a human as needed.
However, the Tesla example discussed at length during the webinar isn’t fully representative of hybrid AI’s structure or possible use cases. In cybersecurity, hybrid AI is less like the potentially dramatic human intervention possible in a Tesla Model S/X and more like subtle, continuous and powerful guided analysis, which helps better identify and defend against threats.
Traditional AI versus hybrid AI in cybersecurity
It is easy to see how this hybrid AI model could be applied to cybersecurity:
- The standard AI component would be one part of a security operations center (SOC), which would leverage machine learning and big data analytics alongside a Security Information and EventManagement (SIEM) platform to set up and enforce policies.
- At the same time, a security engineer would be overseeing its operations and tweaking these policies as needed.
- The combination of the SOC, SIEM and the engineer should make for a smarter security architecture that produces fewer false positives.
Blacklisted IP address, malicious websites and various known threats could be blocked automatically by a hybrid AI-enabled SOC, without any intervention. But human insight would be much more useful in situations involving novel threats with no history, or when entering “gray areas” caused by sophisticated tactics such as spear-phishing.
More specifically, a suspicious email attachment might cause problems for a standard AI-driven defense – is it truly dangerous, or it is just harmless spam? Traditional AI might not excel at picking up on subtle cues such as awkward phrasings, minor typos/alternative spellings, threats and references to government agencies in the body of an email containing such an attachment – all tell-tale signs of a phishing scheme often overlooked.
How hybrid AI can enhance a SOC
To see how hybrid AI might improve a SOC, first we need to think about how a typical SOC works. The normal sequence of operations is for it to gather information such as event logs and network flow data, transform it into correlated observations and then produce alerts, which spur actions such as the quarantining of a server, the blocking of a website or the disabling of a compromised user account. These capabilities dramatically reduce exposure to possible data breaches.
It sounds like a seamless process, yet there are many moving parts that have to work together to complete challenging tasks. In terms of machine learning, there’s the integration of malware analysis, threat intelligence and behavior analytics. Meanwhile, human security engineers have to implement custom rules so that the entire SOC meets organizational security requirements.
Of course, there is the question of what level of human intervention is optimal within hybrid AI. Both autonomous (machine-driven) and non-autonomous (human-directed) processes have their specific use cases:
Autonomous machine learning
- Efficiently screens out noisy benign events on the network.
- Automatically blocks known threats (e.g., connections from untrusted locations or from TOR nodes).
- Detects anomalous behavior in relation to “normal” patterns.
Non-autonomous human activity
- Avoids frequent blocking of legitimate traffic and other false positives.
- Helps catch one-off attempts at infiltrating the network, keeping false negatives to a minimum.
- Can provide innovative solutions, such as safely detonating a possibly new strain of malware in a sandbox.
Basically, human intuition, expertise and experience helps fill in the evaluative gaps in a SOC – which activities cross the line, and which ones don’t. According to a white paper cited during the webinar, human-supervised AI has tenfold the accuracy of an unsupervised alternative in security situations.
The biggest difference is in navigating complex threats, such as polymorphic malware that may appear differently depending on the context. These numbers line up well with the results of a 2016 MIT study, which revealed 85 percent malware prediction accuracy for a prototype system analyzing billions of pieces of log-line data and sending results to humans for further inspection.
Why hybrid AI provides a simpler path forward for security teams
The careful balance of machine learning and human intelligence in a SOC with hybrid AI makes it superior to a traditional SOC in terms of cost-effectiveness, ease of use and overall functionality. A hybrid AI-enabled SOC can take advantage of scalable cloud-based infrastructure, outsourced security experts and reusable playbooks. There’s no need for the expensive hardware upgrades, SIEMs and extra personnel.
“The balance of machine learning and human intelligence in a SOC with hybrid AI makes it superior to a regular SOC.”
Moreover, a SOC with hybrid AI can help solve the common problem of finding and replacing experienced security staff. The scalability and predictability of SOC-as-a-Service, with 24/7 monitoring, means that even one external engineer can manage security operations for multiple customers and extend a shared base of knowledge to all of them. HR departments don’t have to go hunting for months to find the right candidate for a senior security position.
Finally, as hybrid cloud environments become more common, a SOC with hybrid AI is a reliable platform for making sense of the many disparate sources of log and flow data. In early 2017, RightScale found that a majority of enterprises had a hybrid cloud strategy in place, meaning that they were using both on-premises infrastructures and public clouds such as Amazon Web Services and Microsoft Azure. A SOC-as-a-Service with hybrid AI can gather information from all components in a hybrid cloud and enable consistent management of applications across environments, without having to rely on the service provider.
To learn more about hybrid AI’s utility in cybersecurity, be sure to view the entire webcast at BrightTALK. Also click the banner below to see how managed detection and response capabilities from AWN support a modern CyberSOC that helps defend against today’s most advanced threats.