Life, the Universe and AIOps

AIOps is an area of technology that is developing rapidly and is generally accepted to mean “using machine learning to contextualize large amounts of data”.

While data analytics systems are great at identifying unusual patterns in large amounts of data (for example in a data lake) they can be quite poor at providing context to the signals they detect.

A general statement that is common in data analysis is this “I have found something that looks interesting, but I have no idea what it means”.

Providing context to signals within the data is the next big need, and it is where AIOps provides value.

Everyone who has read Douglas Adam’s seminal work “The Hitchhikers Guide to the Galaxy” will be aware that the answer to the ultimate question of life, the universe, and everything is 42. The issue (according to Adams) is that the answer means nothing without a deeper understanding of the question. And this is the challenge with data, without an understanding of the context of the data, then the answer is only of limited value.

AIOps tries to address this by providing an understanding of the importance of the relationships between different flows of data. And it does this by breaking up the data processes into the following components:

1. Ingestion – Getting the data into a single place where it can be analyzed
2. Topology – Presenting a model of how the data flows through the entire system, how it all connects
3. Correlation – Mapping the data against time to see what happened when
4. Recognition – Identifying signals in the data and understanding what they mean in the context of the business
5. Remediation – Understanding what actions must be taken to fix issues, and how these can be automated

We make use of the inherent knowledge built into the configuration of messaging middleware systems to deliver AIOPS. Instead of trying to recreate complex algorithms to describe the business’s topology, we use the very systems that you have already spent untold blood and gold configuring. By reading the configuration information from across your many messaging middleware environments, and even the contents of the messages themselves, we are able to visualize the entire topology of your enterprise application stack, exactly as your users’ transactions see it.

To overlay onto this topology all the data you are already collecting to describe the flow of your business. Then we can compare the historical record of transactions to each new transaction that takes place, allowing any deviance to be recognized before any impact is felt by the user. Using machine learning artificial intelligence (ML AI) we can alert operations staff early enough (predictively or proactively) that remediation can be performed before an event becomes an issue, and using ML AI we create automation to perform these tasks based on alerts or user requests.

The Nastel Platform delivers AIOps today for many large enterprises, and it is being used for both operational management and delivery of regulatory compliance.