“How do I know what I do not know?”
Micro Focus IDOL (Intelligent Data Operating Layer) is an AI platform which derives contextual and conceptual insights from multiple data sources. It can identify relationships that exist within virtually any type of information and Micro Focus runs a demo environment that indexes a limited subset of news articles, broadcast monitoring feeds from various TV and radio channels (via our OSINT model), twitter and RSS feeds every single day. So, I have access to more than 22 million indexed documents. Here’s a video demonstration you can watch too if you’d like:
In this classic case of Open Source Intelligence I will demonstrate how I used this indexed but unstructured mass of information to track the spread of the headline-hitting COVID-19 coronavirus using Micro Focus IDOL.
Using the keyword “Coronavirus” within the demo environment, IDOL was swiftly able to identify and group the following concepts which it deemed important in the corpus of data within the index. This is presented visually as a ‘Topic Map’ which enables an analyst to see a snapshot of all the information related to the given topic:
By selecting “coronavirus cases”, IDOL provides me with new topic map drilling down into the relevant data set further:
Drilling into the details
“Confirmed cases” was my next selection which provided an instant further subset of results. In just 2 clicks IDOL has distilled the information held in 22 million documents into 1519 documents and sorted them by date (which could also be searched by relevance) presented like this:
From the topic map, I can also see that South Korea has been identified as a country of interest in relation to confirmed cases. By selecting “South Korea” as the next topic of interest, the results are further narrowed to 577 documents. Here is a screenshot of the documents in the result set which pertain to “democratic values failing in the midst of virus outbreak” and “tourism” related articles etc.
And this is where it gets really clever. Micro Focus IDOL provides the ability to group articles into pre-trained categories. One of these categories is the health category. So if I now filter by the ‘health category’ I’ve narrowed my results to a mere 56 articles:
From an initial corpus of 22 million articles I now see 56 articles. The top-most result contains the exact information of the number of confirmed cases of Coronavirus in South Korea (sorted results by date, not relevance). This result was the latest at the time of this demo and sadly shows 476 new cases of Coronavirus in South Korea.
By saving the results as a “Coronavirus” set I can now see the trend of the document throughput which conceptually matches confirmed cases of Coronavirus in South Korea. So the image below shows that there has been a marked spike in documents ingested in the last 24 hours directly relating to confirmed cases of Coronavirus in South Korea:
Using IDOL’s geotagging capabilities of unstructured text, we are able to map cities across the world affected by coronavirus cases. As you can see from the map news in the USA is concentrating on Washington State. Again this clearly indicates the location of the outbreak there with 119 incidents of COVID-19 reported in news articles.
IDOL uses a spectrograph to track the evolution of topics of interest across a period. We are now publishing live via News Spectrum which displays the spread of Coronavirus #COVID-19 on a day to day basis. The thickness of the lines between news topics indicates an increase in news articles reflecting those topics. You can also see how we can see a given topic evolves into sub-topic related to the Coronavirus. This is evident when a line forks into two or more lines connecting other related topics on the following day.
We can also use Micro Focus IDOL to track related incidents, like the current, rather annoying but nonetheless newsworthy ‘toilet paper crisis’. If I type ‘Coronavirus’ and ‘Toilet Paper’ into IDOL I am presented with the following Topic Map which needs no further explanation:
This demonstrates how Micro Focus IDOL is able to mine vast quantities of information from open-source data to quickly provide valuable insights. Additionally IDOL has over 150 connectors for enterprise repositories such as SharePoint, File System, Exchange etc. enabling it to provide insights from Internal data within individual Organisations repositories (while respecting the security model governed by the document ACLs (Access Control Lists)) inside a Firewall too.
For more information on Micro Focus IDOL please visit the homepage here. To contact me directly you can find me easily here on Twitter or here on LinkedIN if you prefer. And of course, as an organisation taking Corporate Social Responsibilty very seriously we’d be delighted to discuss ways our technology could help in real world situations like the COVID-19 virus outbreak. Please don’t hesitate to contact Micro Focus directly, via Facebook, LinkedIN or send us a Tweet and we will be in touch soon.