“Traditional" OSINT (Open-Source Intelligence) is the exploration, discovery, and analysis of entity-related content, using open-source resources on the Internet. The activity involves querying various search applications, examining the results, filtering relevant data, processing it, and then creating reports based on the information and context found.

The above process is difficult to automate, given the huge role of the analyst in investigation, so our solution does not even attempt full automation but provides a highly scalable and modularly extensible basic infrastructure that can significantly improve the efficiency of investigation.

Main features and components of the solution

• Automatic text analytical processing of collected data
• Automatic building of an internal knowledgebase (entity database)
• Visualisation of relationships

Data processing

After collecting the relevant data and documents, the analyst can choose between the following three options:

• prepare a report on the entity in the traditional way based on the found information, and can supplement it with information already loaded in the "Internal Knowledgebase"
• load the information extracted from the search results into the "Internal Knowledgebase" using automatic data processing and then use its visual analysis interface to gather additional information for the report
• perform manual data processing using the solution's data and text-mining tools

During data processing, the following entities are extracted

• people - names in the following nationalities: Hungarian, Slavic, Arabic, Russian/Ukrainian, Romanian, Georgian, English
• names of companies, organisations, based on the types of companies in 10 European countries: Austria, Slovakia, Ukraine, Romania, Serbia, Croatia, Slovenia, Germany, Russia, Poland
• E-mail address, IP address, URL

At the end of the data processing phase, a data structure is created which can now be loaded into an "Internal Knowledgebase" (case-related entities and their attributes, relationships, related reports, etc.). The data processing component is based on the predictive and text analytics toolset of IBM SPSS Modeler Premium.

Internal Knowledgebase (data storage)
The "Internal Knowledgebase" has an Entity repository which is a continuously expanding structured database that contains the entities, attributes, and relationships of searches, cases, and subsequently, on demand, the metadata of cases (e.g. relevant links, their content) and the content of reports.

i2 iBase, part of the solution, is an easy-to-use intelligence database application that enables collaborative teams of analysts to capture, control and analyse data from multiple sources in a workgroup with a range of security services. It facilitates the daily tasks of analysts, which include searching across entities and discovering and exploring networks, patterns, and trends. i2 iBase also provides a search interface for analysts to search the Internal Knowledge Base.

Visualisation
The Internal Knowledgebase is linked to a visualisation, analysis, and search interface - i2 Analyst's Notebook, which allows searching in entity data, metadata, and visual analysis of relationships and connections between entities.