Data cleaning using OpenRefine: A case of blast incidents and explosives research data extraction from social media platforms

Abstract

The increase in explosive and blast incidents has resulted in the need for a method or tool to help monitor and keep track of said incidents to provide insight into the methodologies and devices used by attackers to develop counter-active measures. The prevalence of social media in society has harnessed the generation of data and information which can be applied by organisations to improve functions and processes which in turn helps aid decision-making. With regards to the field of explosives, more institutions and news outlets have opted for the use of social media and other online platforms to share real-time data pertaining to explosives and blast related incidents and events that occur on a global scale. This study presents a method for extracting and capturing data pertinent to the field of explosives research from social media sites and other online platforms through monitoring and highlighting any blast related incidents, events and trends. The data used in the paper was collected over a period of 10 months and retrieved and extracted using a predetermined form which specified the type of information that should be mined and processed. The data was then cleaned, visualised and presented using various techniques available on the OpenRefine application. This data then provided insight into, but not limited to questions such as: the most prevalent sources, types of blast incidents or events, the types of devices used; casualties etc. The results of the study helped improve operations within the unit through enabling the identification and improvement of training efforts by shedding light on the effective methodologies and techniques used by explosives ordnance disposal (EOD) groups. The results of the study also helped compile a list of tope perpetrators and users of explosives; potential suppliers and/or collaborators which can be used if the need prevails and the data regarding the countries with the highest incidents helped to shift the focus on which states to keep track of.