Critical Insights Into the Design of Big Data Analytics Research: How Twitter "Moods" Predict Stock Exchange Index Movement

Abstract

The research explored whether one or more of the South African Twitter moods could be used to predict the movement of the Johannesburg Stock Exchange (JSE) All Share Index (ALSI). This is a proof of principle study in the field of big data analytic research in South Africa, which is at a relatively early stage of development. The research methods used secondary data from Twitter’s application programming interfaces (APIs), and formulated a model to extract public mood data and search for a causal effect of the mood on the closing values of the JSE ALSI. Over three million tweets were gathered and analysed over a 55-day period, with data collected from the JSE for 39 weekdays, from which only one variable (mood states) was considered. Four of the South African Twitter mood states did not produce any correlation with the movement of the JSE ALSI. The mood Depression had a significant negative correlation with the same day’s JSE ALSI values. The major finding was that there was a highly significant positive correlation between the Fatigue mood and the next day’s closing value of the JSE ALSI, and a significant causality correlation from the Fatigue mood to the JSE ALSI values. The findings support the behavioural finance theory (Wang, Lin & Lin, 2012), which states that public mood can influence the stock market. Organisations and governments could use Twitter data to gauge public mood and to ascertain the influence of public mood on particular issues. However, very large data sets are required for analytical purposes, possibly five to ten years of data, without which predictability is likely to be low.