Evaluating the economic disparities in the world: Sentiment Analysis on Central Bank Speeches from Third World and First World Countries

doi:10.21203/rs.3.rs-2953752/v1

Download PDF

Research Article

Evaluating the economic disparities in the world: Sentiment Analysis on Central Bank Speeches from Third World and First World Countries

https://doi.org/10.21203/rs.3.rs-2953752/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

This paper provides an evaluation of the financial and simple sentiment of central bank speeches from 2004–2019. The speeches are categorized into two different sub-groups with three different countries for each one, respectively: Zambia, Barbados, and Sri Lanka for the third world [developing] group and France, Canada, and Japan for the first world [developed] group. This paper attempts to answer the following questions: (i) What do the sentiments of each country or sub-group tell us about their economic condition? (ii) How does the sentiment relate to the countries' contextual economic growth or decline? (iii) How do the central or federal banks of the country portray the economy? Using natural-language processing, more specifically BERT transformer models and modern NLP methods, a synthesized evaluation is created of what economic linguistics from central banks can reveal about the world's economy.

sentiment analysis

natural language processing

central banks

speeches

economy

Although the terms "first world" and "third world" are infamously coined from geopolitical alliances during the Cold War, they can be translated into gradings for the economic conditions of different countries. First-world countries contain capitalistic structures, high standards of living, and stable financial systems. On the other hand, third-world countries are continuously developing with low standards of living and median incomes, poor public infrastructures, lack of access to common resources, and unstable financial conditions. There are currently 152 countries that are classified as part of the third world, which is much greater than the 43 which are part of the first world [1]. Majority of the world wakes up to a different economic condition than those in the first world, but why is that? It is not easy to pinpoint a singular factor as there are many different aspects to the fluctuating answer to this question. Components like corruption, historical conditions, geological location, and many more have been thoroughly researched to add value to the growing list of factors. However, there is one section that has not been readily explored for findings, and it has the potential to reveal a new perspective on this topic: central bank speeches.

Central banks are federal institutions that control the monetary policies of their respective countries. They are additionally responsible for revealing certain private economic data, executing crucial decisions to alter the economy, and, most importantly, reporting their workings to the public through a main medium of information: speeches. Central banks' communication of data has been heavily responsible for swaying the public's view and expectations of the financial conditions of the country. Research regarding central bank speeches has been conducted on the evolution of topics they talk about [2], the extent to which it influences the public, their credibility [3], and what they signal for the future. The unexplored field is described as how certain linguistics of the speech can be classified into a labeled sentiment, which is directly applicable to the overall question: What sets developed countries apart from developing ones? By looking at the sentiments of the speeches (positive, negative, and neutral), the attitudes that central banks are trying to exhibit towards two vastly different populations can be understood. Even though there remains some optimistic bias within these speeches that is naturally embedded to not give the public extremity of the economic conditions, these texts can still reveal a lot about the economy and the government's outlook on it.

The two groups contain three countries each: Zambia, Barbados, and Sri Lanka for the developing and Japan, France, and Canada for the developed nations. The reasons for choosing these specific countries are based on the speeches available, their language-translation accuracy, and their economic/geological positions in the world. Each group has countries that vary in their financial position in the world, specifically GDP and continental positioning to create a diverse range of results. This paper contains a section with the economic context (2004–2019) of each country to back up the results and maintain a reference to what the findings are. In terms of what the sentiment shows us, some relevant hypotheses can be applicable: Does a high positive sentiment in speeches exhibit a confident plan for change or a tactic of deceit to not scare the public and damage foreign relations? Is a high negative sentiment on speeches a reference to the truthfulness of data or a lack of care from the governments to alleviate the public's worry? This paper will dive into these possibilities using natural language processing (NLP), a field of machine learning that studies linguistics.

The research is based on the core concept of transformers, a specific architecture in NLP that solves different classification goals. Two differently trained fine-tuned variations of the BERT transformer [4] RoBERTa are used [5]. An array of tool kits is also used to smooth the results and pre-processing tasks, including PyTorch [6], which is the main framework for this experiment. The dataset where the speeches are found is based on the paper Evolution of topics in central bank speech communication [2], which derives the speeches from central banks affiliated with the Bank of International Settlements. Each country is regulated to a range of speeches, sentences, and time frames, for efficiency reasons. Details for the processes and components of the experiment are discussed in the Methods section of the paper. Transitioning from a technical view, the paper also discusses why this experiment is conducted and what impact the results hope to achieve. Given the scope and limitations of this experiment, it is hard to provide a large impression toward the goal. However, the aim is for the results to unveil the verbal habits of central banks and the patterns within both groups. The overall purpose of this paper is to provide readers with a new insight into what differentiates a majority of our world and how central governments handle that issue.

The paper will follow the format of introducing the topic with contextual information, providing economic information for the countries in the groups, describing the methods that were used to execute this evaluation, the results, and its interpretations, and finally a conclusion of the research, which includes a discussion of errors and possible future work.

A. Zambia

Zambia is a developing country in South Africa, with a total population of 18.92 million people [7] as of 2021. Its true modern economic history was born in 1964 when it achieved independence [8]. Since then, Zambia has been one of the fastest-growing countries in Africa due to an array of political reforms and mineral exports. They also achieved a 10.3% economic growth rate in 2010 [9], which heavily reduced their poverty rates and moved them into a better foreign position. However, despite all this newfound trade success, Zambia is still a third- world country due to its poverty rates and trade growth stall due to declining mineral prices.

B. Barbados

Barbados is an island country located in the southeastern Caribbean Sea in North America. They have a total population of roughly 288,000 people as of 2021 [7], which puts them on the lower side of density for countries. It became independent in 1966 when many private merchants lost power within the country's economy [10]. Although their economy and trade ran heavily off sugar, they have found new success in the tourism industry, which they rely heavily upon, and liquor exports [10]. However, regardless of their economic drivers, their financial condition is still fragile due to the amount of debt they owe and declining tourism rates.

C. Sri Lanka

Sri Lanka is also an island south of India, in Asia. It has a population of 22.16 million people as of 2021 [7], with growing financial and social progress. Sri Lanka runs their economy on customer services, competitive export of agricultural products, and manufacturing of various goods. They have successfully cut their poverty rates by nearly half and completed many other goals that third-world countries aspire to finish, in order to increase their global position. Yet, they were still classified as developing in 2019 due to a volatile market and low median salary [11]. The following figure, Fig. 1, represents the relative, GDP growth rates for the respective third world countries in the time that is being analyzed. The graph provides additional context that can be correlated after the results.

Fig. 1. Economic Growth Rate for Third World Countries from 2004–2019 [9].

In Fig. 1, it is evident that the economic growth rate for the developing countries has a relatively flat pattern, with a slight decline in recent years.

D. France

France is a first world and progressive country located on the western edge of Europe, with a total population of 67.5 million [7] as of 2021. France has been a major competitor in global trade for centuries and has had many political moves within other continents, like Asia, Africa, North America, and more. Although having a diversified economy, the most significant contributing industries are manufacturing, power, and pharmaceuticals. Like other third-world countries, they have a highly privatized sector that develops most of the economy. However, in recent years they have had small bursts of growth with a somewhat volatile economy, leading them to a much shorter annual growth [9].

E. Japan

Japan is an island off the east coast of Asia with a population of 125.68 million persons [7] as of 2021. It is the third biggest country regarding [9], adding value to its global economic board position. In addition, they have become major manufacturers in the technology and car field with private companies like Sony, Toyota, and many more. However, like other first-world countries, they have relatively volatile economic statistics and have been experiencing fast or even negative growth in the last few years.

F. Canada

Canada is the second biggest country in the world, in terms of land mass, located in the northern section of North America, with a population of 38.25 million persons [7] as of 2021. They are a significant ally and generally reserved country regarding the economy, as they have fewer volatile numbers than the other two first-world countries. A heavy portion of Canada's economy is based on real estate, mining, manufacturing, and international trade, as they have many exports of natural resources. Even though they have fewer fluctuating statistics, they still have a slowly weakening economy with fast growth and, sometimes, decline [9]. The following figure, Fig. 2, represents the relative, GDP growth rates for the respective third world countries in the time that is being analyzed. The graph provides additional context that can be correlated after the results.

Fig. 2. Economic Growth Rate for First World Countries from 2004–2019 [9].

In Fig. 2, it can be seen that first world countries have had high volatility within the last 15 years, with major spikes and trouphs scattered around diferent periods. In the recent years, it growth or decline has slowed down and been compressed below 2.5%.

The experiment is based on the "Central Bank Speeches" dataset published by Magnus Hansson [2]. The speech text files are scraped from the Bank for International Settlements public database and grouped into their respective country's folders. Since its original intended use differs from the one of this experiment, the goal was to convert these varying .txt files into an array of tokenized sentences for each speech. The chosen countries had a high variance in the number of speeches, sentences per speech, and types of text characteristics unique to the style it was scraped in. The main problem that pops up from this is that there is no consistency in the style of speech or formatting, which indicates that there needs to be a high amount of cleaning and consistent sorting to regulate the data. Although, the character markers before each header or footer made it much easier to remove the unnecessary data. The standard headers, footers, Unicode characters, and other items that weren't classified as valuable toward sentiment were all removed. The next step after cleaning the dataset was to develop the text corpus into individual sentences. The Natural Language Tool Kit (NLKT) sentence tokenizer was utilized to break the text stream into individual sentences per speech [12]. The tokenizer identified breaks in the text by punctuation, and more markers, and made each sentence an essential 'token.' Following the tokenization was the data frame creation, which housed the country, speech number, sentence number within that speech, and the actual sentence itself. The third world group contained 12001 sentences, and the first world group had 12851 sentences in their respective text corpora. Each country averaged around 3 to 4 thousand sentences found and 35 to 65 speeches used. The reasoning on why this range and variance of number of speeches and sentences works is because there was the same number of minimum speeches each year, and each speech was regulated to a specific number of sentences that can be used.

With the newly formatted data, it was time to pick a model to represent our data with the best results. The variation of NLP models that was chosen was BERT, a Google-based transformer [4] with high performance and efficient usage for the data that is being used. Transformers are machine learning architectures that can solve classification and sequence tasks with high performance and speed. Choosing the simple BERT [4] transformer and fine-tuning it would not add much modern value to the results, so RoBERTa [5], an optimized BERT [4] model with different parameters was chosen to decrease scope and optimize the results. which is high enough to yield results that are valuable towards the experiment. RoBERTa's [5] base needed to be fine- tuned for sentiment analysis, which is a text classification task. A new problem arose: which data should the model train on? The real solution comes down to what the analysis is looking for. A straightforward and logical answer would be to train RoBERTa [5] on financial data and news so that its sentiment grading is relevant to what the speeches are talking about. However, an average citizen from majority of these countries would not commonly possess economic knowledge on the same level as the finance model, so a parallel model would be required to compare results on what the data seems to an average person and what it seems to a highly knowledgeable finance expert.

The financial model was a fine-tuned RoBERTa model, trained on 10k, 10q, 8k, earning calls transcripts, CSG reports, ESG news, and financial news text data [13]. The model had 24 layers, including a Gaussian Error Linear Unit, GeLu, activation layer. It processed PyTorch [6] tensors of max size 514 and had an average accuracy rate of 94.5% [13]. The other simple model is a fine-tuned RoBERTa called SiEBERT [14] that was trained on 13 standard English data sets (Yelp, etc.) This model shares the same layer amount and max tensor size as the financial model and has an accuracy rate of 93.2% [14]. Both models functioned the same in terms of usage, except the only difference is that the financial model gave positive, negative, or neutral labels to each sentence, whereas the simple model only had negative and positive. The mediations for this difference in labeling are justified and explained in the “Results” section of this paper. In regards to accuracy, both models met standards to produce results that are reliable and valuable. Both models function the same: they take in the sentence as a singular entity, tokenize into tokens with lemmatization, convert it into an embedding (PyTorch Tensor) [6] and run it through each layer with its trained weights. The result is found within the last hidden layer, which keeps the sentence within the embedding form. The HuggingFace pipeline [15] was also used to convert the results from a tensor to a label and accuracy score. The pipeline had a softmax layer that converted the tensor into a 0–1 value. The sentiment scores were then iterated through in their original label format and appended back to the data frame.

The results that were extracted were the total positive, neutral, and negative sentences for each country from the financial model and positive and negative sentences for each country from the simple model. Although only two imported values are what the experiments provides, the findings can be correlated with economic context to make the evaluations function on a much larger and more profound scale. The following table, Table 1, exhibits the sentiment classification for the sentence corpus, by the financial RoBERTa [5].

TABLE I. Financial Roberta

Economic Group	Positive %	Neutral %	Negative %
Third World	27.24%	15.51%	57.05%
First World	22.67%	20.52%	56.81%

The finance model's results can be considered from a perspective of a financial expert who understands the data and can grade sentiment on it. It is not a representation of a standard citizen's opinion but more of a technical understanding of the speeches. In regards to the accuracy of the results, the model performed close to what was expected, with an average accuracy (confidence) metric of 96% for first world countries and 94% for third world countries. The confidence metrics satisfy the bar for reliable resutlts (90%). Now although the difference between each model’s classification accuracy rates is extremely low, it could still indicate that first world countries are more clear on their financial data than the third world ones, which could provide context for the results evaluation below. Now, the overall results are fascinating as they open many different questions and interpretations of what it could mean. The neutral sentences took up more than half of the data, which indicates that these speeches are not leaning towards a particular sentiment and just stating information. Since both groups' neutral values are highly close, it can inferred that third world and first-world central bank speeches do not necessarily point to something and are valueless to an extent within grading sentiment. On the other hand, each group's positive and negative sentences differ significantly. Even though the percentage changes are minor, in a scope with relativity to the mass of the data, it makes a big difference. Third- world countries are talking more about positive things than first-world ones and have fewer negative things to talk about, with an approximated five percent difference for both. So, what can be inferred from this data? For third world countries, it could mean two things: either these countries are being honest with their data and are having much higher relative growth than the first world countries, or they are hiding certain factors and skewing the speech towards one direction to not scare the public or break foreign relations.

For the first possibility, if they are transparent, the data is optimistic for third-world countries and is showing good growth, even with the slowdown in recent years. As seen in Fig. 1, more growth can be noticed than the first-world countries in Fig. 2, which experienced a slow decline. Although, if the second possibility is true, it shows that these governmental financial institutions skew the data to create political safety for the country. Corruption is a severe factor that is much more present in the third world than in first world countries and can be an aspect that affects the results [16]. The results for the first world countries show the same amount of negative and positive topics talked about, which exhibits that this group's countries are transparent with their data, reaching their economic peaks and not growing as much as they used to. Now, what does this data, from a financial lens, tell us? Third-world countries are growing faster than their first-world counterparts and are catching up with new technology implementations. Alternatively, third-world countries misinform the public through their speeches, which adds global concerns to external factors like extreme corruption being prominent in political ladders. Even with a proper economic context correlating with one of the possibilities, the other cannot be disregarded as there may be a detachment between what the governments say and what is happening. The following table, Table 2, exhibits the sentiment classification for the sentence corpus, by SiEBERT [14].

TABLE II. Simple Roberta

Economic Group	Positive %	Negative %
Third World	75.86%	24.14%
First World	70.82%	29.18%

The simple english model and the financial model both showed impressive accuracy rates in their respective metrics. SiEBERT [14] produced an accuracy rate of 92.5% for first world countries and 91% for third world countries, which are considered to be reliable results given that the confidence metrics met the bar of 90%. These high accuracy rates could be due to the fact that first world countries have better communication means, resulting in average citizens being able to understand these speeches in a more effective manner. However, it is also possible that the accuracy rates are influenced by translated text which the model does not account for. When it comes to sentiment analysis, the results of the simple model were significantly different from those of the financial model. The simple model conveyed a higher percentage of positive sentences, which can be attributed to the exclusion of neutral sentiment, leading to a 15% increase in positive grading compared to negative ones. The difference between the third and first world groups was still very apparent, with the former having less positive and more negative sentiment than the latter. The simple model provides us with valuable insights into how an average person with limited financial knowledge perceives these speeches. This can have important implications for the market and the level of ignorance that citizens may have due to the orientation of the sentences.

While the simple model statistics present the speeches in a positive light, they do not accurately represent the true sentiment of the data. They only show what comes across to an average citizen. The results of the simple model can be correlated with the context of each country and sub-group, allowing for a better understanding of the differences between these groups and how their specific context affects their perception of these speeches. However, it is crucial to keep in mind that the simple model may not accurately reflect the true sentiment of the data and therefore, it is recommended to use it in conjunction with other models or methods to obtain a more comprehensive understanding of the sentiment behind these speeches. The simple model has important limitations that should be taken into consideration when analyzing the sentiment behind these speeches. However, its results can provide valuable insights into the perception of average citizens and should be used in conjunction with other models or methods to gain a more comprehensive understanding of the sentiment behind these speeches. The correlation for the results with the context for each country and sub-group, and some of the charactersitics for each group can be identified, is discussed in the following section.

The evaluation of speeches related to economics has resulted in significant findings. The economic conditions of different countries and sub-groups were determined by analyzing the sentiments present in the speeches. A high negative sentiment was often correlated with declining economic conditions, and vice versa. However, the results also revealed a different perspective where sentiments were not in line with the contextual economic facts, showcasing the behavior of the speeches. The communication about the economy is portrayed in two ways: trustable and non-trustable, where truth can either hurt or help the image of the country or the unreliable data in the speeches can mislead citizens. The findings have answered the questions presented in the abstract to a satisfactory level, but more experiments with different subjects are required to achieve a thorough evaluation.

While the results are promising, there are still potential errors and areas for improvement. The accuracy of the data cleaning process is one such area. Poor data cleaning could have skewed the results to be more neutral, positive, or negative. Another factor to consider is the selection of countries and data. The countries were chosen based on the data's receivability and how well it fit the models, but other vital factors were not taken into account. It is difficult to consider every single aspect, but different countries could lead to different results, providing a different perspective. Increasing the data would increase the accuracy and confidence in the findings. Technology growth must also be taken into consideration when analyzing the results. In first-world countries, technology is highly saturated, while in third-world countries, it is still being explored. This creates a difference in economic conditions and can affect the results. This field of study is vast, and the findings of this paper are just the tip of the iceberg. Further research can be conducted by examining other texts, such as local financial publications and online portrayals of the economy from the media. These sources could either back up or contradict the results. The evaluation of speeches has resulted in significant findings regarding global economic disparities. While there are still potential errors and areas for improvement, the results are promising. More experiments and research are necessary to uncover greater truths about the economy. The growth of technology and selection of data and countries must also be considered when analyzing the results. This field of study is vast and offers many opportunities for further exploration. Even though a singular factor cannot be pinpointed, this paper has found research value in a highly influential literary media of governmental institutions in all countries: Central bank speeches.

Funding

No external or internal funding was provided.

Author contribution

The manuscript was completely created by KV.

conflict of interest

The author declares that they have no competing interests.

Data availability statement

The data that the analysis was performed on originates from this research paper:

https://doi.org/10.48550/arXiv.2109.10058 [2]

Image data were extracted from www.worldbank.org.

Table data was generated based on technical experiments ran on the overall data.

The rest of the data referred to is publicly available online.

List of Developing Countries in the World. (2021). Worlddata.info: https://www.worlddata.info/developing-countries.php
Hansson, M. (2021). Evolution of topics in central bank speech communication. arXiv. doi: 10.48550/ARXIV.2109.10058
Hwang, I. D. (2022). Central Bank Communication and Public Trust: The Case of ECB Speeches. SSRN.
Tenney, I., Das, D., & Pavlick, E. (2019). BERT Rediscovers the Classical NLP Pipeline. doi:10.48550/ARXIV.1905.05950
Liu, Y. a. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv. doi:10.48550/ARXIV.1907.11692
Imambi, S., Prakash, K. B., & Kanagachidambaresan, G. R. (2021). PyTorch. K. B. Prakash & G. R. Kanagachidambaresan, Programming with TensorFlow: Solution for Edge Computing Applications (σσ. 87–104). doi:10.1007/978-3-030-57077-4_10
World Bank Data Population. (2021). Retrieved from The World Bank Data: https://data.worldbank.org/indicator/SP.POP.TOTL
Kanduza, A. M. (n.d.). TOWARDS A HISTORY OF IDEAS IN ZAMBIA. Transafrican Journal of History, 1990. Retrieved from http://www.jstor.org/stable/24328673
World Bank Data GDP. (2021). Retrieved from The World Bank Data: https://data.worldbank.org/indicator/NY.GDP.MKTP.CD
Ramsay, A. O. (2022). Barbados Breaks Free. NACLA Report on the Americas. doi:10.1080/10714839.2022.2084974
Devapriya, U. (2022). The Crisis in Sri Lanka: Economic and Political Dimensions. Journal of Indo-Pacific Affairs.
Hardeniya, N., Perkins, J., Chopra, D., Joshi, N., & Mathur, I. (2016). Natural Language Processing: Python and Nltk. PACKT Publishing.
Soleimanian, M. (2022). Do Firms Walk Their Talk in Corporate Social Responsibility Reports? – Evidence From Forward-Looking Statements. SSRN.
Hartmann, J. a. (2022). More than a feeling: Accuracy and Application of Sentiment Analysis. International Journal of Research in Marketing.
Jain, S. M. (2022). Hugging Face. Στο Introduction to Transformers for NLP: With the Hugging Face Library and Models to Solve Problems (σσ. 51–67). doi:10.1007/978-1-4842-8844-3_4
Olken, B. A., & Pande, R. (2012). Corruption in Developing Countries. Annual Review of Economics, 4(1), 479–509. doi:10.1146/annurev-economics-080511-110917
Correa, R. K. (2017). Sentiment in Central Banks’ Financial Stability Reports. International Finance Discussion Papers 1203.
Geloso, V. (2021). The Economic History of Canada. SSRN. doi: http://dx.doi.org/10.2139/ssrn.3895172
Priola, M. P. (2011). Measuring Central Banks’ Sentiment and its Spillover Effects with a Network Approach. SSRN.
Siakoulis, A. P. (2021). Can central bank speeches predict financial market turbulence? Evidence from an adaptive NLP sentiment index analysis using XGBoost machine learning technique. Science Direct. doi: https://doi.org/10.1016/j.cbrev.2021.12.002
Taghizadeh-Hesary, N. Y. (2015). Japan’s Lost Decade: Lessons for Other Economies. ADBI Institute.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Evaluating the economic disparities in the world: Sentiment Analysis on Central Bank Speeches from Third World and First World Countries

Status:

Version 1

Abstract

Figures

I. Introduction

II. ECONOMIC CONTEXT

III. Materials & Methods

IV. Results and discussion

V. Conclusions

Declarations

References

Additional Declarations

Status:

Version 1