1.1 Background and Significance
Understanding human capital is vital for grasping the mechanisms of economic growth and development. Human capital encompasses the attributes that individuals bring to the labour market, which include education, skills, and health. In the knowledge economy, these attributes are critical drivers of productivity and innovation (Blonigen et al., 2003; Hanushek & Woessmann, 2022).
The role of education in fostering economic growth has been a topic of intense debate and research in the fields of economics and education policy. Knowledge Capital Theory (KCT), which underpins this claim, posits that the knowledge and skills acquired by students are the principal assets that contribute to economic growth as measured by per capita GDP (Hanushek & Woessmann, 2015a). While KCT has gained significant traction in policy circles, it has also faced criticism for its potentially reductionist approach to education and development. Whereas KCT provides a framework to quantify educational outcomes and their economic implications, its methodology has been criticised for its narrow focus on measurable variables and its inability to capture the full complexity of education systems and the impact of cultural and social capital (Komatsu & Rappleye, 2019; Rappleye & Komatsu, 2019b).
To address these methodological challenges and provide a more robust evaluation of KCT, this study employs the Abductive Theory of Science Method (ATOM) (Haig, 2022; Williams, Forthcoming-b). ATOM serves as a critical framework for evaluating the hard cores of research programmes in a consistent and replicable manner (Williams, Forthcoming-b). This approach aligns with Lakatos' concept of scientific research programmes and allows for a systematic assessment of KCT's progressive or degenerative nature (Lakatos, 1978; Williams, 2022).
Imre Lakatos's (1978) concept of research programmes offers a framework for understanding the evolution of scientific theories, although it is not without its critics. Unlike Karl Popper's falsificationism, which suggests that theories should be abandoned when contradicted by empirical evidence (Popper, 1959), Lakatos proposed that scientific theories are part of larger research programmes that can persist and develop even in the face of apparent contradictions (Lakatos, 1970). These research programmes consist of a 'hard core' of fundamental assumptions protected from falsification by a 'protective belt' of auxiliary hypotheses. However, critics like Feyerabend (1975) argue that this approach can lead to the dogmatic protection of core theories, potentially hindering scientific progress.
Lakatos argued that research programmes can be evaluated as either 'progressive' or 'degenerating' based on their theoretical and empirical progressiveness (Lakatos, 1978). Theoretical progressiveness refers to the programme's ability to predict novel facts, while empirical progressiveness is demonstrated when these predictions are subsequently corroborated by evidence. A programme is considered progressive if it is both theoretically and empirically progressive; otherwise, it is deemed degenerating. This distinction is crucial, as it acknowledges that a programme may continue to make novel predictions (theoretical progress) even if these predictions are not immediately confirmed (lack of empirical progress). However, this framework has been criticised for its potential subjectivity in determining what constitutes 'novel facts' and for its difficulty in providing clear criteria for when a programme should be abandoned (Laudan, 1977). Moreover, Kuhn (1962) argued that this view may not adequately capture the revolutionary nature of paradigm shifts in science, where entire conceptual frameworks, rather than just specific theories, are replaced. Nevertheless, the framework is helpful to framing the discussion regarding the strengths and areas of challenge for a research programme.
The Sustainable Development Goal (SDG) 4 emphasises the importance of inclusive and equitable quality education for all, with Target 4.1 specifically focusing on ensuring that all children achieve minimum proficiency in literacy and numeracy (United Nations (UN), 2018; United Nations Department of Economic and Social Affairs (UN DESA), 2016). To effectively monitor progress towards this target, globally comparable measures of learning outcomes are essential (Sandoval-Hernández, 2022).
In response to these challenges, two significant datasets have emerged: the Rosetta Stone Project and the Harmonised Learning Outcomes (HLO) database. The Rosetta Stone Project, led by the UNESCO Institute for Statistics (UIS) and the International Association for the Evaluation of Educational Achievement (IEA), aims to address the challenge of harmonising data from different assessments by linking existing regional assessments to a common global metric (Sandoval-Hernández, 2022). Similarly, the HLO database, established by the World Bank, is a pivotal step forward in measuring educational outcomes globally, collecting and harmonising data from various international and regional student assessments (Angrist et al., 2021).
These datasets offer unique advantages over the Programme for International Student Assessment (PISA) when evaluating the legitimacy of Knowledge Capital Theory. They encompass a wider array of regional assessments, including ERCE and PASEC, alongside TIMSS and PIRLS, allowing for a more diverse and representative sample of educational outcomes, particularly from developing regions that may be underrepresented in PISA (Angrist et al., 2021; Sandoval-Hernández, 2022).
This study aims to leverage these comprehensive datasets to critically examine KCT through a Critical Realist lens, addressing the methodological gaps in traditional educational data analysis and applying the ATOM framework. By doing so, we seek to provide a more nuanced understanding of the relationship between education quality and economic growth, potentially refining theoretical and methodological models while underscoring the profound implications of harmonised educational data for future research directions and educational policy development. The use of ATOM in this context allows for a systematic and replicable evaluation of KCT as a research programme, offering insights into its explanatory power and empirical adequacy across diverse global contexts.
The study's significance lies in its attempt to bridge the gap between the empirical support for KCT and the need for a more nuanced understanding of the education-economy relationship. By employing novel datasets and a Critical Realist perspective, this research aims to provide a more comprehensive evaluation of KCT's claims while acknowledging the complex, context-dependent nature of educational outcomes and their economic impacts.
1.2 Knowledge Capital Theory and its Limitations
Knowledge Capital Theory (KCT) extends the concept of human capital to a more nuanced understanding that includes the value of an individual's knowledge – both formal and tacit – and their ability to apply it. This theory highlights the importance of educational quality and expertise in fostering economic growth (Hanushek & Woessmann, 2015a, 2015b). KCT posits that the knowledge and skills individuals acquire through education are crucial for personal and national economic success (Blonigen et al., 2003; Hanushek & Woessmann, 2022).
In the framework of PISA, KCT underpins the assessment of reading, mathematics, and science competencies, which are seen as indicators of a country's future economic prowess. The empirical evidence gathered from PISA is used to support the argument that higher levels of knowledge capital correlate with greater economic output (Hanushek & Woessmann, 2015a, 2015b), which in turn are used to inform national and international policy changes.
However, there are significant deficiencies in the measurement of knowledge capital, as traditional metrics may not fully account for the complexities of knowledge application and its impact on productivity. These deficiencies highlight the need for more robust metrics that can capture the breadth and depth of knowledge capital.
KCT's focus on these narrow competencies, while useful for benchmarking, does not encompass the full spectrum of skills relevant in modern economies, such as critical thinking, creativity, or socio-emotional abilities (Brown et al., 2020; Lauder et al., 2018). Moreover, the theory's assumptions regarding the transferability of education to economic productivity do not account for the complexities of different economic structures or the changing nature of work (Brown et al., 2020; Shields & Sandoval Hernandez, 2020).
Recent critiques have also highlighted the potential cultural bias in KCT's approach, suggesting that its emphasis on certain cognitive skills may reflect Western educational priorities rather than universal indicators of economic potential (Zhao, 2020). Furthermore, the theory has been criticised for potentially oversimplifying the complex relationship between education and economic development, neglecting the role of institutional factors and social contexts (Furuta, 2022).
Another limitation of KCT lies in its reliance on PISA data, which primarily focuses on OECD member countries and a selection of partner economies. This narrow focus may lead to a skewed understanding of the relationship between education and economic growth, particularly in developing countries or regions with different educational and economic contexts (Rappleye & Komatsu, 2019a).
Furthermore, KCT's emphasis on quantifiable outcomes may overlook the qualitative aspects of education that contribute to economic and social development. This includes the role of cultural context, social capital, and non-cognitive skills in shaping economic outcomes (Carnoy, 2015).
Despite these limitations, it is important to note that substantial research suggests a real relationship between cognitive skills and economic growth, albeit mediated by multiple other factors (Apostu et al., 2022; Burhan et al., 2023; Meisenberg & Lynn, 2013; Rindermann & Thompson, 2011). This underscores the need for a more nuanced approach that can account for both the empirical support for KCT and the complex, context-dependent nature of the education-economy relationship.
The limitations of KCT underscore the need for a more comprehensive approach to understanding the relationship between education and economic growth. This approach should consider a broader range of skills, account for diverse economic and cultural contexts, and utilise more inclusive datasets that represent a wider array of countries and regions. By addressing these limitations, researchers can develop a more nuanced and accurate understanding of the complex interplay between education and economic development.
1.3 Aim and Research Questions
The aim of this paper is to apply a Critical Realist approach to examine the relationship between education quality and economic growth using harmonised data from the Rosetta Stone Project and the Harmonised Learning Outcomes (HLO) database. By leveraging these comprehensive datasets, we seek to critically evaluate the claims of Knowledge Capital Theory (KCT) and provide a more nuanced understanding of the education-economy nexus across diverse global contexts.
This study employs the Abductive Theory of Science Method (ATOM) as a framework for systematically assessing KCT as a research programme. Through this approach, we aim to uncover underlying mechanisms and contextual factors that influence the relationship between educational outcomes and economic growth, thereby addressing the limitations of current methodologies used in International Large-Scale Assessments (ILSAs).
To achieve these aims, the study seeks to address the following research questions:
Is the relationship between education quality and economic growth as indicated by H&W, replicated when analysed with the Rosetta Stone Dataset and the Harmonised Learning Outcomes (HLO) database?
This question is designed to test the robustness of KCT's claims across a broader range of countries and educational contexts than previous studies, while also exploring the potential of a Critical Realist approach to provide deeper insights into the education-economy relationship.
These questions are shaped to abductively hypothesise and empirically scrutinise the intricate dynamics between educational quality and economic development. By addressing these questions, we aim to:
a) Test the robustness of KCT's claims across a broader range of countries and educational contexts.
b) Identify potential limitations in the current understanding of the education-economy relationship.
c) Explore the potential for a Critical Realist approach to provide a more comprehensive understanding of the complex, context-dependent nature of the education-economy relationship.
Through this comprehensive analysis, we aim to contribute to the refinement of theoretical models in educational economics and provide policymakers with more nuanced, culturally and contextually appropriate insights for harnessing the full potential of education in diverse socio-economic landscapes. The significance of this study lies in its use of novel datasets and a Critical Realist perspective to bridge the gap between the empirical support for KCT and the need for a more nuanced, context-sensitive approach to understanding the education-economy relationship.
1.4 Ontological, Epistemological, and Methodological Perspectives
This study adopts a Critical Realist (CR) ontological and epistemological framework, offering a nuanced approach to examining the relationship between education quality and economic growth. This perspective addresses longstanding critiques of econometric approaches in both economics and educational research, providing a more robust methodological foundation.
1.4.1 Critical Realism as a Fresh Lens
Critical Realism, as developed by Bhaskar (2008) and elaborated by Archer (1998), offers a philosophical middle ground between positivism and interpretivism. It acknowledges the existence of an objective reality while recognising that our knowledge of it is socially constructed and fallible.
CR's stratified ontology distinguishes between the real (underlying structures and mechanisms), the actual (events that occur whether observed or not), and the empirical (our observations and experiences) Bhaskar (2008). This perspective is particularly valuable in educational research, as it allows us to investigate not only observable outcomes but also the underlying mechanisms and structures that generate them.
Importantly, CR addresses the critiques raised by scholars like Blaug (1980) regarding the prevalence of 'measurement without theory' in economics and, by extension, in educational economics. Blaug's observation that many studies rely on 'cookbook econometrics', where theoretical arguments are adjusted post hoc to fit statistical results, highlights the need for a more rigorous approach to theory development and testing.
In the context of this study, Critical Realism provides a framework for examining Knowledge Capital Theory that goes beyond simple empirical testing. It allows us to explore the potential underlying mechanisms that link educational outcomes to economic growth, while also acknowledging the complex, open systems in which these relationships operate. This approach is particularly relevant given the diverse global contexts represented in the Rosetta Stone and HLO datasets.
Furthermore, CR's emphasis on the interplay between structure and agency aligns well with the study's aim to provide a more nuanced understanding of the education-economy relationship. It allows us to consider how educational systems and policies (structures) interact with individual and societal factors (agency) to produce observed outcomes.
While this study does not claim to definitively identify underlying causal mechanisms, the CR approach guides the interpretation of results and informs the discussion of potential causal processes. It provides a framework for moving beyond mere correlations to consider explanatory factors, albeit with the understanding that in open social systems, causality is complex and context-dependent.
1.4.2 Critical Realism and the ATOM Framework
The Abductive Theory of Science Method (ATOM) (Haig, 2018, 2019, 2022; Haig & Evers, 2015), recently modified by Williams (forthcoming), aligns well with Critical Realist principles and addresses the concerns raised by Blaug. ATOM provides a structured approach to theory development and evaluation, offering a systematic framework for assessing research programmes like Knowledge Capital Theory (KCT).
The model consists of several stages which are outlined briefly in Table 1. It is important to note that this study does not proceed from steps 1 to 14 of the ATOM framework but rather focuses on step 9, as outlined in Table 1. This step involves the evaluation of explanatory coherence, which is particularly relevant to the study's aim of critically examining KCT using novel datasets.
By employing ATOM within a CR framework, we can enhance our analytical approach in several ways. Firstly, it allows us to identify empirical demi-regularities in the Rosetta Stone and HLO datasets, moving beyond simple correlations to uncover more complex patterns. Secondly, this approach enables us to abductively hypothesise potential causal mechanisms, eschewing post hoc rationalisations in favour of theoretically grounded explanations. Furthermore, we can retroductively elaborate on these hypotheses, carefully considering the necessary contextual conditions for their operation. In the context of this study, the ATOM framework guides the analysis of the Rosetta Stone and HLO datasets, providing a structured approach to theory evaluation that aligns with Critical Realist principles. It allows for a systematic assessment of KCT's explanatory power and empirical adequacy across diverse global contexts, while also qualitatively exploring potential underlying mechanisms and contextual factors with the view of extending this research in the future with more rigorous empirical methods.
Moreover, the CR perspective, combined with ATOM, provides a methodological framework that can accommodate the complexity and diversity of global educational contexts. This is particularly crucial when working with datasets like Rosetta Stone and HLO, which encompass a wider range of countries and educational systems than typically considered in KCT research.
By adopting this approach, we aim to move beyond the 'measurement without theory' critique, ensuring that our analysis is grounded in robust theoretical foundations while remaining open to empirical insights. This methodology allows us to critically evaluate KCT while also exploring new avenues for understanding the complex interplay between education and economic development across diverse global contexts.
The combination of CR and ATOM in this study represents an attempt to bridge the gap between the empirical support for KCT and the need for a more nuanced, context-sensitive understanding of the education-economy relationship. Rarely are the ontological and epistemological perspectives of researchers and studies explicitly stated within ILSAs and so this is attempt to help redress this current situation. While acknowledging the limitations of what can be conclusively established in complex social systems, especially within the limits of a single paper, this approach provides a robust and transparent framework for advancing understanding in this critical area of educational research.
In summary, a Critical Realist stance, coupled with the ATOM framework, offers a fresh perspective on the relationship between education quality and economic growth. It provides a rigorous approach to theory development and testing, addressing longstanding concerns about the misuse of econometric techniques in educational research and economics more broadly.
Table 1
Summary of the revised ATOM framework (Williams, Forthcoming-b)
| Stage | Process Stage | Description |
| 1 | Description of Datasets | Detailed characterisation of the data, including sources, types, and potential for explanatory analysis. |
| 2 | Theoretical Justification of Variables | Selection and justification of variables based on theoretical grounds and their potential for contributing to explanatory hypotheses. |
| 3 | Establishment of Data Patterns | Identifying robust patterns in the data that suggest potential explanatory relationships. |
| 4 | Analogical Modelling | Using analogies to form initial hypotheses that can explain the identified data patterns. |
| 5 | Theory Construction | Developing theoretical models that are consistent with data patterns and the hypotheses formed through analogical reasoning. |
| 6 | Initial Exploratory Data Analysis | Preliminary examination of the data to uncover any obvious patterns or relationships without formal hypotheses testing. |
| 7 | Core Data Analysis | In-depth analysis using statistical and methodological tools to investigate the robustness of the data patterns and their alignment with the generated hypotheses. |
| 8 | Close Replication | Attempting to replicate the findings within the same context to ensure the reliability of the data patterns and explanatory models. |
| 9 | Constructive Replication | Replicating the findings in different contexts or with different methods to test the generalisability of the explanatory models. |
| 10 | Theory Generation | Formulating a coherent theory based on the replicated data patterns and initial models. |
| 11 | Inference to the Best Explanation | Evaluating and refining the generated theories by determining which explanation best fits the data. |
| 12 | Theory Development | Further refining and developing the theoretical models to enhance their explanatory power and scope. |
| 13 | Theory Appraisal | Critical assessment of the theories through empirical validation, peer review, and consistency checks. |
| 14 | Explanatory Coherence Assessment | Assessing how the developed theories integrate with existing knowledge and their coherence as explanations for the phenomena in question. |
1.5. Theoretical Framework: Knowledge Capital Theory
Knowledge Capital Theory (KCT), primarily developed by Hanushek and Woessmann (2015a, 2015b), narrows the concept of human capital by emphasising the role of cognitive skills and educational quality in driving economic growth. KCT posits that the knowledge and skills acquired through education are crucial determinants of both individual and national economic success.
The core proposition of KCT can be expressed through the following growth model shown by Eq. 1 (Hanushek & Woessmann, 2015b):
$$\:g={\beta\:}_{0}+{\beta\:}_{1}H+{\beta\:}_{2}X+\epsilon\:$$
1
Where \(\:g\) represents the rate of economic growth, \(\:H\) is the measure of knowledge capital (typically cognitive skills), \(\:X\) is a vector of other relevant factors, and ε is the error term.
This theory is supported by empirical analyses using international student assessment data. KCT research has revealed a strong correlation between cognitive skills and economic growth rates across countries, emphasising the primacy of educational quality over quantity in explaining economic outcomes (Hanushek & Woessmann, 2022). Furthermore, the theory underscores the importance of both basic and advanced skills in fostering innovation and adaptation to technological change (Hanushek & Woessmann, 2015a).
Hanushek and Woessmann (2015a) estimate that an increase of one standard deviation in a country's PISA scores is associated with a 2-percentage point increase in annual GDP per capita growth rates. This relationship is demonstrated in their analyses of cross-country data, suggesting significant economic returns to improved cognitive skills.
The policy implications derived from KCT include a focus on improving educational quality rather than merely increasing years of schooling, emphasising the development of cognitive skills, particularly in mathematics and science, and implementing educational reforms aimed at enhancing teacher quality and school accountability (Hanushek & Woessmann, 2015b).
However, KCT has faced criticism for its reliance on standardised test scores as proxies for cognitive skills and its assumption of a universal relationship between these skills and economic growth across diverse cultural and economic contexts. Critics such as Komatsu and Rappleye (2017, 2019) have questioned the validity of about the relationship between test scores and economic growth, particularly in East Asian contexts. They note that the correlation between test scores can vary significantly depending on the period and countries included in the analysis.
Despite these criticisms, KCT remains influential in shaping educational policy discourse globally, particularly in relation to international large-scale assessments like PISA. The theory's claims about the economic returns to cognitive skills have been used to justify educational reforms and investments in many countries (Hanushek & Woessmann, 2022).
The variability in findings highlighted by Komatsu and Rappleye (2017, 2019) underscores the need for more varied and expansive data that help provide nuanced understanding of the relationship between education and economic growth, which this study aims to address using the Rosetta Stone and HLO datasets. In the following sections, we describe the data sets and provide the context needed for their application to this study.
1.6 The Rosetta Stone Project dataset
1.6.1 Background and Objectives
The Rosetta Stone Project has emerged as an innovative approach to address the challenges of harmonising learning data across diverse assessments. Led by the UNESCO Institute for Statistics (UIS) and the International Association for the Evaluation of Educational Achievement (IEA), this project aims to link existing regional assessments to global metrics (Sandoval-Hernández, 2022). The primary objective is to enable comparisons of education systems across different regions and contexts, facilitating cross-national learning and knowledge-sharing.
The project's development is rooted in the context of Sustainable Development Goal 4 (SDG 4), which emphasises the importance of learning outcomes in ensuring inclusive and equitable quality education for all. However, producing globally comparable learning metrics has been a challenging endeavour, fraught with technical and political difficulties (Fontdevila, 2023). The UIS, tasked with leading the development of SDG 4 indicators, has faced numerous obstacles in navigating the complex landscape of learning assessments and harmonising data from diverse sources.
UNESCO has had a long but irregular history of producing learning data, with limited success in past harmonisation efforts (Addey, 2019; Cussó & D'Amico, 2005). The establishment of the Learning Metrics Task Force (LMTF) in 2012 revitalised the debate on harmonising learning data, but concerns about the perceived dominance of certain institutions persist (Fontdevila, 2023).
1.6.2 Methodology and Data Collection
The Rosetta Stone Project employs a complex methodology to establish concordance between assessments, addressing challenges posed by their distinct sampling and test designs. The study focuses on linking regional assessments such as the Regional Comparative and Explanatory Study (ERCE) in Latin America and the Programme for the Analysis of Education Systems (PASEC) in francophone Africa to the global metrics of the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS).
The methodology involves administering ERCE and PASEC instruments along with TIMSS and PIRLS linking items to the same students in five countries (two for ERCE, three for PASEC). Using item response theory (IRT) models and plausible values (PVs), the study constructs comparable scales for each assessment and examines the relationship between the ERCE/PASEC and TIMSS/PIRLS constructs. Concordance tables are then established by identifying score levels on ERCE/PASEC and estimating the conditional distribution of TIMSS/PIRLS scores at each level, accounting for measurement error (Sandoval-Hernández, 2022).
This approach aligns with the UIS's broader strategy of adopting a pragmatic, multi-pronged approach to accommodate different data sources and harmonisation methods. The UIS has emphasised country ownership and fit-for-purpose data (Crouch & Bernard, 2017; Montoya, 2017), while also advocating for the creation of a global bank of test items to enable low- and middle-income countries to generate assessment data at a comparatively low cost (UNESCO Institute for Statistics (UIS), 2019).
1.6.3 Criticisms and Limitations
While the Rosetta Stone methodology aligns with established practices in large-scale assessment and enables robust linking of different assessments, it also faces several challenges and limitations. Different assessments may measure somewhat different constructs, limiting the extent to which scores can be equated. The concordance tables estimate the distribution of scores on one assessment conditional on scores on another but do not provide one-to-one score conversions.
There are also significant data requirements and psychometric complexities involved in establishing reliable concordances (Sandoval-Hernandez, 2022). These challenges reflect broader issues in the field of global learning metrics, including the collective nature of indicator production and the organisational effects of indicator production (Davis et al., 2012; Espeland & Stevens, 2008).
Despite these limitations, the Rosetta Stone concordance tables represent an important step forward in harmonising data from regional assessments to enable international comparisons and progress monitoring. As regional programs refine their performance benchmarks and more countries participate in the linking studies, the global comparability of the resulting proficiency estimates is expected to improve. The tables provide a starting point for bridging different assessment scales and leveraging their collective insights to inform educational policy and practice (Sandoval-Hernández, 2022).
However, it is important to note that the UIS's position in this field remains fragile due to several factors, including limited in-house expertise, an uncertain rapport with UNESCO, and economic challenges (UNESCO, 2018). The World Bank's launch of the Human Capital Index and Learning Poverty Indicator could potentially impact the UIS's centrality in the learning measurement field (Altinok et al., 2018; World Bank, 2019). Additionally, the UIS's reliance on partnerships with various organisations may pose risks in terms of sustainability and legitimacy (Fontdevilia, 2021). The next section seeks to present this new data set from the World Bank.
1.7 Harmonized Learning Outcomes (HLO) Dataset
1.7.1 Development and Purpose
The Harmonized Learning Outcomes (HLO) database emerged, like Rosetta Stone, as a response to the critical need for globally comparable measures of learning outcomes. Developed through a large-scale initiative by the World Bank, this database addresses a significant gap in understanding human capital formation, particularly in developing countries (World Bank, 2019). The primary purpose of the HLO database is to provide a comprehensive and comparable measure of education quality across a wide range of countries, moving beyond traditional measures that relied solely on years of schooling.
Historically, measures of human capital were hampered, by being primarily focused on quantitative aspects such as years of schooling (Lutz & KC, 2011; Mankiw et al., 1992; Mincer, 1958). However, this approach assumes that time spent in school is automatically translated into learning, an assumption that has been challenged by empirical evidence (Pritchett, 2013). The HLO database addresses this limitation by providing a direct measure of learning outcomes, thereby offering a more accurate representation of human capital.
Angrist et al. (2021) highlight the ongoing relevance and utility of the database for researchers and policymakers, noting that it is publicly available and updated regularly as new learning data become available. This commitment to regular updates ensures that the HLO database remains a valuable resource for understanding global trends in education quality.
1.7.2 Data Sources and Harmonization Process
The HLO database incorporates data from seven assessment regimes: three international tests, three regional standardised achievement tests, and the Early Grade Reading Assessment. This comprehensive approach allows for the inclusion of 164 countries, covering 98% of the global population, with developing economies comprising two-thirds of the included countries (Angrist et al., 2021).
The methodology underpinning the HLO database is innovative and robust. It exploits the expansion of international assessments to construct globally comparable learning outcomes, linking international assessments to their regional counterparts. Angrist et al. (2021) describe the full process of converting regional test scores to international test scores within subjects and schooling levels, and within adjacent years. This approach enables the inclusion of many developing countries often excluded from international comparisons (Hanushek & Kimko, 2000; Hanushek & Woessmann, 2012b).
The harmonisation process involves converting regional test scores to international test scores within subjects and schooling levels, minimising the likelihood that test differences are a function of time, proficiency, schooling level, or data availability. Angrist et al. (2021) emphasise that this method, which includes tests across the same testing round and at the disaggregated schooling and subject level, helps to ensure that the differences observed are primarily due to actual differences in learning outcomes rather than methodological artefacts.
1.7.3 Strengths and Weaknesses
The HLO database boasts several significant strengths. Firstly, its comprehensive coverage provides comparable learning outcomes for 164 countries from 2000 to 2017, making it one of the largest and most current global learning databases. Secondly, the data is highly detailed, including mean scores and standard errors for each measure, disaggregated by schooling level (primary and secondary), subject (reading, mathematics, and science), and gender (male and female). This level of detail allows for a nuanced analysis of learning outcomes across different demographics and educational stages.
Another key strength is its focus on developing countries. By linking international assessments to their regional counterparts, it includes many nations often excluded from international comparisons. Angrist et al. (2021) highlight that this database is one of the first to disaggregate learning results by gender and to introduce methodological improvements such as the inclusion of standard errors to quantify uncertainty around mean scores.
The database also provides a measure of education quality, not just quantity. This is crucial as evidence suggests that when human capital is measured by learning, it is more strongly associated with growth than when measured by schooling alone (Hanushek & Woessmann, 2012a; Krueger, 2003; Pritchett, 2013).
However, the HLO database is not without limitations. One potential weakness is the reliance on standardised tests as a measure of learning outcomes. While these tests provide a comparable measure across countries, they may not capture all aspects of learning or educational quality. Additionally, the process of harmonising scores across different assessments may introduce some level of uncertainty or error, although the methodology employed aims to minimise these issues.
Despite these potential weaknesses, the HLO database represents a significant advancement in measuring human capital globally. Angrist et al. (2021) argue that their approach extends existing methods by linking aggregate country results from separate assessments while relying on the underlying psychometric structure and microdata.