A new multidimensional graduate work readiness scale: An exploratory study of its factor structure and reliability

doi:10.21203/rs.3.rs-4886175/v1

Download PDF

Research Article

A new multidimensional graduate work readiness scale: An exploratory study of its factor structure and reliability

https://doi.org/10.21203/rs.3.rs-4886175/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Graduate work readiness (GWR) has become increasingly important as countries capitalise on skilled workers’ contribution to economic growth. The GWR discourse has matured; however, the literature highlights a need for more conceptual clarity to develop skills and sound measurement tools. This study aimed to devise a new GWR scale to address this discrepancy. It defines a theoretical model as a foundation for developing a GWR scale and uses an exploratory sequential mixed methods design to investigate its underlying factor structure. Phase 1 drew from extant literature to establish a coherent theoretical model with comprehensive definitions. Phase 2, the scale development, started with an extensive item pool development, which labour market stakeholders (n = 17) reviewed. A Bayesian Exploratory Factor Analysis with observations (n = 101) from an employer survey revealed four underlying latent constructs: cognitive, metacognition, interpersonal, and intrapersonal skills. Reliability tests indicate good instrument internal consistency. This paper provides initial evidence of a multidimensional GWR scale and its psychometric properties. For reproducibility and generalisability, further studies can determine the validity of this hypothetical factor structure and investigate the relationships between factors.

Graduate work readiness

scale development

measurement tool

factor structure

theoretical model

theoretical framework

Graduates' transition into the labour market is often challenging for countries as skill mismatches can weigh heavily on their economic growth and development (Lau et al., 2020). With policies emphasising universities’ role in student skill development, producing employable graduates has become a ‘performance indicator’ for universities (Winterton & Turner, 2019). Graduate work readiness (GWR) is an expected tangible outcome from higher education used by employers to assess graduates exiting university (Hoque et al., 2023). GWR skills’ range and complexity have increased across jobs, industries and organisations to become “multifaceted and holistic” (Enstroem & Schmaltz, 2024, p. 4). This has resulted in increased difficulties in measuring GWR, a critical performance criterion for all stakeholders (policymakers, universities, employers) who are either “producers” or “consumers” in this macroenvironment.

GWR assessment is in a developmental phase. First, while there are several ways to assess GWR, only one, the scale developed by (Caballero et al., 2011), has been extensively explored. Second, a recent review indicated that GWR lacks conceptual clarity (Orr et al., 2023). The absence of a proper definition makes assessment tools’ reliability and validity challenging. Third, literature on theoretical frameworks used for GWR scale development is scarce. This is problematic as explicit theoretical framing improves the robustness of assessment tools (Schwab, 1980; Hinkin, 1995).

Our aim in this study is to address these challenges in assessing GWR. Universities need relevant tools to achieve their goals for ‘work-ready’ graduates, without which skill acquisition remains an inconceivable expectation. We present a new assessment tool that is more robust, reliable and valid than currently available for measuring GWR. Drawing on Peersia et al. (2024) and using a deductive approach, we develop an explicit theoretical model to inform a GWR scale for factor structure determination. Thus, this paper defines graduates as individuals who have successfully completed an undergraduate programme in the last two years. It presents a new, valid, reliable scale established on a theoretical model with a defined hierarchical structure. We anticipate that this theoretical model will serve as a foundation for new skill measurement tools in other contexts and spark new ideas about integrating skill development within universities’ curriculum.

Human capital theory underscores that highly knowledgeable and skilled individuals contribute to a country’s economic growth through their input into innovation, social well-being, equality, and productivity (Robeyns, 2006). Employers highly value these skills as they equip employees to effectively perform and navigate work-related problems (Fajaryati et al., 2020). Therefore, skills are an essential employment criterion that shapes the labour market. However, from employers' perspectives, there has been constant debate over the last thirty years about graduates being ill-equipped and not meeting recruitment standards (Winterton & Turner, 2019). This can lead to graduates having difficulty securing jobs after graduation and higher education institutions being blamed for it, thus overlooking that organisational, institutional, and individual factors impact skill development and acquisition.

“Employability is not linear”; personal, sociocultural, economic, and political factors influence it (Nghia et al., 2020, p. 1). Similarly, there are factors influencing GWR. A change in any factor would affect employability or GWR. Consequently, holding universities solely liable for graduates who are not work-ready is unrealistic. Both personal agency and contextual factors affect skill development (Nghia et al., 2020). An individual’s skill learning is shaped by their mental processes and learning attitudes, behaviours and physical skills, creating opportunities for cognitive, intrapersonal and interpersonal competence development (Hoque et al., 2023). Similarly, skill acquisition is also influenced by factors beyond their control, such as teaching strategies, contexts and evaluation methods (Dunne et al., 2000).

Evaluating learning and an individual’s preparedness to work requires reliable and valid assessment methods and tools (Grummon, 1997). The labour market's complex requirements and overlapping definitions of skills make GWR measurement challenging (Curtis, 2004). To meet this need, Caballero and colleagues (2011) developed a GWR scale comprising four factors: work competence, social intelligence, personal characteristics, and organisational acumen. They used an inductive approach and qualitative analysis of employer perspectives to develop a measurement tool that cuts across disciplines. The researchers argued that building the scale with broad constructs “rather than dissecting” it into individual behaviours presents a clearer overview of GWR for “more effective and valid measurement” (Caballero et al., 2011, p. 52).

While the scale by Caballero et al. (2011) is the most widely used in literature, we agree with (Shaw, 2017) that the lack of an explicit theoretical framework is a shortcoming. Its four broad constructs provide an overarching framework, but a detailed taxonomic definition and conceptualisation would bring greater clarity and leeway for improved methods of quantifying GWR behaviours. There is a compelling need for a solid conceptual framework established with GWR definition and component identification (Orr et al., 2023).

We contribute to the field by being the first to employ a deductive approach for a better conceptual understanding of GWR. We developed the measurement tool in two phases to achieve construct clarity and reliability. Phase One draws from Peersia et al.’s (2024) GWR framework to develop a theoretical model, which is then used to inform scale development in Phase 2.

We agree with Bedwell et al. (2014) that it is imperative to comprehensively understand the concept’s composition and structure for better integration into the curriculum and effective learning using robust assessment tools. Considering the scanty research about the concept’s theories and frameworks (Peersia et al., 2024), this paper defines GWR and its underlying constructs prior to item identification in scale development.

Method

There has yet to be a consensus on the definition of GWR and its structure (Syed et al., 2021). We drew from the framework Peersia et al. (2024) developed to highlight the significance of a conceptual definition in model development with three key elements: definition, structure and influencing factors (Schwab, 1980; Tay & Jebb, 2017). This deductive approach establishes the GWR taxonomy, constructs, and dimensions (Hinkin, 1995). The first step was examining the literature to create a classification schema and skill hierarchy and construct articulation and definition (Peersia et al., 2024) to delimit what each reflects and measures. The result is a comprehensive set of definitions (Shaw, 2017) for constructing a coherent GWR structure.

Results

The concept was examined from a demand approach. The definition looks at the system holistically, considering GWR evolves within a macroenvironment rather than in isolation. A classification system was sought, and a schematic diagram was laid to guide the scale development (Schwab, 1980).

Definition

The definition draws from Human Capital Theory, which states that as education level increases, skills increase, followed by increases in productivity, which is rewarded through higher earnings (Robeyns, 2006). We aimed for a definition that establishes boundaries and distinguishes GWR from other overlapping terms, such as employability and career readiness. We define GWR as a set of skills constituting selection measures indicating an individual's ability to successfully transition into the workplace to contribute to economic and organisational growth and their social and personal well-being.

Theoretical model

In line with procedures described by deVellis (2017) and Tay & Jebb (2017), we examined the extant literature to define each construct, a crucial step towards identifying measurement items. We did not use the scale by Caballero et al. (2011) for three reasons: 1) each factor is explained by a skill description rather than a definition; 2) the factors and dimensions are ambiguous (see Fig. 1), with some dimensions sometimes repeating themselves, e.g. self-direction appears under personal characteristics and organisational acumen; 3) the boundaries between measured skills lack clarity. For example, it needs to be clarified how personal skills differ from personal characteristics or how professionalism differs from work ethics. GWR is no longer in its infancy and has evolved within a dynamic labour market environment, so a scale review is needed. Constructing a classification and delineation for clarity would have been feasible by defining the factors. This paper overcomes the lack of classification and definition with Peersia et al.’s (2024) theoretical framework to develop its theoretical model.

Figure 1

GWR domains and components from Caballero et al.(2011)

This paper builds on a recent macro conceptual understanding of GWR by Peersia et al. (2024). Individuals are expected to demonstrate a range of cognitive and intrapersonal abilities (Scheerens et al., 2020) and interpersonal skills (Hilton & Pellegrino, 2013). Drawing its primary foundation from Hilton and Pellegrino (2013), this paper organises GWR domains as human behaviour. Peersia et al.’s (2024) framework comprises three broad domains representing distinct human thinking facets: cognitive, interpersonal and intrapersonal. Hilton and Pellegrino’s (2013) subdomains characterising each broad domain are not used due to taxonomic inconsistency. They are sometimes described as a skill or illustrate a behaviour rather than related skill groups. Thus, it is challenging to provide construct definitions and explain how their interactions reflect that construct. Instead, Lamb et al.’s (2017) Key Skills for the 21st Century are selected as subdomains (see Fig. 2) in response to Hilton and Pellegrino’s (2013) recommendation for better discernment of skills level-wise and understanding of their clustering. Further, Lamb et al.’s framework (2017) is more aligned with the changing workplace needs. Next, the three domains are articulated.

Figure 2

GWR-Theoretical model

Hilton and Pellegrino (2013) built their skill map to “identify and organise human behaviour” (p.21). They drew the cognitive and intrapersonal domains from Bloom’s taxonomy and the interpersonal domain from Bedwell et al. (2014). The cognitive domain refers to remembering, understanding, applying, analysing, evaluating and creating (Krathwohl & Anderson, 2010). A strong relationship exists between cognitive skills and individual earnings, social income distribution, and economic growth (Hanushek & Woessmann, 2008). The intrapersonal domain is an individual’s nature that influences how an individual reacts and navigates in different work situations. It involves attitudes, behaviours and physical skills that individuals can learn (Krathwohl & Anderson, 2010). Intrapersonal skills directly affect an individual’s effectiveness by stimulating workplace synergism and innovativeness (Spreitzer, 1995). Interpersonal skills are essential for the workplace (Bedwell et al., 2014) as they create agency and induce results through relational means (Hayes et al., 2022). Klein and colleagues (2006) defined interpersonal skills as an “umbrella” term for “goal-directed behaviours” with interpersonal communication and relationship-building as dimensions. Bedwell et al. (2014) stated that attitudinal and cognitive processes drive these interpersonal competencies. Individual differences influence effective interpersonal behaviour, suggesting that bridging any skill gap requires maximising their development and learning among undergraduate students before they enter the workforce (Bedwell et al., 2014).

Our deductive approach has resulted in a domain-specific and structured GWR scale with each domain and subdomain defined. A two-level model is organised as shown in Fig. 2 (onwards, the term first- and second-order level/ construct will be used). These domains are subdivided into nine subdomains reflecting the 21st-century skill needs (see Appendix 1 for definitions). The GWR skills classification and taxonomy have given rise to a ‘definitional framework’ for item development (Schwab, 1980), the third-order skills development.

Assessing learning processes' effectiveness in developing GWR skills requires sound measurement tools. We used an exploratory sequential mixed methods design to investigate the latent construct’s underlying factor structure. The process involved: 1) item development, 2) stakeholder review, 3) scale administration, 4) factor analysis, and 5) reliability testing, as recommended by (Kyriazos & Stalikas, 2018). For scale development, we define broad domains as first-order constructs and domains as second-order constructs. The items are dimensions or third-order skills expressed as statements. We tested the theoretical model in Mauritius, a small island state aspiring to become a knowledge economy. The first author has in-depth knowledge of and familiarity with this context, ensuring robust data collection, analysis and interpretation. Moreover, model testing in Mauritius allowed us to assess the theoretical model adaptability of the, which was informed by Australian frameworks.

Method

Item extraction.

The second-order construct definition guided the extraction of measurement items from the literature (Furr, 2011) or writing new ones. This step required discussion among the researchers before discarding or revising them (Furr, 2011). The item selection criteria included conceptual relevance, language simplicity, clarity, reading level, grammar, conciseness, objectivity, and singular focus (Worthington & Whittaker, 2006; DeVellis, 2017). The number of items for the second-order construct varied as some had more facets to measure (Furr, 2011). Item identification proceeded with the expectation that it would yield a theoretically stable factor set (Worthington & Whittaker, 2006). The item development identified the third-order skills to capture the breadth and depth of the second-order construct constituting the first-order construct. All items were positively worded. As the most crucial stage in developing sound measures, item development was iterative (Hinkin, 1995).

Scale and item refinement

The scale items were reviewed for content validity. The process involved evaluating items’ relevance in measuring the construct, contextual relevance and need for new item inclusion (Worthington & Whittaker, 2006). Seventeen participants from recruitment and employment agencies, the research and innovation, accounting, information technology and engineering field, administrators, academics and policymakers from the higher education sector reviewed the second-order constructs and corresponding items. Participants chose either a focus group discussion or an individual face-to-face or online interview. There were three focus group discussions of two to five participants and eight individual interviews. The sessions were audio recorded, and de-identified transcripts were analysed using NVivo.

Factor determination

The items were used to develop a questionnaire for an employer survey to provide empirical evidence of scale alignment with the theoretical model. The survey was sent to 6690 participants, from both public and private, via email addresses sourced from official employer databases provided by relevant authorities in Mauritius. Participation was voluntary and non-rewarded. A minimum sample size of 101 was required with a set effect size (f²) of 0.2, power at 0.80, and a significance level of 0.05, using the G*Power statistical program.

Data collection. Over three months, participants from establishments employing graduates could access the survey hosted on the Qualtrics platform after providing their consent. The questionnaire comprised three parts: (a) a GWR scale, (b) an open-ended question about any other relevant GWR skill and (c) demographic items with a five-point Likert scale to measure each item from strongly disagree (1) to strongly agree (5). The scaling technique is more reliable, valid, and easy to use (Spooren et al., 2007). There were no reverse scoring items as it may introduce systematic error to a scale (Jackson et al., 1993). The demographic information encompassed establishment size by number of employees, field of operation, and number of graduates employed. All statistical analyses were conducted using R version 4.3.1.

Analysis. Data collected was used to examine the item's relevancy to the scale and explore the new scale’s factor structure. A sample size of 100–200 is considered reasonable for examining psychometric properties (Tay & Jebb, 2017), with 200 being large and 50 being the absolute minimum (Gorsuch, 1983). Given its small sample size, this study used a Bayesian Exploratory factor analysis (BEFA) to identify latent constructs (van de Schoot & Miocević, 2020; Schmid, 2022) instead of conventional factor extraction methods. The BEFA represents a coherent, straightforward, theoretically based and true alternative for factor analysis by taking away arbitrary choices which take place during the frequent data analysis process (Conti et al., 2014; Dux Speltz et al., 2023). The BEFA used the R package “BayesFM” and guidelines from Conti et al. (2014) and Dux Speltz et al. (2023). The underlying factors and their structure were extracted following recommended practices in MCMC modelling and results reporting (van de Schoot & Miocević, 2020; Schmid, 2022).

Result

Item development

The minimum psychometric requirement to ensure measurement adequacy is content validity, built-in through item development (Hinkin, 1995). This ensures that each first-order construct, for example, cognitive skills, is adequately measured by the corresponding second-order construct ‘critical thinking skills’, which is itself measured by third-order skills reflected by items. The second-order constructs form composite measures to reflect cognitive skills (Hinkin, 1995). Building from a theoretical foundation allowed consideration for item inclusion or exclusion (Worthington & Whittaker, 2006) to avoid poor scale validity and ambiguous meaning (Furr, 2011). The outcome was a set of observable items reflecting expected behaviours lined up with their corresponding second-order and first-order construct. This process generated sixty-seven items. According to deVellis (2017), the item generation phase is considered completed after experts review it. The next stage involved seeking the labour market stakeholders’ views on the proposed constructs and items.

Construct and item refinement

The labour market stakeholders review provided empirical evidence to refine and contextualise the theoretical model (Hinkin, 1995). The review process generated seventy-one items spanned over a 13-factor scale (see Fig. 3). The findings are as follows: (1) the term ‘innovation’ was added to ‘creativity’ although the researcher explained that creativity encompasses the ability to propose innovative ideas; (2) some items were critical but did not fit their allocated constructs, resulting in item reshuffling to four new constructs to underscore their importance: planning and organisational skills, digital proficiency, leadership and communication. Some of the new constructs had fewer items, and as item reduction was expected with factor analysis, four new items were added to ensure at least three to four items in each second-order construct. This phase produced thirteen constructs and seventy-one items (see Fig. 3).

Figure 3

GWR scale -Reviewed theoretical model

Establishing the factor structure

This phase explored the empirical structure alignment with the theoretical one by identifying common factors in employers’ perceptions using BEFA.

Item responses. The response rate (Appendix 2) for each item (n = 101) was computed. Next, the factor analysis established the underlying factor structure that would be meaningful for understanding the required skill characteristics.

Demographics. The minimum sample size requirement was met. After data cleaning, out of 156 participants' consent, 101 employer responses were usable. Incomplete responses were deleted. Response to scale items was not forced, leading to one missing response. Data imputation was performed using the median value (Zhang, 2016). Figure 4 summarises the participant characteristics. Most respondents (78.35%) were private employers, with 78.39% being large establishments, primarily private ones and employing fewer than ten graduates. Participants spread across different industries, including agriculture, manufacturing, transportation, construction, IT, professional services, administration, and human resources.

Factor structure. The 71 scale items were suitable for factor analysis. The overall Kaiser-Meyer Olkin index of sampling adequacy (0.81) indicated that the sample was factorable. Figure 5 illustrates that, in general, most items show a positive correlation; therefore, the variables were appropriate for factor analysis. These interrelationships were examined using BEFA to extract the underlying factors.

Table 1

*Summary of Posterior Results of BEFA*
	Overall	Cognitive	Metacognition	Interpersonal	Intrapersonal
Maximum number of factors (Kmax)	4	2	2	3	3
Identification restriction (Nid)	10	3	3	5	4
MCMC iterations	50000	50000	200000	50000	50000
Burn-in period	5000	5000	5000	5000	5000
Metropolis-Hastings Acceptance rate	1	1	1	1	0.78
Posterior frequency of number of factors	K = 4 (100%)	K = 2 100%	K = 2 100%	K = 3 (100%)	K = 3 (93.14%)

After examining the dataset, the researchers retained four factors, as the aim was first to identify the first-order constructs. No item loaded on more than one factor. The four factors were named cognitive, metacognition, intrapersonal and interpersonal skills. Table 2–6 shows the item factor loading for each factor and sub-factors. Items with factor loading less than 0.5 were removed from the scale. This led to the elimination of four digital proficiency items: DP1, DP2, DP3 and DP4.

Given the likelihood of subfactors being present, the item groupings were examined to identify them. The extraction generated two factors for cognitive and metacognition and three factors for interpersonal and intrapersonal skills. Some items loaded poorly on the subfactors; however, at this stage, the researchers refrained from discarding any item without prior knowledge of each scale's reliability. The posterior correlation matrix indicated that the factors were moderately associated (Table 7)

Table 2

*Posterior means of factor loadings for a four-factor model*
Item	Cognitive	Metacognition	Interpersonal	Item	Metacognition	Interpersonal	Intrapersonal
CT1	0.60 [0.43,0.78]			LD1		0.62 [0.45, 0.79]
CT2	0.73 [0.57,0.90]			LD2		0.69 [0.53, 0.86]
CT3	0.71 [0.55, 0.88]			LD3		0.72 [0.55, 0.88]
CT4	0.63 [0.46, 0.81]			LD4		0.71 [0.56, 0.88]
CT5	0.70 [0.53, 0.86]			LD5		0.76 [0.61, 0.92]
CI1	0.81 [0.66, 0.96]			CM1		0.66 [0.50, 0.83]
CI2	0.75 [0.59, 0.91]			CM2		0.48 [0.30, 0.67]
CI3	0.78 [0.63, 0.94]			CM3		0.63 [0.46, 0.80]
CI4	0.83 [0.68, 0.98]			CM4		0.68 [0.52, 0.85]
CI5	0.76 [0.61, 0.93]			CM5		0.79 [0.64, 0.95]
CI6		0.65 [0.43, 0.86]		CM6		0.53 [0.35, 0.71]
MC1		0.61 [0.44, 0.78]		DP1	0.35 [0.16, 0.54]
MC2		0.63 [0.46, 0.80]		DP2	0.41 [0.23, 0.60]
MC3		0.73 [0.58, 0.89]		DP3	0.37 [0.18, 0.56]
MC4		0.68 [0.52, 0.85]		DP4	0.46 [0.29, 0.65]
MC5		0.68 [0.52, 0.86]		DP5			0.57 [0.39, 0.75]
MC6		0.68 [0.52, 0.86]		MO1			0.74 [0.59, 0.91]
PS1		0.73 [0.58, 0.89]		MO2			0.72 [0.56, 0.89]
PS2		0.74 [0.59, 0.90]		MO3			0.84 [0.70, 1.00]
PS3		0.80 [0.66, 0.96]		MO4			0.87 [0.72, 1.02]
PS4		0.81 [0.66, 0.96]		MO5			0.80 [0.65, 0.96]
PS5		0.78 [0.63, 0.94]		SE1			0.63 [0.46, 0.81]
PO1		0.67 [0.50, 0.84]		SE2		0.68 [0.51, 0.84]
PO2		0.71 [0.55, 0.87]		SE3		0.72 [0.56, 0.89]
PO3		0.76 [0.60, 0.91]		SE4		0.66 [0.49, 0.83]
PO4		0.68 [0.52, 0.85]		SE5		0.60 [0.43, 0.77]
CO1			0.40 [0.23, 0.61]	SE6		0.68 [0.52, 0.85]
CO2			0.54 [0.36, 0.71]	CS1			0.62 [0.44, 0.79]
CO3			0.66[ 0.49, 0.83]	CS2			0.75 [0.59, 0.91]
CO4			0.62 [0.45, 0.79]	CS3		0.72 [0.56, 0.89]
CO5			0.57 [0.40, 0.74]	CS4		0.58 [0.41, 0.76]
CO6			0.58 [0.41, 0.76]	GR1			0.66 [0.49, 0.83]
CO7			0.54 [0.36, 0.71]	GR2		0.61 [0.44, 0.79]
CO8			0.61 [0.44, 0.79]	GR3			0.68 [0.52, 0.85]
				GR4			0.56 [0.38, 0.74]
				GR5		0.69 [0.53,0.86]
				GR6			0.73 [0.58, 0.90]

Table 3

*Cognitive: Posterior factor loading and 95% High Probability Density (HPD)*
Item	Critical thinking	Innovative thinking
CT1	0.72 [0.55, 0.90]
CT2	0.84 [0.68, 1.01]
CT3	0.83 [0.67, 0.99]
CT4	0.70 [0.52, 0.88]
CT5		0.70 [0.54, 0.88]
CI1		0.83 [0.68, 1.00]
CI2		0.80 [0.64, 0.97]
CI3		0.85 [0.69, 1.01]
CI4		0.86 [0.71, 1.02]
CI5		0.86 [0.70, 1.01]

Table 4

*Metacognition: 2 Posterior factor loading and 95% HPD*
	Sub-factor 2 Loading
Item	Problem-solving	Planning & Organisation
CI6	0.63 [0.46,0.80]
MC1	0.60 [0.43,0.78]
MC2	0.64 [0.47, 0.78]
MC3	0.74 [0.59, 0.91]
MC4	0.70 [0.54,0.86]
MC5	0.69 [0.52, 0.85]
MC6	0.69 [0.53,0.86]
PS1	0.75 [0.59, 0.91]
PS2	0.76 [0.60, 0.92]
PS3	0.80 [0.65, 0.95]
PS4	0.83 [0.68, 0.98]
PS5	0.81 [0.66, 0.97]
PO1		0.77 [0.61, 0.93]
PO2		0.86 [0.71, 1.01]
PO3		0.82 [0.66, 0.97]
PO4		0.76 [0.60,0.92]

Table 5

*Interpersonal: Posterior factor loading and 95% HPD*
Item	Collaboration	Leadership	Self-efficacy
CO1	0.54 [0.35, 0.72]
CO2	0.65 [0.47, 0.82]
CO3		0.64 [0.48, 0.84]
CO4	0.81 [0.65, 0.97]
CO5	0.77 [0.61, 0.94]
CO6	0.80 [0.65, 0.97]
CO7	0.76 [0.60, 0.93]
CO8	0.76 [0.60, 0.93]
LD1		0.73 [0.57, 0.90]
LD2		0.82 [0.67, 0.98]
LD3		0.77 [0.61, 0.93]
LD4		0.82 [0.67, 0.98]
LD5		0.84 [0.70, 1.00]
CM1			0.66 [0.49, 0.83]
CM2			0.48 [0.30, 0.67]
CM3			0.66 [0.50, 0.83]
CM4			0.69 [0.53, 0.86]
CM5			0.79 [0.64, 0.96]
CM6			0.55 [0.37, 0.73]
SE2			0.75 [0.59, 0.91]
SE3			0.80 [0.64, 0.95]
SE4			0.71 [0.55, 0.88]
SE5			0.62 [0.45, 0.79]
SE6			0.72 [0.55, 0.88]
CS3			0.76 [0.60, 0.93]
CS4			0.59 [0.42, 0.77]
GR2			0.59 [0.43, 0.78]
GR5			0.72 [0.55, 0.88]

Table 6

*Intrapersonal: Posterior factor loading and 95% HPD*
Item	Grit	Motivation	Conscientiousness
DP5	0.64 [0.47, 0.83]
MO1	0.77 [0.60, 0.93]
MO2		0.74 [0.58, 0.91]
MO3		0.92 [0.77, 1.06]
MO4		0.92 [0.78, 1.07]
MO5	0.86 [0.71, 1.02]
SE1			0.86 [0.71, 1.02]
CS1			0.66 [0.49, 0.84]
CS2			0.82 [0.66, 0.99]
GR1	0.74 [0.57, 0.91]
GR3			0.75 [0.59, 0.93]
GR4			0.61 [0.43, 0.79]
GR6			0.74, [0.58, 0.92]

Table 7

*Posterior factor correlation matrix*
	Mean	[95% HPD]
F1↔ F2	0.70	[0.61, 0.83]
F1↔ F3	0.56	[0.44, 0.75]
F1↔ F4	0.59	[0.46, 0.73]
F2↔ F3	0.73	[0.64, 0.83]
F2↔ F4	0.59	[0.47, 0.76]
F3↔ F4	0.64	[0.54, 0.79]
F1.1↔ F1.2	0.77	[0.67, 0.87]
F2.1↔ F2.2	0.75	[0.65, 0.85]
F3.1↔ F3.2	0.58	[0.45, 0.74]
F3.1↔ F3.3	0.57	[0.43, 0.72]
F3.2↔ F3.3	0.73	[0.62, 0.83]
F4.1↔ F4.2	0.81	[0.73, 0.91]
F4.1↔ F4.3	0.74	[0.32, 0.95]
F4.2↔ F4.3	0.73	[0.33, 0.93]

The result indicates that literature and empirical data played an essential role in laying a sound theoretical foundation, developing a systematic approach to derive a skill classification system, and establishing a set of interrelated latent constructs and their corresponding measurement instrument in the form of work behaviours (Peersia et al., 2024). For details of items dropped and retained, see Appendix 3. The findings of this study identified four first-order constructs as the GWR scale backbone, cascading into ten second-order constructs (scales) and 67 items (Fig. 6). In line with the theoretical construct from literature, GWR skills can be grouped under three broad constructs, namely cognitive, interpersonal and intrapersonal, and the study identified a fourth one, ‘metacognition skills, ' promoted from cognitive skills to a first-order construct. The ten second-order constructs from literature, critical thinking, problem-solving, motivation, self-efficacy, conscientiousness, grit and collaboration, were retained. The second-order construct ‘creativity and innovation’ (renamed ‘innovative thinking’), ‘planning’ and ‘leadership skills’ generated from the stakeholders’ review were also kept. Self-efficacy was grouped under interpersonal skills instead of intrapersonal skills. The reason might be that social self-efficacy is of interest in the workplace context, that an individual trusts their ability to make personal decisions when participating in a collaborative task at the workplace (Smith & Betz, 2000).

Figure 6

GWR scale hypothetical structure

Internal reliability. This study performed a Bayesian reliability analysis using Package Bayesrel version 0.7.7. The analysis assessed the overall scale’s internal consistency, reported as the posterior mean, with Bayes Cronbach’s α and Bayes MacDonald’s ω.

The reliability test results for the overall scale are shown in Table 8. Cronbach’s α and McDonald’s ω show internal consistencies of data as a scale. The findings indicated that Bayes Cronbach’s α coefficients for the four extracted factors range from 0.93 to 0.95, indicating a high reliability. Similarly, Bayes ω_t indicated good internal consistency with figures ranging from 0.95 to 0.96, and Bayes ω_h total was slightly lower, ranging from 0.85 to 0.89. The prior and posterior probability of those reliability statistics is given in Table 9.

Table 8

*Bayesian reliability statistics*
			95% Confidence interval
Factor	Bayes Reliability statistics	Posterior mean	lower	upper
Cognitive	alpha	0.93	0.91	0.95
	omega_t	0.95	0.93	0.93
	omega_h	0.89	0.85	0.93
Metacognition	alpha	0.95	0.94	0.96
	omega_t	0.96	0.95	0.96
	omega_h	0.90	0.86	0.84
Interpersonal	alpha	0.95	0.94	0.97
	omega_t	0.96	0.95	0.97
	omega_h	0.85	0.78	0.91
Intrapersonal	alpha	0.93	0.92	0.95
	omega_t	0.95	0.93	0.96
	omega_h	0.88	0.84	0.92

Table 9

*The probability of Bayesian multidimensional reliability statistic*
		Probability
Factor	Reliability tests	Prior	Posterior
Cognitive	omega_t > 0.8	0.28	1.00
	omega_h > 0.6	0.28	1.00
Metacognition	omega_t > 0.8	0.17	1.00
	omega_h > 0.6	0.15	1.00
Interpersonal	omega_t > 0.8	0.19	1.00
	omega_h > 0.6	0.17	1.00
Intrapersonal	omega_t > 0.8	0.24	1.00
	omega_h > 0.6	0.21	1.00

While scales of GWR exist, they need a robust theoretical basis to overcome the difficulties of measurement tool development. We addressed this shortcoming by developing a new scale using a deductive approach, thus strengthening the scale with definitions and setting boundaries for every component. This process eliminated subjectivity that might be introduced by generating items using an inductive approach and interpretation when labelling factors in exploratory factor analysis (Morgado et al., 2017).

The GWR framework from Peersia et al. (2024) allows for GWR skills’ organisation and classification, thus laying down the concept’s taxonomic structure. This paper unpacked the first highest taxonomic level (first-order construct) to create a ‘kingdom classification’ describing the second highest taxonomic level (second-order construct) divided into lower-level dimensions (third-order skills). The diagrammatic representation with an overarching structure constitutes constructs that interact and integrate with the real world to produce desired GWR outcomes (Bedwell et al., 2014).

Unlike Caballero et al. (2011), this new scale has not been developed solely from employer perspectives. Instead, it is based on a theoretical model with universal application to establish the factor structure. Like Caballero et al. (2011), our scale shows that GWR is multi-dimensional. In general, our scale content matches recent studies, which indicate that employers are increasingly searching for graduates with second-order skills beyond the foundational, ‘technical proficiency and domain-specific knowledge’ skills (Scheerens et al., 2020). It encompasses skills that empower employees to thrive in various work situations, such as critical thinking, collaboration and communication (Enstroem & Schmaltz, 2024).

Our empirical findings further refined the scale. The focus group discussion with the labour market stakeholders further distilled the second-highest taxonomical level into thirteen second-order constructs measured by seventy-one items. However, the factor analysis suggested a more parsimonious four-factor structure subdivided into subfactors that effectively represent the first-order level skills. The newly generated model resembles the theoretical model except that ‘metacognition’ emerges as a fourth construct. Under Bloom’s taxonomy, metacognitive knowledge falls under the knowledge dimension of the cognitive domain (Krathwohl & Anderson, 2010). This could highlight the increased importance of metacognition in work readiness, and further investigation is needed to find out how it relates to the other first-order level constructs.

Only one item was retained out of five items from the ‘digital proficiency’ construct, which were redistributed among other second-order factors. This suggests that digital skills are more of a foundational literacy subject in an era of extensive technology adoption. This could indicate that essential technical skills should be included as core skills in the curriculum. The sub-construct ‘communication’ was also eliminated as the items were redistributed. Many items under motivation and grit were redistributed. These suggest that the items did not adequately measure the same construct, and since they are newly proposed GWR constructs, more research is needed to refine these constructs and their items.

This study offers more than a measurement tool. It proposes a common language for GWR skills studies and explains graduates' expected achievements to ensure a smooth transition into the workplace. It facilitates communication across individuals, organisations and sectors. The four-factor model might help achieve a consensus about the meaning of GWR and the skills it constitutes for effective performance in the workplace. It can also assist universities in identifying skills to integrate into the skill development curriculum.

This new GWR scale sheds light on the essential features needed for translating employers' skill expectations into skills academics need to focus their teaching and student learning on. Although evidence of instruction and students' learning is essential, assessments are valuable for initiating curricular changes at this stage.

Limitations and future directions

This study provides a hypothetical factor structure drawn from literature and factors derived from an employer dataset. It provides initial evidence about the scale composition. However, future research should examine the scale validation and model fit using confirmatory tests.

To demonstrate the generalisability of the factor structure, stability and reproducibility of scale properties, future research could investigate the factor structure with independent samples from other contexts to cross-validate the sample (Schwab, 1980; Tay & Jebb, 2017). The sample size is a study limitation; studies with larger samples will indicate scale consistency.

The conceptual model merits more research on its structural validity in establishing a nomological network. This will require investigating the hypothetical linkages among the theoretical constructs (Schwab, 1980), as several factors can influence the development of GWR. Future research could also probe the hierarchical structure of the sub-constructs within the broad construct.

Given the evolving nature of the workplace, the scale might need a review in the future to meet the new requirements and developments. New research could focus on item development for constructs that were theoretically identified but did not emerge from the factor analysis. Researchers might consider exploring how the shift in the nature and importance of digital skills as a core skill could be integrated into the university curriculum. In addition, this paper has identified the foundational elements to build a skill teaching matrix that identifies the skills and their proficiency levels and how to teach them across disciplines, as Caballero et al. (2011) suggested, to assist skill development.

GWR skills are crucial to foster. Constantly evolving contemporary work environments, technological advancements, and industry shifts due to volatile, complex and ambiguous economic environments create new challenges towards ensuring GWR. Lack of conceptual clarity about GWR presents an additional hurdle, as it is easier to measure and improve with a good understanding of the concept. The new GWR scale presented in this paper addresses this gap, and its robustness lies in the deductive approach used to derive a conceptual model with a classification map. The theoretical model goes beyond a skill list, descriptions, or statements to comfortably map onto a more elaborated classification comprising clearly defined skill clusters, drawing boundaries and giving a common understanding of GWR skills. The result indicates that a theoretical framework provides a priori knowledge of the skill to be developed and measured. This four-factor model indicates that GWR is the combined effect of cognitive, metacognition, interpersonal and intrapersonal skills. Measuring GWR would imply developing assessment tools that measure those four skill domains. This conceptually and empirically derived scale lays the basis for more robust measurements of GWR behaviours. It is an instrument that has practical value for employers, researchers and higher education institutions to measure and understand graduates’ readiness for the workplace. It can be useful for developing strategies to promote skill development in the workplace and universities.

Bedwell WL, Fiore SM, Salas E (2014) Developing the future workforce: An approach for integrating interpersonal skills into the MBA classroom. Acad Manage Learn Educ 13(2):171–186. https://doi.org/10.5465/amle.2011.0138
Caballero CL, Walker A, Fuller-Tyszkiewicz M (2011) The work readiness scale (WRS): Developing a measure to assess work readiness in college graduates. J Teach Learn Graduate Employab 2(1):41–54
Conti G, Frühwirth-Schnatter S, Heckman JJ, Piatek R (2014) Bayesian exploratory factor analysis. J Econ 183(1):31–57. https://doi.org/10.1016/j.jeconom.2014.06.008
Curtis DD (2004) The assessment of generic skills. In Generic skills in Vocational Education and Training: Research Readings (pp. 136–156). National Centre for Vocational Education Research. https://researchnow.flinders.edu.au/en/publications/the-assessment-of-generic-skills
DeVellis RF (2017) Scale development: theory and applications (Fourth edition). SAGE
Dunne E, Bennett N, Carré C (2000) Skill development in higher education and employment. In F. Coffield (Ed.), Differing Visions of a Learning Society, Vol 1: Research Findings (pp. 105–138). Policy Press. https://doi.org/10.56687/9781847425126-005
Dux Speltz ED, Roeser J, Torrance M, Chukharev-Hudilainen (2023) e. Behavioral stability in writing process measures. https://rstudio-pubs-static.s3.amazonaws.com/993354_e194d8a69a1c4e98ab8fd0a942ec9d94.html
Enstroem R, Schmaltz R (2024) Striking gold: Navigating the education massification maze for work readiness. J Work-Applied Manage Adv online publication. https://doi.org/10.1108/JWAM-10-2023-0100
Fajaryati N, Budiyono, Akhyar M, Wiranto (2020) The employability skills needed to face the demands of work in the future: Systematic literature reviews. Open Eng 10(1):595–603. https://doi.org/doi:10.1515/eng-2020-0072
Furr RM (2011) Scale construction and psychometrics for social and personality psychology. https://doi.org/10.4135/9781446287866
Gorsuch RL (1983) Factor analysis, 2nd edn. Lawrence Erlbaum
Grummon PTH (1997) Assessing students for workplace readiness. Centerfocus, 15
Hanushek EA, Woessmann L (2008) The role of cognitive skills in economic development. J Econ Lit 46(3):607–668. https://doi.org/10.1257/jel.46.3.607
Hayes S, Freudenberg B, Delaney D (2022) Work-ready graduates for Australian small and medium accounting firms. J Teach Learn Graduate Employab 13(1):1–19. https://doi.org/10.21153/jtlge2022vol13no1art1026
Hilton ML, Pellegrino JW (2013) Education for life and work: Developing transferable knowledge and skills in the 21st century (1st ed.). The National Academies Press
Hinkin TR (1995) A review of scale development practices in the study of organizations. J Manag 21(5):967–988. https://doi.org/10.1177/014920639502100509
Hoque N, Uddin M, Ahmad A, Mamun A, Uddin MN, Chowdhury RA, Noman Alam AH M (2023) The desired employability skills and work readiness of graduates: Evidence from the perspective of established and well-known employers of an emerging economy. Ind High Educ 37(5):716–730. https://doi.org/10.1177/09504222221149850
Jackson PR, Wall TD, Martin R, Davids K (1993) New measures of job control, cognitive demand, and production responsibility. J Appl Psychol 78(5):753–762. https://doi.org/https://doi.org/10.1037/0021-9010.78.5.753
Klein C, DeRouin RE, Salas E (2006) Uncovering workplace interpersonal skills: A review, framework, and research agenda. In G. P. Hodgkinson & J. K. Ford (Eds.), International Review of Industrial and Organizational Psychology 2006 (pp. 79–126). Wiley. https://doi.org/10.1002/9780470696378.ch3
Krathwohl DR, Anderson LW (2010) Merlin C. Wittrock and the revision of Bloom's taxonomy. Educational Psychol 45(1):64–65. https://doi.org/10.1080/00461520903433562
Kyriazos TA, Stalikas A (2018) Applied psychometrics: The steps of scale development and standardization process. Psychology 9(11):2531–2560. https://doi.org/10.4236/psych.2018.911145
Lai M (2013) Chapter 6 Markov Chain Monte Carlo. https://bookdown.org/marklhc/notes_bookdown/
Lau PL, Anctil T, Ee GT, Jaafar JLS, Kin TG (2020) Self-concept, attitudes toward career counseling, and work readiness of Malaysian vocational students. Career Dev Q 68(1):18–31. https://doi.org/https://doi.org/10.1002/cdq.12210
Lamb S, Maire Q, Doecke E (2017) Key skills for the 21st century: An evidence-based review. Centre for International Research on Education Systems. https://education.nsw.gov.au/content/dam/main-education/teaching-and-learning/education-for-a-changing-world/media/documents/Key-Skills-for-the-21st-Century-Executive-Summary.pdf
Morgado FFR, Meireles JFF, Neves CM, Amaral ACS, Ferreira MEC (2017) Scale development: Ten main limitations and recommendations to improve future research practices. Psicologia: Reflexão e Crítica 30(1):3. https://doi.org/10.1186/s41155-016-0057-1
Nghia TLH, Pham T, Tomlinson M, Medica K, Thompson CD (2020) The way ahead for the employability agenda in higher education. In: Nghia TLH, Pham T, Tomlinson M, Medica K, Thompson C (eds) Developing and Utilizing Employability Capitals. Routledge
Orr P, Forsyth L, Caballero C, Rosenberg C, Walker A (2023) A systematic review of Australian higher education students' and graduates' work readiness. High Educ Res Dev 42(7):1714–1731. https://doi.org/10.1080/07294360.2023.2192465
Peersia K, Rappa N, Perry LB (2024) Work readiness: Definitions and conceptualisations. High Educ Res Dev. https://doi.org/10.1080/07294360.2024.2366322
Robeyns I (2006) Three models of education: Rights, capabilities and human capital. Theory Res Educ 4(1):69–84
Scheerens J, Werf M, Boer H (2020) Soft skills in education: Putting the evidence in perspective, 1st edn. Springer. https://doi.org/10.1007/978-3-030-54787-5
Schmid D (2022) The technology acceptance scale: Its Bayesian psychometrics assessed in a factor analysis via Markov chain Monte Carlo models. Hum Factors Ergon Manuf Serv Ind 32(1):66–88. https://doi.org/https://doi.org/10.1002/hfm.20896
Schwab DP (1980) Construct validity in organization behavior. In: Staw BM, Cummings LL (eds) Research in Organizational Behavior, vol 2. JAI
Shaw JD (2017) Advantages of starting with theory. Acad Manag J 60(3):819–822. https://doi.org/10.5465/amj.2017.4003
Smith HM, Betz NE (2000) Development and validation of a scale of perceived social self-efficacy. J Career Assess 8(3):283–301. https://doi.org/10.1177/106907270000800306
Spooren P, Mortelmans D, Denekens J (2007) Student evaluation of teaching quality in higher education: development of an instrument based on 10 Likert-scales. Assess Evaluation High Educ 32(6):667–679. https://doi.org/10.1080/02602930601117191
Spreitzer GM (1995) An empirical test of a comprehensive model of intrapersonal empowerment in the workplace. Am J Community Psychol 23(5):601–629. https://doi.org/https://doi.org/10.1007/BF02506984
Syed ASS, Nadarajah VDV, Kwa SK, Seow LL, Chong DWK, Molugulu N, Khoo EJ, Keng PS (2021) Validation of a 'work readiness scale' for health professional (HP) graduates. Medical Teacher, 43, S33-S38. https://doi.org/10.1080/0142159x.2019.1697434
Tay L, Jebb AT (2017) Scale Development. In: Rogelberg S (ed) The SAGE Encyclopedia of Industrial and Organizational Psychology, 2nd edn. Sage, Thousand Oaks, CA
van de Schoot R, Miocević M (2020) Small sample size solution: A guide for applied researchers and practitioners, 1st edn. Taylor & Francis. https://doi.org/10.4324/9780429273872
Winterton J, Turner JJ (2019) Preparing graduates for work readiness: an overview and agenda. Educ Train (London) 61(5):536–551. https://doi.org/https://doi.org/10.1108/ET-03-2019-0044
Worthington RL, Whittaker TA (2006) Scale development research: A content analysis and recommendations for best practices. Couns Psychol 34(6):806–838. https://doi.org/10.1177/0011000006288127
Zhang Z (2016) Missing data imputation: Focusing on single imputation. Annals Translational Med 4(1):9. https://doi.org/10.3978/j.issn.2305-5839.2015.12.38

The authors declare no competing interests.

Appendix1.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

A new multidimensional graduate work readiness scale: An exploratory study of its factor structure and reliability

Status:

Version 1

Abstract

Figures

Introduction

Background

Phase 1: Theoretical model development and design

Method

Results

Definition

Theoretical model

GWR domains and components from Caballero et al.(2011)

GWR-Theoretical model

Phase 2: Scale Development

Method

Scale and item refinement

Factor determination

Result

Item development

Construct and item refinement

GWR scale -Reviewed theoretical model

Establishing the factor structure

GWR scale hypothetical structure

Discussion

Limitations and future directions

Conclusion

References

Additional Declarations

Supplementary Files

Status:

Version 1