The process of the content validation of the Abilitator in terms of content relevance and coverage can be split into five phases: 1) the construct to be measured was defined and the situation of the use of the instrument was specified, 2) expert panels were invited to assess the content of the instrument during the development process, 3) the information about the content of the instrument was considered and provided, 4) the assessment of whether the content of the instrument corresponded with the construct was conducted, 5) the correspondence between the instrument and the ICF framework was assessed and compared to relevant ICF Core Sets and instruments [36]. These phases included the evaluation of face validity, which is the degree to which the measurement instrument seems to be an adequate reflection of the construct being measured [37]. In terms of the content validation of the Abilitator, these phases are described under the following sub-headings.
Specification of measured construct and context of use
First, we reviewed the related theoretical models and the existing self-report instruments of work ability and functioning used in Finland and the EU (Phase 1, Figure 1). We searched Google Scholar and PubMed using search terms such as work ability, functioning, functional capacity, model, method, concept, theoretical, self-report, self-assessment, and questionnaire. We also identified self-report instruments from the Finnish TOIMIA network’s database, which contains guidelines for the measurement of functioning and evaluations of measures. The TOIMIA network of experts aims to create uniform national practices for the measuring and evaluation of functioning and is coordinated by the National Institute of Health and Welfare (THL) [38].
The search yielded eight different theoretical models for work ability [40] and two models for functioning. We identified 55 self-report instruments of work ability and functioning in the review. Fourteen instruments were utilised either partly or fully in Abilitator 0.1 [Additional file 1]. The inclusion criteria for the instruments were their proven reliability, validity and wide use in previous research of the working-age population. Further inclusion criteria were free access, availability in the Finnish language and different occupational groups being able to use the instrument safely.
Concept of work ability and functioning
The concept of work ability varies in research and according to different operational contexts [39]. Work ability can be used to specify the expectations of employees in terms of the competence needed for different kinds of work. The concept can also be used by health professionals in disease prevention and health promotion and as an instrument to determine the degree and type of rehabilitation needed by individuals. In addition, work ability can be used as an instrument to evaluate rehabilitation and for constructing new valid means for measuring work ability. It is also a central legal concept regulating sickness and social insurance policies. [12].
In the bio-medical model of work ability, an existing illness, impediment or disability determines a person’s attributes and qualities as a worker [13, 41, 42]. In the balance model, work ability is the equilibrium between the individual and work-related factors [43, 41]. The psychosocial model emphasises the psychological and psychosocial factors connected to work participation and return to work [44, 45, 46, 47]. In the multi-dimensional models and the bio-psycho-social models, work ability is a holistic, comprehensive unity in which individual resources and work-related factors are combined by the operational environment and social support [20, 48, 13, 49]. In the employability model, work ability combines all the individual and societal actions that help a person become employed, stay employed and advance their career [50, 51]. According to the model emphasising the integration of the individual at the workplace, the concept of work ability is based on continuous change in work and work organisation [52, 53]. Work ability can also be considered a social construct that is constituted by and differs between different societies and systems [54].
Functioning is closely related to health and comprises a psychological, social, physical and cognitive dimension [48, 55]. Psychological functioning is the ability to feel, experience, form perceptions of oneself and the surrounding world, plan life, find solutions, and make decisions [56]. Social functioning is manifested through one’s role as an actor with and among others, interaction with social networks, social activities and participation, as well as experiences of coexistence and inclusion [57]. Inclusion means that a person feels they are a significant part of an entity with others. Inclusion is a process that can be observed through material, spiritual, social, and physical dimensions and can be viewed from a variety of perspectives, such as education or work. [58]. Physical functioning includes the ability to physically perform everyday basic activities and meaningful leisure activities, as well as to work and study [59]. Cognitive functioning is the mental function related to the reception, processing, preservation and use of knowledge [60].
As in the concept of work ability, functioning and health have biomedical and bio-psycho-social models [49]. An internationally accepted way of structuring the concept of functioning is ICF [55]. The ICF framework provides a standard language and multi-purpose classification of disability and health [61]. Functioning is a collective umbrella term of the ICF that describes a person’s body structures and functions and their capacity to perform daily activities in the environment in which they live. Personal factors such as age, sex, cultural and ethnic background, socioeconomic status and social and physical environment contribute substantially to a person’s ability to function [62]. The ICF is a bio-psycho-social model that combines the bio-medical, social and environmental aspects of human functioning, health and disability [48, 63]. It can be used as an instrument to collect comparable data to support evidence-based decision-making in health and health-related sectors.
The ICF framework reflects six different aspects of health and disability: health condition, body structure and body function, activity, participation, environmental factors, and personal factors [64]. Diseases or disorders, i.e. health conditions, are included in the conceptual model of health, but are classified in the International Classification of Diseases and Related Health Problems (ICD) [65]. The ICF and the ICD are complementary; both classifications should be used to describe an individual’s health status. Functioning should be understood as a continuum ranging from completely able (non-problematic) to completely disabled (problematic) and is the result of complex multifactorial interaction between the six components. [66].
Concept of work ability and functioning in the Abilitator
The Abilitator is based on the multidimensional model of work ability [13] as this was considered the best model for describing both individual resources and the operational environment. We chose the bio-psycho-social model [48] for functioning because it is widely accepted in situations of multiple and long-term impairments of health [45].
The selected multi-dimensional work ability model is called the House of Work Ability [13, 67]. It has four levels that depict the relationship between individual resources, work-related demands, and the social and the operational environments that affect both individual resources and the working life. The three lower levels of the model describe individual resources such as health and functioning, competence and work experience, and values, attitudes and motivation. The top level is the level of work and includes factors related to work, working conditions, work community and leadership. Individual work ability is created by the balance between all the levels of the house, which are also significantly affected by social networks, communities and environments outside the workplace [68, 69, 13]. The Abilitator does not cover the top level, because those in a weak labour market position are to a large extent without employment.
The bio-psycho-social model of functioning [55, 49] sees operational constraints as a mismatch between the health of a person and the requirements of their life situation. To minimise this disparity, the impact of environmental and individual factors must also be considered in addition to the person’s health-related factors. These include available support and services, work situation, family, hobbies, motivation, and religion [62, 70].
The construct of the Abilitator can be further described using a framework of four central and partly overlapping concepts that can be linked to the population in a weak labour market position. These concepts are: 1) work ability [13, 67, 12], 2) health and functioning [55], 3) inclusion [58] and 4) employability [20, 51]. They include a variety of factors, some of which are defined in the Abilitator and some not, as shown in Figure 2.
Figure 2. Construct of the Abilitator consisting of concepts of work ability, health and functioning, inclusion and employability. The inner square features the factors of each concept that are defined in the Abilitator and the outer square those that were excluded. The dotted lines reflect the overlapping of the four concepts.
Specification of the Abilitator’s context of use
The Abilitator was developed to be suitable for the individual and multidimensional self-assessment of work ability and functioning of the population in a weak labour market position. The service clients participating in the ESF Priority 5 projects represented the target group. These people were mainly of working age, had been unemployed for several years, and faced various problems with their health, lifestyles and life situations. They participated voluntarily in the projects to improve their work ability, functioning and employment opportunities. The service actors working in the projects reflected the diversity of the different occupational professionals working in health, rehabilitation, social, education and employment services.
The ESF projects used the Abilitator to assess the service clients’ situations individually, to set goals and to design the best service plans to reach the set goals. They also used it to make the changes in work ability and functioning apparent to both clients and service actors. This information was further used to analyse the effect of the different actions on larger groups of service clients taking part in ESF Priority 5 projects.
Utilisation of expert panels in co-development and assessment of content
The Abilitator’s content and use were developed according to the situation of the service clients and the professional needs of the service actors. On the one hand, the digital format made the Abilitator quick and easy to administer at any phase of the service process. On the other hand, the option of completing the Abilitator on paper was crucial for some service clients. The questions were phrased positively and simply to help the service clients self-report their situation in a neutral way. Multidimensionality and individuality were considered so that both the respondents and the service actors could receive enough information to advance in the most suitable service process. At the same time, the length of the questionnaire was restricted to prevent it becoming too long and heavy for the respondents to answer and the service actors to analyse. The interpretation of the results was made easy for the service clients to comprehend through short and positively phrased written feedback. For the service actors, the resulting interpretation was made as uncomplicated as possible through the given educational material and user support.
During the development process, the content of the Abilitator was modified twice [Additional file 1]. Three different expert panels were utilised during the process: an internal group of experts, an external group of academic specialists and an external group of practical experts. The essential development of the Abilitator was carried out by the internal group of experts (n = 8) from FIOH representing medical, health, sport, behavioural, and social sciences. One member of this group was from THL. The first external expert panel included academic specialists (n = 30) from the fields of work ability, functioning and social inclusion. The second external expert panel included both service actors and target group clients, bringing expertise by experience to the development. The service actors (n = 700) had varying lengths of work experience with the target population, and their occupational backgrounds ranged from university research scientists to social workers and sports coaches. They mostly worked in ESF Priority 5 projects. The target group experts (n = 29) were clients who received services in the ESF Priority 5 projects. On average they were 40 years of age, had been unemployed from three to seven years and had little post-primary education.
During Phase 1 (Figure 1), the internal group of experts created the structure and content of Abilitator 0.1 based on face validity. This was done by finding and combining the most relevant questions from pre-existing questionnaires and some newly formed questions into one self-report questionnaire. At this stage, we consulted several academic specialists (n = 15) from different organisations within the field of work ability, functioning and social inclusion to improve the 0.1 pilot version’s content.
We chose the following topics as the main elements of Abilitator 0.1: 1) Work ability and perceived health, 2) Everyday skills, 3) Social functioning and social involvement, 4) Psychological functioning 5) Cognitive functioning, 6) Physical functioning, and 7) Background information. These topics would cover the first three levels of the House of Work Ability and its dimensions of family, close community and society. Abilitator 0.1 contained 57 questions of which 30 (54%) were taken directly from pre-existing questionnaires [Additional file 1]. The rest were newly-formed questions covering target group-specific topics that had either not been evaluated by a self-assessment method before or for which the formulation of the pre-existing questions did not directly meet the criteria set for the Abilitator; for example, positive question format, equality, generality, and comprehensiveness.
During Phase 2 (Figure 1), the Finnish version of the Abilitator 0.1 was offered to the participants (n = 1573) in 44 national and local ESF priority 5 projects. The questionnaire could be completed in paper format or online. During Phase 3 (Figure 1), the service actors’ (n = 600) experiences of and suggestions regarding the content of Abilitator 0.1 were collected in 22 local or national co-development workshops. In every co-development workshop, the content of the whole questionnaire was covered in a similar manner to determine: 1) if each question was relevant for the target group, 2) if each question was formulated in a way that was appropriate for the target group, and 3) what kind of alterations should be made to each question for them to better suit the target groups’ needs or situations. Similar feedback was gathered from the service actors during the 10 visits to different EFS Priority 5 projects. One group of service clients (n = 7) also suggested question alterations and academic experts (n = 15) gave their input in separate encounters.
All the feedback was systematically gathered in written format and reviewed by the internal group of experts. The suggestions were grouped into similar feedback units and the decisions regarding changes to the questionnaire were made in the internal expert group’s consensus meetings. During this phase, 25% of the questions in Abilitator 0.1 remained unchanged, 50% were modified and 25% were removed. The unchanged questions were perceived as feasible for and by the target group and for evaluative purposes. The content or formulation of the questions was changed if: 1) the questions were not perceived as equal, 2) the original design of the question was not perceived as suitable for the target group, 3) the original design of the questions did not reveal the desired issue precisely enough, 4) the questions required more text to support their comprehension, 5) the themes of the questions were perceived as too narrow or extensive, 6) the questions lacked important areas or response options and 7) the questions had too many or too few response options. The questions removed from Abilitator 0.1 were 1) not answered as regularly as the others, 2) perceived as repetition, 3) not perceived as appropriate for or by the target population, 4) not perceived to cover the desired aspect, 5) not perceived as equal and 6) too difficult to answer. Nineteen completely new questions were added to the Abilitator 0.2. If important issues or sub-issues were completely missing, or if new questions were needed to better suit the target groups’ situation, the removed question was replaced with a new one.
In Abilitator 0.1, the recall period varied from the present to two weeks or a month. According to the feedback, this was confusing to both the respondents and the service actors. Therefore, in Abilitator 0.2, the recall period was harmonized to current situation except for Section D (Mind) in which the recall period was set as one month. In addition, the scales were harmonized and presented as either horizontally or vertically, and the best option was always at the furthest right or at the top, respectively.
An example of the expert panels’ influence on the Abilitator content are questions D8 and D9 [Additional file 2]. Abilitator 0.1 included two items for screening depression in primary care [71]. During Phase 3, systematic negative feedback received from the target group, the service actors and the academic experts led to the removal of these items from the Abilitator. The questions were considered too diagnostic to be used by service actors, too invasive to be answered by the respondents, and too difficult to evaluate in the context of its use. However, issues such as taking the initiative in everyday activities were still considered important. Therefore, the group of experts formulated two completely new items, D8 and D9. These questions were added to Abilitator 0.2 [Additional file 1] and the practical group of experts assessed their feasibility during Phases 5 and 6. Due to the systematically positive feedback received, Questions D8 and D9 of the Abilitator remained unchanged.
During Phase 4 (Figure 1), Abilitator 0.2 was released in Finnish and English for use with the client participants (n = 1787) in 43 national and local ESF Priority 5 projects. The questionnaire contained 76 questions and the online version also offered personal feedback. The content and format of this feedback was developed with the expert panels along with the content of the Abilitator. The internal expert group decided not to include all the questions in the feedback the Abilitator gives the respondent because one kind of answer is not always enough to make meaningful assumptions about the respondent’s situation. However, the interpretations of all the questions were analysed for the service actors in the Abilitator user manual.
In Phase 5 (Figure 1), cognitive interviews were conducted with clients (n = 21) participating in five different national or local ESF Priority 5 projects [35]. The groups interviewed had good geographical, gender and target group coverage. The aim of the cognitive interviews was to obtain information on how the respondents had processed and interpreted the questions of Abilitator 0.2. The method used in the interviews was a four-step question-answer process [72], and the interviews were related to the format, feasibility and comprehension of the questions presented in Abilitator 0.2. All the interviews were conducted by two interviewers and progressed following the same pattern. Each was recorded and transcribed.
In Phase 6 (Figure 1), we sent an online survey to all the service actors (n = 144) using Abilitator 0.2 to collect additional feedback on the content, feasibility and format of the instrument. We also held a second national co-development workshop for the service actors and other professionals (n = 35). We used the information and feedback gained from the survey (n = 42), the workshop and the cognitive interviews to steer the development of the Abilitator’s content and layout.
During this phase, all the feedback was again systematically gathered in written format and reviewed by the internal group of experts. The suggestions were grouped into similar feedback units and the decisions regarding changes to the questionnaire were made in the internal expert group’s consensus meetings. Sixty per cent of the questions in the Abilitator 0.2 remained unchanged, 38% were modified and 2% were removed. The unchanged questions were perceived as feasible for and by the target group and for evaluation purposes. The content or formulation of the questions was changed if: 1) the questions required more text to support their comprehension, 2) the order of the questions was not logical within the sections of the questionnaire, 3) the topic of a question was too extensive to answer and needed splitting into two separate questions. Based on the feedback received, we added three new questions to the questionnaire to obtain a broader view of the respondent’s situation.
In Phase 7 (Figure 1), the Finnish, English, Swedish and plain Finnish version of the Abilitator was offered to all health, social, rehabilitation, employment and education service actors working with the target group. The Abilitator contained 84 questions, 17 of which were items from existing questionnaires, and 67 were either modifications or completely new items.
Information on the content of the Abilitator and its use in practice
The Abilitator contains nine sections: A. Personal information, B. Well-being, C. Inclusion, D. Mind,
E. Everyday life, F. Skills, G. Body, H. Background information, and I. Work and the Future (Figure 3). Each section contains 4–14 questions. The whole questionnaire is presented in Additional file 2 and can also be accessed online [73].
Figure 3. Sections of the Abilitator in relation to its general concepts and concept framework.
The interpretation of the results as it is given in a respondent’s written feedback can be seen in Additional file 3. The feedback is built directly on the response options and has no external benchmark figures. The measure of each section is a summary scale of the selected item. The points received are converted into percentages: the minimum score is 0% and the maximum 100%. The feedback is grouped based on the respondent’s situation per sections B–G: 1) the situation is good, 2) the situation is fairly good, but has some possible challenges and 3) the situation is fairly poor or poor. If the respondent evaluates some items as very poor and others as good, the feedback indicates possible challenges. The content of the Abilitator and its development versions 0.1 and 0.2, the scales, and the ICF codes by question are illustrated in Additional file 1. Another way in which to interpret the results is to do so question by question. The instructions for this are presented in the Abilitator’s user manual, currently only available in Finnish [73].
In practice, the Abilitator can be used in different ways. A service actor in, for example, employment services can send the client a personal link to the Abilitator via email well before a scheduled appointment. On average the questionnaire takes 15–20 minutes to complete. The client can complete the questionnaire online independently or with someone close to them. Another option is that the service actor interviews the client and enters the responses directly into the online version of the Abilitator. A third option is that the service actor either gives or sends the Abilitator questionnaire in paper format to the client. The client then completes the questionnaire and returns it to the service actor, who enters the information into the online version.
The advantage of the online version of the Abilitator is that both the client and the service actor can see the results and personal feedback and prepare for their appointment accordingly. During the appointment, the client and the service actor can discuss the results, and plan targets and actions to improve or sustain the client’s work ability or functioning if necessary. In an ideal situation, they arrange a follow-up appointment during which they evaluate whether the targets have been met.
Correspondence of the Abilitator’s content to its construct
When the process of development was complete, the Abilitator was linked to the ICF. The first purpose of linking was to translate the instrument’s content into the internationally unified and consistent language of human functioning, which can be used as a reference for comparing health information. This linking was conducted in co-operation with the national ICF concept working group and followed the updated linking rules [74]. The linking was first conducted by two research scientists separately and consensus was reached in two separate sessions with three other ICF experts. The second purpose was to position the Abilitator among the ICF-linked self-report instruments measuring work ability and functioning, and to compare the Abilitator’s list of ICF codes to three ICF Core Sets [75].
The Abilitator covered 79 ICF codes, of which 14 (19%) described body functions and structures (b), 40 (54%) described activities and participation (d), 10 (13%) described environmental factors (e) and 15 (20%) described personal factors (pf). The ICF codes describing body structures and functions were related to global and specific mental and respiratory system functions. The codes related to activities and participation covered learning and applying knowledge, carrying out general tasks and demands, communication, mobility, self-care, domestic life, interpersonal interactions and relationships, and major life areas. The codes covering environmental factors described products and technology, support, relationships, and attitudes. The correspondence of all the Abilitator’s items to the ICF categories is illustrated in Figure 4 and Additional file 1.
Figure 4. Content of the Abilitator described using the ICF framework.
The assessment framework for correspondence between the Abilitator and its construct
To facilitate the everyday use of the ICF, WHO and the ICF Research Branch have created Core Sets of ICF through a scientific process. These ICF Core Sets are lists of the essential relevant categories for specific health conditions and health care contexts [75].
We compared the Abilitator’s 79 ICF codes with the three ICF Core Sets most relevant to the target population; the generic set (6 codes), the brief vocational rehabilitation set (11 codes) and the minimal environmental set (12 codes) [Additional file 4]. The direct equivalence of the Abilitator to the generic set was 3/6 codes (50%); to the brief vocational rehabilitation set, 8/11 codes (73%); and to the minimal environmental set, 2/12 codes (17%). In addition, two d4-category codes of the generic set, one e4-category code of the brief vocational rehabilitation set, and the e3-category of the minimal environmental set were indirectly represented in the Abilitator at another category level.
The Abilitator’s ICF codes were further compared with two validated, central self-report instruments: the WAI (13 codes) and WHODAS 2.0 (23 codes) [Additional file 4]. The WAI is used in occupational health care and research to assess employee work ability in health examinations and workplace surveys [76]. Its purpose is to help define the necessary actions for maintaining and promoting work ability. The direct equivalence of the Abilitator was 4/14 (29%) WAI codes. In addition, similar aspects of four codes were indirectly covered in the e3 and e4 categories. WHODAS 2.0 is a generic assessment instrument that provides a standardised method for measuring health and disability across cultures. It was developed from a comprehensive set of ICF items that are sufficiently reliable and sensitive for measuring the effects of a given intervention [77]. The direct equivalence of the Abilitator was 9/23 (39%) codes of WHODAS 2.0, and there were only minor differences in the codes concerning categories d4 and d5.