Evaluation of Nursing Competences in the Licensure Exam: an Italian Observational Study

doi:10.21203/rs.3.rs-1451073/v1

Download PDF

Research Article

Evaluation of Nursing Competences in the Licensure Exam: an Italian Observational Study

https://doi.org/10.21203/rs.3.rs-1451073/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

In recent years, the licensure exam in nursing has focused on the range of curricula and teaching methods used in international universities. ‘Tuning’ was the first document that sought to harmonise training purposes in terms of competences and learning outcomes in Europe. Nursing has 47 specific competences divided into 5 domains.

Methods

A multicentric observational perspective study was conducted in four universities in the Lazio region for to describe which of 47 Tuning competences are evaluated during the licensure exam in Italian nursing degree courses, at what level of performance, in which clinical area and setting are contextualised, and what types of tests are used for their evaluation. The commissioners of the Professional Order of Rome – OPI – collected data via tests in the licensure exams for the academic years 2017–2019 using 2 grids, one for cognitive and one for psychomotor assessments.

Results

The Tuning competences were requested 7522 times, 5130 for the cognitive trials and 2392 for the psychomotor trials. The most frequently requested competences were those associated with domain 2, and the level of performance most required in cognitive tests was autonomy of judgement (52.30%). In both tests, the most-investigated competences concerned the area of non-communicable diseases and the hospitalised adult patient.

Conclusions

The competences assessed most often were associated with nursing practice and clinical decision-making. These areas coincided with those deemed essential for the first cycle of studies and are contextualised for the priority health problems of hospitalised adult patients. This indicates that nursing education still focuses on the preparation of nursing students for hospital settings. Moreover, there is still a high degree of discrepancy in the types of tests used in different schools. Their choices are influenced by the number of students and the structural resources available.

Bachelor’s Degree

Nursing Education

Nursing Student

Nursing Competences

Tuning Nursing Project

Licensure Exam

Evaluation Nursing Competences

The concept of competence has played a leading role in the international debate in all professions, including nursing [1], coupled with a deep reflection on the competences that nurses must possess and on the methods that allow evaluation of the real acquisition of them [2]. The definitions given for the concept of competence are numerous and are all influenced by specific disciplinary cultural languages [3]. The training debate distinguishes ‘competence’ from other terms, and, according to the holistic approach, it is characterised by general and contextual attributes, which are considered essential for effective performance [4]. In this line of reasoning, the Bologna Process in Europe, set up in 1999 as an intergovernmental cooperation agreement in the field of higher education, proposes the adoption of a system of academic qualifications based on 3 comparable training cycles within the European Community [5] through the cooperation of all countries in assessing the quality, transparency and readability of training courses [6].

A concrete guide to the implementation of the policies of the Bologna Process is offered by the Project Tuning Educational Structures in Europe [7], started and financed by the European Commission in 2000. It consists of a methodology to design, develop, and evaluate courses of study according to the new cycle reform) [5]. Specifically, Tuning is a reference for the development of platforms useful to academic bodies to improve educational offerings in terms of competences and learning outcomes [7]. For the learning outcomes achieved by the student at the end of a cycle of education (understood as ‘Performance Levels’), the Dublin Descriptors are utilised. These descriptors represent enunciation of the learning outcomes and are built on the following elements: knowledge and understanding (knowledge and understanding); applied knowledge and understanding (applying knowledge and understanding), judgement autonomy (making judgements), communication competences (communication competences), and learning competences (learning competences)[8]. The Tuning Educational Structures have been offered in various disciplines, including nursing with the development of the Tuning Nursing Project [9]. The Tuning Nursing Project offers 47 specific nursing competences divided into 5 domains: competences associated with professional values and the role of the nurse, nursing practice and clinical decision-making knowledge and cognitive competences, communication, and interpersonal competences (including technology for communication) and leadership, management, and team competences [9].

The Tuning Nursing Project is the first document urging many countries to reform and tune their training courses at the European level. In the United Kingdom, for example, the study reform ended with the full and articulated acceptance of Tuning competences in nursing study plans [5]. In Italy, scholars have carried out the translation and cultural linguistic validation of the questionnaire Tuning Nursing Educational, developed by the European Commission, to harmonise the educational offering of Degree in Nursing courses [7, 10]. Indeed, many of the problems identified in Italian nursing training are linked to the lack of a single definition of the core competences of nurses. This is particularly evident during the professional licensure exam, which represents, in Italy, the formal act in which the acquired competences are certified and the right of the new professional to be registered and to practice [11, 12].

Despite the efforts of the training centres to standardise the content, methods, and times of the exam in the national territory, there are still relevant differences among the various educational programmes used in Italian universities. To try to explain these differences, the Authors (2013) [12] have developed a model to guide nursing evaluators in the assessment of competences during the licensure exam. The model consists of 4 elements: the 47 competences Tuning [9]; the learning outcomes required, offered according to the Performance Levels and defined by the Dublin Descriptors [8], the Clinical Area, which refers to the Priority Health Problems reported in the latest Italian National Health Plan [13] and the Setting, where these competences are investigated; and the types of tests used for the assessment of these competences [14].

Undeniably, if, on the one hand, the Authors (2015) [15] verified the nursing core competences considered by evaluators in the specific nursing sector for the 3-year period, on the other hand, they also proved that there were no studies that investigated whether the same core competences were those evaluated during the licensure exam and the level of performance they were investigating.

The same authors (2014), [16] discussing on evaluation methods, highlighted that the choice seemed to be guided by criteria that responded to different needs of the training schools: the number of students to be evaluated, resources and spaces available, evaluation habits of the individual locations, and preferences of those who prepare the tests. Also, the times of the examination and the number of tests and questions vary by location. Therefore, this study aims to fill a gap in the literature related to studies that describe which Tuning competences are evaluated and how they are evaluated during the licensure exam in the Degree of Nursing.

Objectives

The main objectives of this study were to:

1) outline the 47 Tuning competences evaluated during the licensure exam in the nursing degree courses;

2) determine the levels of performance of these competences;

3) determine the clinical areas and settings these competences were assessing; and

4) determine the types of tests utilised for competence evaluation.

Design

A multicentric observational perspective study was conducted.

Sample and setting

Data were collected from the graduation sessions in nursing (2017–2019) of 4 universities in the Lazio Region that realise 22% (n = 3744) of the 17 394 places available nationally for all Nursing Degrees [17].

Instruments

Two data collection grids, built according to the 4 elements of the Tuning Model [12] (Fig. 1), were used during the licensure exam for cognitive and psychomotor competences.

Both grids are divided into 3 sections:

Section 1 collects information on the training place.
Section 2 is divided into 4 subsections (both grids).
Section 3 collects information on cognitive and psychomotor tests.

Section 1 has no subsections. In section 2, there are 4 subsections as follows:

Subsection 1: ‘Code of Competence’. In this subsection, it is reported, each time and for each line, which of the 47 competences of the Tuning Nursing Project is investigated during the test. Competences are divided into 5 domains: (1) competences associated with professional values and the role of the nurse (6 items); (2) competences associated with nursing practice and clinical decision-making (14 items); (3) knowledge and cognitive competences (10 items); (4) communication and interpersonal competences including communication technologies (9 items); and (5) leadership, management, and group dynamics management competences (8 items) [9] (Include Table. 1).

Subsection 2: ‘Performance Levels’. In this subsection, for each type of test, the level at which the selected competence is investigated is defined. Performance levels refer to the Dublin Descriptors [18, 8]. The cognitive test measures 3 levels of performance: a) knowledge; b) ability to understand; and c) autonomy of judgement.

The psychomotor test measures a single level of performance: applied knowledge and understanding.

Subsection 3: ‘Clinical Area’ refers to the clinical area in which each competence is contextualised compared to priority health problems and provides 7 options: a) cardiovascular diseases, b) oncology, c) respiratory, d) psychiatric, e) metabolic, f) infectious, and g) not specified diseases [13, 19].

Subsection 4: ‘Setting’ defines the area where each competence is investigated and provides 6 options: a) adult, b) child, c) non specified e) hospital, f) territory and g) non specified.

In section 3, there are 3 types of tests for cognitive and 3 for psychomotor abilities. For cognitive, we have the following:

o Open or closed test, a questionnaire where, if the answer is open, the student freely expresses their thoughts on the subject of the questions; if it is closed, they choose one or more answers among those proposed in the questions of the questionnaire;

o Oral and/or written discussion of a clinical case consists of a care plan of a clinical case and application of the nursing process and critical thinking; and

o Oral and/or written discussion of protocols and procedures involves discussion of the operational tools needed to standardise nursing care, such as guidelines, protocols, and procedures.

For psychomotor, we have

o Practical test of a simulated case, which evaluates, in reference to predetermined criteria, the ability of the student to implement the nursing procedures via grids and checklists;

o Test on the bed, which consists of evaluation of the activities performed by the student on a real patient and within the work context. The activities are selected on the basis of 3 criteria: low invasiveness, low risk for patients, and consent; and

o Decontextualised practical test, which evaluates the technical competences of the student independently by context and clinical case [14] (Include Fig. 2, 3).

Data collection

Data were collected in the Rome area by the OPI commissioners who were part of the evaluation committee of the professional licensure exam. The commissioners accomplished, beforehand, a specific lifelong learning programme aimed at instructing them on the use of the grids.

Data analysis

Data were analysed with descriptive statistics. Frequencies and percentages are used to describe the number of times each of the 47 Tuning competences was required, the relative levels of performance, the setting and clinical area in which they were contextualised, and the types of tests most used for their evaluation. Statistical software SPSS version 22.0 (IBM SPSS Inc. 2012, New York) was utilised for data analysis.

Overall, Tuning competences were requested 7522 times, of which 5130 (68.20%) were requested in cognitive tests and 2392 (31.80%) in psychomotor evaluations.

Regarding the first and second objectives of the study, Table 2 shows in detail the frequencies and percentages of competences required for cognitive and psychomotor tests and their connected performance levels (See Table 2).

Table 1

Tuning Nursing Specific Competences
1. Competences associated with professional values and the role of the nurse
1. Demonstrates the ability to practise within the context of professional, ethical, regulatory and legal codes, recognising and responding to moral/ethical dilemmas and issues in day to day practice.
2. Demonstrates the ability to practise in a holistic, tolerant, non judgmental, caring and sensitive manner, ensuring that the rights, beliefs and wishes of different individuals and groups are not compromised.
3. Demonstrates the ability to educate, facilitate, promote, support and encourage the health, wellbeing and comfort of populations, communities, groups and individuals whose lives are affected by, ill death, distress, disease, disability or death.
4. Within the scope of his/her professional practice and accountability, demonstrates awareness of the different roles, responsibilities and functions of a nurse.
5. Within the scope of his/her professional practice and accountability, demonstrates the ability to adjust their role to respond effectively to population/patient needs. Where necessary and appropriate is able to challenge current systems to meet population/patient needs.
6. Demonstrates the ability to accept responsibility for his/her own professional development and learning, using evaluation as a way to reflect and improve upon his/her performance so as to enhance the quality of service delivery.
2. Competences: nursing practice and clinical decision making
7. Demonstrates the ability to undertake comprehensive and systematic assessments using the tools/frameworks appropriate to the patient/client taking into account relevant physical, social, cultural, psychological, spiritual and environment factors.
8. Demonstrates the ability to undertake an effective risk assessment and take appropriate actions.
9. Demonstrates the ability to recognise and interpret signs of normal and changing health/ ill health, distress, or disability in the person (assessment/diagnosis).
10. Demonstrates the ability to respond to patient/client needs by planning, delivering and evaluating appropriate and individualised programmes of care working in partnership with the patient/client, their carers, families and other health/social workers.
11. Demonstrates the ability to critically question, evaluate, interpret and synthesis a range of information and data sources to facilitate patient choice.
12. Demonstrates the ability to make sound clinical judgments to ensure quality standards are met and practice is evidence based.
13. Demonstrates the ability to use modern technologies to assess and respond appropriately to patient/client need (for example through telenursing, multimedia and web resources).
14. Demonstrates the ability to appropriately use a range of nurse skills, medical devices and interventions/activities to provide optimum care.
15. Using nursing skills, medical devices and interventions/activities to provide optimum care, demonstrates the ability to maintain patient/client dignity, advocacy and confidentiality.
16. Using nursing skills, medical devices and interventions/activities to provide optimum care, demonstrates the ability to practice principles of health and safety, including moving and handling, infection control; essential first aid and emergency procedures.
17. Using nursing skills, medical devices and interventions/activities to provide optimum care, demonstrates the ability to safely administer medicines and other therapies.
18. Using nursing skills, medical devices and interventions/activities to provide optimum care, demonstrates the ability to consider emotional, physical and personal care needs, including meeting the need for comfort, nutrition, personal hygiene and enabling the person to maintain the activities necessary for daily life.
19. Using nursing skills, medical devices and interventions/activities to provide optimum care, demonstrates the ability to respond to a person’s needs throughout the life span and health/illness experience e.g. pain, life choices, revalidation, invalidity or when dying.
20. Demonstrates the ability to inform, educate and supervise patient/carers and their families.
3. Knowledge and cognitive competences
21. Demonstrates current and relevant knowledge of the theories of nursing and nursing practice that can be appropriately applied to nursing practice, patient/client care and situations of uncertainty.
22. Demonstrates current and relevant knowledge of theories concerning the nature and challenge of professional practice that can be appropriately applied to nursing practice, patient/client care and situations of uncertainty.
23. Demonstrates current and relevant knowledge of the natural and life sciences that can be appropriately applied to nursing practice, patient/client care and situations of uncertainty.
24. Demonstrates current and relevant knowledge of the social, health and behavioural sciences that can be appropriately applied to nursing practice, patient/client care and situations of uncertainty.
25. Demonstrates current and relevant knowledge of ethical theory, law and humanities that can be appropriately applied to nursing practice, patient/client care and situations of uncertainty.
26. Demonstrates current and relevant knowledge of technology and health care informatics that can be appropriately applied to nursing practice, patient/client care and situations of uncertainty.
27. Demonstrates current and relevant knowledge of international and national policies that can be appropriately applied to nursing practice, patient/client care and situations of uncertainty.
28. Demonstrates current and relevant knowledge of problem solving, decision making and conflict theories that can be appropriately applied to nursing practice, patient/client care and situations of uncertainty.
29. Demonstrates current and relevant knowledge of theories related to personal and professional development so as to enhance their professional practice.
30. Demonstrates current and relevant knowledge of the research process and current nursing research that can be appropriately applied to nursing actions nursing activities to provide nursing care that is rigorous and evidence based.
4. Communication and interpersonal competences (including technology for communication)
31. Demonstrates the ability to communicate effectively (including the use of new technologies): with patients, families and social groups, including those with communication difficulties.
32. Demonstrates the ability to enable patients and their carers to express their concerns and worries and can respond appropriately e.g. emotional, social, psychological, spiritual or physical worries.
33. Demonstrates the ability to appropriately represent the patient/client's perspective and act to prevent abuse.
34. Demonstrates the ability to appropriately use counselling skills to promote patient well being;
35. Demonstrates the ability to identify and manage challenging behaviour (using communication techniques to promote patient well being).
36. Demonstrates the ability to recognise anxiety, stress and depression (using communication techniques to promote patient well being).
37. Demonstrates the ability to give emotional support and identify when specialist counselling or other interventions are needed.
38. Demonstrates the ability to identify and use opportunities for health promotion and health education activities.
39. Demonstrates the ability to accurately report, record, document and refer care using appropriate technologies.
5. Leadership, management and team competences
40. Demonstrates the ability to realise that patient/client well-being is achieved through the combined resources and actions of all members of the health/social care team.
41. Demonstrates the ability to lead and co-ordinate a team, delegating care appropriately and meaningfully.
42. Demonstrates the ability to work and communicate collaboratively and effectively with other nurses in the best interests of the patient/client.
43. Demonstrates the ability to work and communicate collaboratively and effectively with all support staff to prioritise and manage time effectively while quality standards are met
44. Demonstrates the ability to assess risk and actively promote the well-being, security and safety of all people in the working environment (including themselves).
45. Demonstrates the ability to critically use tools to evaluate and audit care according to relevant quality standards.
46. Within the clinical context, demonstrates the ability to educate, facilitate, supervise and support nursing students and other health/social care students or workers.
47. Demonstrates an awareness of the principles of health/social care funding and uses resources effectively.
https://www.unideusto.org/tuningeu/competences/specific/nursing.html
5 Domains of 47 Tuning Competences:
1. Competences associated with professional values and the role of the nurse (6 items);
2. Competences associated with nursing practice and clinical decision-making (14 items);
3. Knowledge and cognitive competences (10 items);
4. Communication and interpersonal competences including communication technologies (9 items); 5. Leadership, management, and group dynamics management competences (8 items).

Table 2

Competences and Levels of Performance required in cognitive and psychomotor tests (N = 7522)
	Cognitive tests	Levels of Performance in cognitive tests						Psychomotor tests
DOMAINS	COMPETENCES REQUIRED	N of times each competence is required		Autonomy		Capacity		Knowledge		Level of Performance not specified		N of times each competence is required
		n	%	n	%	n	%	n	%	n	%	n	%
1. competences associated with professional values and the role of the nurse	1. Demonstrates the ability to practise within the context of professional. ethical. regulatory and legal codes. recognising and responding to moral/ethical dilemmas and issues in day to day practice	36	0.70	8	22.22	21	58.33	6	16.67	1	2.78	5	0.21
	2. Demonstrates the ability to practise in a holistic. tolerant. Non-judgmental. caring and sensitive manner. ensuring that the rights. beliefs and wishes of different individuals and groups are not compromised.	14	0.27	7	50.00	5	35.71	2	14.29	0	0.00	5	0.21
	3. Demonstrates the ability to educate. facilitate. promote. support and encourage the health. wellbeing and comfort of populations. communities. groups and individuals whose lives are affected by. ill death. distress. disease. disability or death.	92	1.79	43	46.74	29	31.52	17	18.48	3	3.26	21	0.88
	4. Within the scope of his/her professional practice and accountability. demonstrates awareness of the different roles. responsibilities and functions of a nurse.	54	1.05	17	31.48	13	24.07	22	40.74	2	3.71	52	2.17
	5. Within the scope of his/her professional practice and accountability. demonstrates the ability to adjust their role to respond effectively to population/patient needs. Where necessary and appropriate is able to challenge current systems to meet population/patient needs.	14	0.27	10	71.43	3	21.43	0	0.00	1	714	1	0.04
	6. Demonstrates the ability to accept responsibility for his/her own professional development and learning. using evaluation as a way to reflect and improve upon his/her performance so as to enhance the quality of service delivery	9	0.18	5	55.56	1	11.11	3	33.33	0	0.00	6	0.25
2. Competences: nursing practice and clinical decision making	7. Demonstrates the ability to undertake comprehensive and systematic assessments using the tools/frameworks appropriate to the patient/client taking into account relevant physical. social. cultural. psychological. spiritual and environment factors.	463	9.03	243	52.48	113	24.41	102	22.03	5	1.08	85	3.55
	8. Demonstrates the ability to undertake an effective risk assessment and take appropriate actions.	441	8.60	259	58.73	94	21.31	81	18.37	7	1.59	258	10.79
	9. Demonstrates the ability to recognise and interpret signs of normal and changing health/ ill health. distress. or disability in the person (assessment/diagnosis).	479	9.34	245	51.15	145	30.27	80	16.70	9	1.88	150	6.27
	10. Demonstrates the ability to respond to patient/client needs by planning. delivering and evaluating appropriate and individualised programmes of care working in partnership with the patient/client. their carers. families and other health/social workers.	495	9.65	309	62.42	114	23.03	66	13.34	6	1.21	70	2.93
	11. Demonstrates the ability to critically question. evaluate. interpret and synthesis a range of information and data sources to facilitate patient choice.	45	0.88	32	71.12	6	13.33	6	13.33	1	2.22	25	1.05
	12. Demonstrates the ability to make sound clinical judgments to ensure quality standards are met and practice is evidence based.	62	1.21	37	59.68	12	19.35	10	16.13	3	4.84	102	4.26
	13. Demonstrates the ability to use modern technologies to assess and respond appropriately to patient/client need (for example through telenursing. multimedia and web resources).	10	0.19	3	30.00	1	10.00	6	60.00	0	0.00	7	0.29
	14. Demonstrates the ability to appropriately use a range of nurse skills. medical devices and interventions/activities to provide optimum care.	645	12.57	249	38.60	192	29.77	200	31.01	4	0.62	493	20.61
	15. Using nursing skills. medical devices and interventions/activities to provide optimum care. demonstrates the ability to maintain patient/client dignity. advocacy and confidentiality.	72	1.40	23	31.94	32	44.45	12	16.67	5	6.94	147	6.15
	16. Using nursing skills. medical devices and interventions/activities to provide optimum care. demonstrates the ability to practice principles of health and safety. including moving and handling. infection control; essential first aid and emergency procedures.	380	7.41	221	58.15	62	16.32	91	23.95	6	1.58	194	8.11
	17. Using nursing skills. medical devices and interventions/activities to provide optimum care. demonstrates the ability to safely administer medicines and other therapies.	400	7.80	166	41.50	137	34.25	93	23.25	4	1.00	209	8.74
	18. Using nursing skills. medical devices and interventions/activities to provide optimum care. demonstrates the ability to consider emotional. physical and personal care needs. including meeting the need for comfort. nutrition. personal hygiene and enabling the person to maintain the activities necessary for daily life.	236	4.60	153	64.83	36	15.25	43	18.22	4	1.70	42	1.76
	19. Using nursing skills. medical devices and interventions/activities to provide optimum care. demonstrates the ability to respond to a person’s needs throughout the life span and health/illness experience e.g. pain. life choices. revalidation. invalidity or when dying.	68	1.33	43	63.24	8	11.76	13	19.12	4	5.88	10	0.42
	20. Demonstrates the ability to inform. educate and supervise patient/carers and their families.	175	3.41	104	59.43	36	20.57	30	17.14	5	2.86	74	3.09
3. Knowledge and cognitive competences	21. Demonstrates current and relevant knowledge of the theories of nursing and nursing practice that can be appropriately applied to nursing practice. patient/client care and situations of uncertainty.	72	1.40	33	45.83	22	30.56	16	22.22	1	1.39	31	1.30
	22. Demonstrates current and relevant knowledge of theories concerning the nature and challenge of professional practice that can be appropriately applied to nursing practice. patient/client care and situations of uncertainty	4	0.08	3	75.00	0	0.00	1	25.00	0	0.00	2	0.08
	23. Demonstrates current and relevant knowledge of the natural and life sciences that can be appropriately applied to nursing practice. patient/client care and situations of uncertainty.	185	3.61	66	35.68	54	29.19	62	33.51	3	1.62	76	3.18
	24. Demonstrates current and relevant knowledge of the social. health and behavioural sciences that can be appropriately applied to nursing practice. patient/client care and situations of uncertainty	54	1.05	50	92.60	1	1.85	2	3.70	1	1.85	2	0.08
	25. Demonstrates current and relevant knowledge of ethical theory. law and humanities that can be appropriately applied to nursing practice. patient/client care and situations of uncertainty.	2	0.04	1	50.00	1	50.00	0	0.00	0	0.00	1	0.04
	26. Demonstrates current and relevant knowledge of technology and health care informatics that can be appropriately applied to nursing practice. patient/client care and situations of uncertainty.	3	0.06	0	0.00	1	33.33	2	66.67	0	0.00	0	0
	27. Demonstrates current and relevant knowledge of international and national policies that can be appropriately applied to nursing practice. patient/client care and situations of uncertainty.	13	0.25	4	30.77	1	7.69	8	61.54	0	0.00	54	2.26
	28. Demonstrates current and relevant knowledge of problem solving. Decision-making and conflict theories that can be appropriately applied to nursing practice. patient/client care and situations of uncertainty.	148	2.88	116	78.37	10	6.76	21	14.19	1	0.68	58	2.42
	29. Demonstrates current and relevant knowledge of theories related to personal and professional development so as to enhance their professional practice.	2	0.04	1	50.00	0	0.00	0	0.00	1	50.00	8	0.33
	30. Demonstrates current and relevant knowledge of the research process and current nursing research that can be appropriately applied to nursing actions nursing activities to provide nursing care that is rigorous and evidence based.	16	0.31	3	18.75	7	43.75	6	37.50	0	0.00	26	1.09
4. Communication and interpersonal competences (including technology for communication)	31. Demonstrates the ability to communicate effectively (including the use of new technologies): with patients. families and social groups. including those with communication difficulties.	33	0.64	18	54.55	13	39.39	2	6.06	0	0.00	10	0.42
	32. Demonstrates the ability to enable patients and their carers to express their concerns and worries and can respond appropriately e.g. emotional. social. psychological. spiritual or physical worries.	73	1.42	40	54.79	22	30.14	8	10.96	3	4.11	19	0.79
	33. Demonstrates the ability to appropriately represent the patient/client's perspective and act to prevent abuse.	1	0.02	1	100	0	0.00	0	0.00	0	0.00	1	0.04
	34. Demonstrates the ability to appropriately use counselling skills to promote patient well being; (using communication techniques to promote patient well-being)	33	0.64	13	39.40	10	30.30	10	30.30	0	0.00	12	0.50
	35. Demonstrates the ability to identify and manage challenging behaviour (using communication techniques to promote patient well-being).	13	0.25	11	84.62	2	15.38	0	0.00	0	0.00	2	0.08
	36. Demonstrates the ability to recognise anxiety. stress and depression (using communication techniques to promote patient well-being).	61	1.20	23	37.70	27	44.26	9	14.75	2	3.29	11	0.46
	37. Demonstrates the ability to give emotional support and identify when specialist counselling or other interventions are needed.	26	0.51	15	57.69	5	19.23	3	11.54	3	11.54	7	0.30
	38. Demonstrates the ability to identify and use opportunities for health promotion and health education activities.	40	0.78	17	42.50	11	27.50	11	27.50	1	2.50	6	0.25
	39. Demonstrates the ability to accurately report. record. document and refer care using appropriate technologies.	14	0.27	4	28.57	4	28.57	6	42.86	0	0.00	57	2.38
5. Leadership. management and team competences	40. Demonstrates the ability to realise that patient/client well-being is achieved through the combined resources and actions of all members of the health/social care team.	9	0.18	4	44.44	1	11.12	4	44.44	0	0.00	5	0.21
	41. Demonstrates the ability to lead and co-ordinate a team. delegating care appropriately and meaningfully.	30	0.58	18	60	0	0.00	12	40.00	0	0.00	0	0
	42. Demonstrates the ability to work and communicate collaboratively and effectively with other nurses in the best interests of the patient/client.	14	0.27	4	28.57	5	35.71	2	14.29	3	21.43	17	0.71
	43. Demonstrates the ability to work and communicate collaboratively and effectively with all support staff to prioritise and manage time effectively while quality standards are met.	20	0.40	18	90	0	0.00	2	10.00	0	0.00	0	0
	44. Demonstrates the ability to assess risk and actively promote the well-being. security and safety of all people in the working environment (including themselves).	36	0.70	21	58.33	5	13.89	10	27.78	0	0.00	27	1.13
	45. Demonstrates the ability to critically use tools to evaluate and audit care according to relevant quality standards.	22	0.43	20	90.91	0	0.00	2	9.09	0	0.00	4	0.17
	46. Within the clinical context. demonstrates the ability to educate. facilitate. supervise and support nursing students and other health/social care students or workers.	13	0.25	1	7.70	2	15.38	10	76.92	0	0.00	0	0
	47. Demonstrates an awareness of the principles of health/social care funding and uses resources effectively.	3	0.06	1	33.33	0	0.00	2	66.67	0	0.00	0	0
Total		5130	100	2683	52.30	1264	24.64	1094	21.33	89	1.73	2392	100
N = Total number of Tuning competences required in cognitive and psychomotor tests
n = frequencies and % = percentages of 47 Tuning competences required for cognitive and psychomotor tests and their Levels of Performance

In cognitive tests, the most frequently investigated competence was number 14 (n = 645; 12.57%), followed by number 10 (n = 495; 9.65%) and number 9 (n = 479; 9.34%). The 3 most investigated competences belong to the second domain (competences associated with nursing practice and clinical decision-making).

In contrast, the least-required competence for cognitive tests was number 33 (n = 1; 0.02%), followed by number 29 (n = 2; 0.04%) and number 25 (n = 2; 0.04%). Of the 3 least-investigated competences, numbers 25 and 29 belong to domain 3 and number 33 to domain 4.

In psychomotor tests, the most frequently investigated competence was number 14 (n = 493; 20.61%), followed by number 8 (n = 258; 10.79%) and number 17 (n = 209; 8.74%).

Also, for the psychomotor test, the 3 most-valuated competences fitted in the second domain.

The least-required competence for psychomotor tests was number 25 (n = 1; 0.04%), followed by number 33 (n = 1; 0.04%) and number 22 (n = 2; 0.08%). Among the 3 least-investigated competences, numbers 22 and 25 were part of domain 3 and number 33 of domain 4.

Competence 26, which belongs to domain 3, and 41, 43, 46, and 47, all included in domain 5, were never requested during the psychomotor test.

Of the 5130 times in which the Tuning competences were investigated in the cognitive tests, the level of performance most frequently required was the autonomy of judgement (n = 2683; 52.30%), with the ability to understand (n = 1264; 24.64%) and knowledge (n = 1094; 21.33%) ensuing. Eighty-nine times (1.73%), the level of performance was not specified. A single level of performance was required in the psychomotor test: knowledge and applied comprehension.

Regarding the third objective of the study, Tables 3 and 4 show in detail the frequencies and percentages of the clinical area and setting for cognitive and psychomotor tests (Include Table 2)

Regarding the 5130 times in which the competences were demanded in the cognitive tests, 2779 times (54.18%), the clinical area was not specified; in 576 cases (11.23%), the competences regarded the clinical area of the cardiovascular diseases, followed by the oncological area (n = 543; 10.58%), metabolic (n = 465; 9.06%), respiratory (n = 450; 8.77%), infectious (n = 210; 4.09%), and psychiatric (n = 107; 2.09%). Of all the competences detected, 83.29% (n = 4273) concerned the adult patient, and 70.25% (n = 3604) were contextualised in hospital.

Regarding the 2392 times in which the competences were required in the psychomotor tests, in 1639 (68.52%), the clinical area was not specified; in 227 (9.49%), the competences considered the clinical area of the respiratory diseases, followed by the cardiovascular (n = 164; 6.86%), oncological (n = 146; 6.10%), metabolic (n = 120; 5.02%), infectious (n = 73; 3.05%), and psychiatric (n = 23; 0.96%). In all, 75.5% (n = 1806) of the competences surveyed involved the adult patient, and 59.91% (n = 1433) were contextualised in hospital (Tables 3, 4).

Table 3

Contextualization Clinical Area of the competences required in cognitive and psychomotor tests (N = 7522)
	COMPETENCES REQUIRED		Cardiovascular diseases		Oncological diseases		Respiratory diseases		Metabolic diseases		Psychiatric diseases		Infectious diseases		Not specified Clinical area
	n	%	n	%	n	%	n	%	n	%	n	%	n	%	n	%
Cognitive tests	5130	100	576	11.23	543	10.58	450	8.77	465	9.06	107	2.09	210	4.09	2779	54.18
Psychomotor tests	2392	100	164	6.86	146	6.1	227	9.49	120	5.02	23	0.96	73	3.05	1639	68.52

Table 4

Contextualization Setting of the competences required in cognitive and psychomotor tests N = 7522)
	COMPETENCES REQUIRED		Adult		Child		Not specified setting		COMPETENCES REQUIRED		Hospital		Territory		Not specified
	n	%	n	%	n	%	n	%	n	%	n	%	n	%	n	%
Cognitive tests	5130	100	4273	83.29	161	3.10	696	13.61	5130	100	3604	70.25	619	12.10	907	17.65
Psychomotor tests	2392	100	1806	75.50	57	2.40	529	22.10	2392	100	1433	59.91	217	9.07	742	31.02

Regarding the fourth objective of the study, Table 5 shows in detail the frequencies and percentages of the types of tests used in the cognitive and psychomotor tests.

Altogether, 144 tests were completed; 95 (65.97%) were cognitive tests and 49 (34.03%) for psychomotor tests. The type of test most used in cognitive assessments was the oral or written discussion of a clinical case (n = 72; 75.79%), followed by oral or written discussion of protocols and procedures (n = 21; 22.11%) and open- or closed-ended tests (n = 2; 2.10%).

The type of tests most used in psychomotor tests was the decontextualised practical test (n = 26; 53.06%), followed by the simulated case test (n = 17; 34.70%) and the on-the-bed test (n = 6; 12.24%) (Table 5).

Table 5

Frequency of the types of tests most used during the licensure exam in cognitive and psychomotor tests (N 144)
Cognitive tests			Psychomotor test
Types of tests	n	%	Types of tests	n	%
Oral and / or written discussion of a clinical case	72	75.79	Decontextualized practical test	26	53.06
Oral and / or written discussion of protocols and procedures	21	22.11	Practical test on a simulated case	17	34.70
Open or closed answer test	2	2.10	Test on the bed	6	12.24
Total	95	100	Totale	49	100

The aims of the study were to describe which of the 47 Tuning competences were assessed during the qualification exam for the Degree in Nursing, the level of performance, the clinical area and setting in which they were contextualised, and which types of tests were used for their evaluation.

In both cognitive and psychomotor tests, the competences most frequently requested during the qualification exam were those associated with nursing practice and clinical decision-making (scope 2), which coincide with core competences in nursing outlined by different scholars [15, 20].

The competence most in demand in both tests was number 14: demonstrates the ability to appropriately use a range of nurse skills, medical devices and interventions/activities to provide optimum care. This is in line with that expressed by Italian scholars, who believe that nursing degree courses should provide students with preparation that would allow them to make clinical decisions independently with their knowledge, technical competences, and attitudes [20–21]. Students must be able to provide quality nursing care with proven effectiveness [15, 22].

Among the competences most investigated, especially in psychomotor tests, there was number 17: Using nursing skills, medical devices and interventions/activities to provide optimum care, demonstrates the ability to safely administer medicines and other therapies. This data consolidates the welfare aspect linked to the ‘medical paradigm’ [23], according to which nurses guarantee the correct application of diagnostic–therapeutic prescriptions. This confirms the opinion of Italian academics that include this competence in the shortlist of those deemed core [20].

In the literature, there are differing opinions of scholars and students with respect to the ability of recent graduates to manage drugs safely. Indeed, if, in some studies, it is asserted that the hours of teaching in pharmacology are sufficient to prepare students for the correct and safe management of drugs [24, 25], in other studies, the authors state exactly the opposite [26].

In this research, the least-frequently required competences in both tests were related to communicative and interpersonal aspects, including communication technologies (domain 4) and leadership competences and group dynamics management (domain 5). The competences of leadership, management, and group dynamics are considered ‘core’ to be acquired in the second cycle of studies [20]. This suggests that Italian nursing education interprets post-basic training as a process of ‘additional training.’ According to the Bologna Process, however, postgraduate courses should include a ‘deepening of training’ of all competences already acquired in the first cycle of studies [7].

Regarding ‘Performance Levels’, only one level of performance is provided for psychomotor tests: autonomous, effective, and safe execution of simple procedures and the ability to provide information, communication, and/or a therapeutic relationship. In this study, in the 3 likely levels of cognitive tests, the most required was ‘autonomy of judgement’. This outcome is consistent with Lewallen and Van Horn (2019) [27], who state that at the end of the training course, it is expected that the student can critically assess decisions that were employed [28, 8].

Concerning the clinical area, the most-frequently investigated competences were to be found mainly in the field of chronic non-communicable diseases. Chronic non-communicable diseases are the main causes of death worldwide [19].

Regarding the setting, the most-investigated competences dealt with the adult patient, treated mainly in hospital rather than outside of it. These results show that the educational process remains anchored to the hospital-centred approach [29]. It would be appropriate to redesign educational programmes to train nurses who will deal with home care [30].

Compared to the types of tests used for the assessment of competences, the field mainly evaluated was the cognitive one. The test most used in cognitive evaluations was the discussion of a clinical case, even if it required a notable commitment from the assessors, both in its elaboration and in its correction.

This finding matches with opinions expressed by Italian academics [16] urging further investigation of the cognitive sphere. This also depends on the logistical difficulty of organising a complex test able to evaluate all nursing competences, such as the test on the bed [2], which, in this study, was the least used. Indeed, even if the test on the bed allows the evaluation of all nursing competences, it is not frequently used because it is costly and requires the use of sizable close human, instrumental, and structural resources. Furthermore, this could be the reason for the lower demand for communicative, leadership, and management competences [31], since they imply a real relationship between the student, the patient, and other health professionals [32]. However, these tests are organised based on the availability of the operational units and the planning of their activities and are difficult to recreate during the examination session. The least-used test in cognitive assessment was the open- or closed-response test. This is probably because it entails considerable efforts by educators in the structuring and correction of the test, especially if it is open-ended [33].

The most suitable tests for the evaluation of psychomotor competences were the practical tests on the simulated case and on the bed [34], which, in this study, however, were the least used. The reason that the simulated case practical test was little used can be explained by the difficulty in applying the Objective Structured Clinical Examination (OSCE) [35].

Considering the results obtained, there are many differences with respect to how the licensure exam is regulated in the various universities and the methods used during the exam to assess the competences [15, 3].

Limitations and future implications

The study was conducted in a single geographical area, and it focused only on the process of evaluating competences during the examination for qualification as a newly registered nurse.

There was a high degree of discrepancy in the types of tests used from site to university, the choice of which is influenced by the number of students and the structural resources available, elements which lead teachers to essentially choose the tests that explore the cognitive sphere.

A radical cultural change is expected, involving all the actors responsible for nursing education to reorientate themselves and to adopt a single model for evaluation in the licensure exam.

It is also strongly desirable to conduct a longitudinal study to collect data on the methods of assessing competences throughout the nursing degree courses in the various areas of the country.

In recent decades, nursing training has undergone a major change, and the transformation is still underway; however, there are still inconsistencies with regard to which competences are evaluated and how they are evaluated during the licensure exam.

In Italy, the competences considered most relevant for the first cycle of studies are those associated with nursing practice and clinical decision-making. A clear congruence has emerged between the competences that professors define as fundamental to acquire for the I cycle of studies and those really evaluated in the examination of nursing ability. It is also evident that the difficulty, on the part of training centres, in evaluating the communicative competences and the management and leadership competences of the student is closely linked to the ability to define the tests themselves by educators and to the logistical commitment that characterises the tests aimed at investigating these competences.

Nursing training still focuses on the preparation of student nurses to work in hospitals. Therefore, it is essential to proceed with an urgent reformulation and updating of training curricula.

Ethics approval and consent to participate

The study was designed, conducted, registered, and reported consistently with the international ethical and scientific quality standards indicated by Good Clinical Practice (GCP) and Standard Operating Procedures (SOP). The study complies with the Declaration of Helsinki, and it was approved ethically by the Centre of Excellence for Nursing Scholarship OPI Rome Protocol number 1.17.1.

This study did not involve patients. All participants were voluntarily involved and fully informed of the study’s purpose and participants were also informed of the confidentiality and anonymity of their responses during the data collection and analysis processes. Valuators expressed written informed consent.

Consent for publication

Not applicable

Availability of data and materials

All data generated or analysed during this study are included in this published article

Competing interests

The authors declare that they have no competing interests

Funding

The authors received no financial support for the research, authorship, and/or publication on this article

Authors' contributions

FP contributed to literature review, data curation, writing - original draft and writing-reviewing;

DDA conceptualization and data curation;

AS contributed to supervision and writing-reviewing;

BA contributed to data curation and supervision;

AP contributed to investigation, data curation and formal analysis;

IN contributed to investigation and formal analysis;

AMLP contributed to conceptualization, methodology, data curation, resources, supervision and project administration;

MGDM contributed to conceptualization, methodology, formal analysis, data curation, resources, writing-original draft, writing-review and editing, visualization and funding acquisition;

AM contributed to conceptualization, methodology, investigation, data curation, resources, writing - original draft, writing - review and editing, supervision, project administration, visualization and funding acquisition.

Acknowledgements

The authors wish to acknowledge the Registered Nurses Fabio Giorgi, Maria Donata Zecca, Giuliana Evangelisti and Nadia Lolli who participated in data collection. Furthermore The authors wish to acknowledge the Center of Excellence for the OPI Nursing Scholarship in Rome for supporting the research and the coordinators of the Degree Courses in Nursing of the four universities of Lazio, which made it possible to carry out the study.

Liu Y, Aungsuroch Y. Current Literature Review of Registered Nurses’ Competency in the Global Community. J Nurs Scholarsh. 2018;50(2):191–9.
Lejonqvist GB, Eriksson K, Meretoja R. Evaluating clinical competence during nursing education: A comprehensive integrative literature review. Int J Nurs Pract. 2016;22(2):142–51.
Charette M, McKenna LG, Deschênes MF, Ha L, Merisier S, Lavoie P. New graduate nurses’ clinical competence: A mixed methods systematic review. J Adv Nurs. 2020;76(11):2810–29.
Garside JR, Nhemachena JZZ. A concept analysis of competence and its transition in nursing. Nurse Educ Today. 2013;33(5):541–5.
Gobbi M. A review of nurse educator career pathways: A European perspective. J Res Nurs. 2009;14(2):123–4.
Cabrera E, Zabalegui A. Bologna process in European nursing education. Ten years later, lights and shadows. J Adv Nurs. 2021;77(3):1102–4.
Tuning Educational Structures in Europe: Guidelines and Reference Points for the Design and Delivery of Degree Programmes in Nursing. 2018. https://core.ac.uk/download/pdf/161659845.pdf. Accessed 27.02.2021.
Ministry of Education University and Research: The Framework of Italian Titles. Cycle Descriptors. 2012. Available from http://quadrodeititoli.it/descrittori.aspx?descr=172&IDL=1. Accessed 05.06.2021.
Tuning Project: Nursing specific competences. 2012. http://www.unideusto.org/tuningeu/competences/specific/nursing.html. Accessed 03.03.2021.
Venturini G, Pulimeno AML, Colasanti D, Barberi S, Sferrazza S, De Marinis MG. Linguistic and cultural validation of the Italian version of the Tuning Educational Structures in Europe questionnaire for the evaluation of nursing specific competences. Nurs.2012; 49(3) e39–e48. Available from https://www.infermiereonline.org/2012/07/18/validazione-linguistico-culturale-della-versione-italiana-del-questionario-sulle-competenze-infermieristiche-del-progetto-tuning-educational-structures-in-europe/.Accessed 07.06.2021.
Ministry of Education University and Research: Decree of 22 October 2004. 270. Amendments to the Regulations Concerning the Curricular Autonomy of Universities. 2004. Official Gazette of 12 November 2004.2004. Available from http://www.quadrodeititoli.it/files/1271_decreto_22_ottobre_2004_n_270-it-it.pdf. Accessed 10.11.2021.
Authors, 2013.
Ministry of Health: Directorate General of Health Prevention. National Prevention Plan 2020–2025. 2020. Available from http://www.statoregioni.it/media/2883/p-5-csr-atto-rep-n-127-6ago2020.pdf. Accessed 07.06/2021.
Guilbert JJ. Pedagogical guide for healthcare professionals.1a ed. Bari: Editions From the South; 2008.
Authors, 2015
Authors, 2014.
Mastrillo A. Degree Courses of Health Professions. Data on access to courses and programming placed in the A.Y. 2021–2022. Healthcare panorama. Information & analysis of welfare systems. 2021. https://www.panoramasanita.it/wp-content/uploads/2021/11/Report-Mastrillo-2021-def.pdf. Accessed 24.01.2021.
Tuning Project: Reference points for the design and delivery of degree programmes in nursing.2010. http://www.unideusto.org/tuningeu/. Accessed 03/02/3/2021.
World Health Organization: The top 10 causes of death.2020. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death.Accessed 29.06.2021.
Chan TE, Lockhart JS, Thomas A, Kronk R, Schreiber JB. An integrative review of nurse practitioner practice and its relationship to the core competencies. J Prof Nurs. 2020;36(4):189–99. https://doi.org/10.1016/j.profnurs.2019.11.003.
Hodges AL, Konicki AJ, Talley MH, Bordelon CJ, Holland AC, Galin FS. Competency-based education in transitioning nurse practitioner students from education into practice. J Am Assoc Nurse Pract. 2019;31(11):675–82.
Lam CK, Schubert C. Evidence-Based Practice Competence in Nursing Students: An Exploratory Study With Important Implications for Educators. Worldviews Evidence-Based Nurs. 2019;16(2):161–8.
Sulosaari V, Huupponen R, Hupli M, Puukka P, Torniainen K, Leino-Kilpi H. Factors associated with nursing students’ medication competence at the beginning and end of their education. BMC Med Educ. 2015;15(1):1–11. http://dx.doi.org/10.1186/s12909-015-0513-0.
López-Entrambasaguas OM, Martínez-Yebenes R, Calero-García MJ, Granero-Molina J, Martínez-Linares JM. Newly qualified nurses’ perception of their competency achievement on leaving university: A qualitative study. Int J Environ Res Public Health. 2019;16(21).
Romero-Collado A, Raurell-Torreda M, Zabaleta-Del-Olmo E, Rascon-Hernan C, Homs-Romero E. Nurse prescribing in Spain: The law and the curriculum. Nurs Heal Sci. 2017;19(3):373–80.
Caboral-Stevens M, Ignacio R V., Newberry G. Undergraduate nursing students’ pharmacology knowledge and risk of error estimate. Nurse Educ Today. 2020;93:104540. https://doi.org/10.1016/j.nedt.2020.104540
Lewallen LP, Van Horn ER. The state of the science on clinical evaluation in nursing education. Nurs Educ Perspect. 2019;40(1):4–10.
Arli SK, Bakan AB, Ozturk S, Erisik E, Yildirim Z. Critical Thinking and Caring in Nursing Students. Int J Caring Sci. 2017;10(1):471–8.
Couture M, Sasseville M, Gascon V. Facilitators and Barriers to Implementing Transitional Care Managers Within a Public Health Care System. J Gerontol Soc Work.2016;59(4):364–77.
Istat, National Institute of Statistics: Population and families. https://www.istat.it/it/popolazione-e-famiglie (2019). Accessed 10.10.2021.
McPherson C, MacDonald C. Blending simulation-based learning and interpretative pedagogy for undergraduate leadership competency development. J Nurs Educ. 2017;56(1):49–54.
Foronda C, MacWilliams B, McArthur E. Interprofessional communication in healthcare: An integrative review. Nurse Educ Pract. 2016;19:36–40.
Sartain AF, Wright VH. The Effects of Frequent Quizzing on Exam Scores in a Baccalaureate Nursing Course. Nurs Educ Perspect. 2021;42(1).
Klenke-Borgmann L, Cantrell MA, Mariani B. Nurse Educators’ Guide to Clinical Judgment: A Review of Conceptualization, Measurement, and Development. Nurs Educ Perspect. 2020;41(4):215–21.
Taylor I, Bing-Jonsson PC, Johansen E, Levy-Malmberg R, Fagerström L. The Objective Structured Clinical Examination in evolving nurse practitioner education: A study of students’ and examiners’ experiences. Nurse Educ Pract. 2019;37:115–23.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Evaluation of Nursing Competences in the Licensure Exam: an Italian Observational Study

Status:

Version 1

Abstract

Figures

Introduction

Background

Methods

Objectives

Design

Sample and setting

Instruments

Data collection

Data analysis

Results

Discussion

Limitations and future implications

Conclusion

Declarations

References

Additional Declarations

Status:

Version 1