Measuring Governance: Developing a Novel Metric for Assessing the Translation of Policies into Action in Nepal

doi:10.21203/rs.2.11618/v1

Download PDF

Research article

Measuring Governance: Developing a Novel Metric for Assessing the Translation of Policies into Action in Nepal

https://doi.org/10.21203/rs.2.11618/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

The Nutrition Governance Indicator (NGI) defines a first standardized approach to quantifying the ‘quality of governance’ in relation to national plans of action to accelerate improvements in nutrition. It was created in response to growing demand for evidence-based measures that reveal opportunities and challenges as nutrition-related policies on paper are translated into outcomes on the ground. Past efforts to measure ‘governance’, most notably the WHO’s Nutrition Governance Indicator and Hunger and Nutrition Commitment Index (HANCI) lack granularity below the national level, and failure to capture pinch points relating to necessary cross-sectoral actions.

The NGI addresses these caveats. This paper introduces the approach that underpins the NGI, discusses its methodology and demonstrates its application in the context of Nepal.

Methods

The NGI was developed using principal components analysis techniques. The resulting score was standardized to a more intuitive scale ranging from 0 to 100, and categorized into quintiles. Cronbach’s alpha was used to test for reliability in addition to testing for goodness of fit using confirmatory factor analysis methods. The data used to derive the index were obtained from 520 participants in the Policy and Science for Health, Agriculture and Nutrition (PoSHAN) study, a prospective and nationally representative longitudinal survey conducted across 21 districts and three agro-ecological zones of Nepal.

Results

The NGI passed validity tests with the exception of two fit indices. Two of the six domains passed the reliability test (Understanding nutrition and Collaboration), the Financial resources domain marginally passed at 0.70 criteria of Nunally, while the remaining domains need re-assessment to improve their reliability. The Nepal Health sector showed the most improvement in mean NGI score over a two-year period followed by the Agricultural sector. About half of all health sector representatives were more likely to have a high score compared to other sectors (OR 0.52, 95% CI 0.36 – 0.74).

Conclusion

Using two panels of data collected in Nepal, this paper has shown that policy-to-action translation can be quantified and applied to sub-national institutions. These findings underscore the practical utility of the index and its importance to the nutrition policy realm.

Nutrition & Dietetics

Metrics

governance

policy

nutrition

Child malnutrition continues to be a significant global public health concern. Roughly 155 million preschool children were recorded in 2016 as being stunted (too short for their age against internal standards), which represents growth failure at some point in their life from conception through to 5 years of age. (1) While the trend is downward at a global level, progress in resolving malnutrition is patchy geographically (with Africa and South Asia lagging behind the rest of the world) and too slow. As a result, generations of children in mostly low-income countries continue to face the many health risks, impaired psychosocial development, impeded educational attainment and longer-term economic hurdles associated with malnutrition.(2)

The international community has responded to this challenge by agreeing to a United Nations-endorsed Decade of Action for Nutrition (2016-2025).(3) Among the targets set for this decade was a 40% reduction in the number of children who are stunted by 2025, to be achieved by an effective consolidation and alignment of international actions and actors.(2),(4). A set of targeted interventions have been agreed, such as accelerating the promotion of exclusive breastfeeding, greater coverage of ante-natal care services, and increasing the access of vulnerable consumers to foods fortified with key vitamins and minerals.(5)

However, while there is broad agreement on many of the technical components of these kinds of programmatic actions for nutrition, it is also widely agreed that successful interventions are ones that are implemented in a conducive or ‘enabling’ policy environment. That is, individual programs are generally cost-effective and sustained if supported by appropriate “structures and policies amenable to project goals.”(6),(7),(8) That is because interventions do not operate in a vacuum; they are implemented in a context that has institutional and individual (human capacity) characteristics that shape how professionals, and non-government stakeholder partners, are able to carry out their responsibilities. This is where “governance” comes in. The ability of countries to effectively translate policies on paper into desired outcomes on the ground is key to the achievement of national goals. Weak governance has been repeatedly identified as a threat to achieving national and global nutrition goals.(9), For example, a recent review of 75 studies on the drivers of effective action by governments found that an inability to implement even well-designed policies for nutrition was often linked to” the absence of institutional ownership for nutrition, and institutional failure.”(10)

Unfortunately, the study of governance (successes or failures) has been hampered by the lack of agreed metrics for empirically measuring the processes involved. Most of the literature pertaining to nutrition governance is based in qualitative interviews with key informants and/or desk reviews of the presence/absence of key policy documents and legislation that would be supportive of national nutrition goals. There have been few attempts to establish more quantifiable metrics that could be compared across country situations or monitored over time.(11),(12),(13) In their review of the state of evidence on processes that underpin political and policy successes for nutrition, Gillespie et al. (2013) concluded that “analyses about how to shape and sustain enabling environments is essential”, and that “the collection of credible metrics …is desperately needed in this area.”(6)

This paper seeks to address that important gap by introducing a novel metric for assessing the effectiveness of nutrition governance at all levels of administrative responsibility, from central level line ministry down to grassroots services providers on the ground. Using empirical data collected in Nepal, we used a principal component analysis (PCA) approach to calculate a nutrition governance index based on participants’ weighted responses on many different facets of governance. We then validated the tool using a confirmatory factor analysis (CFA). The goal of this work is to demonstrate that this new metric offers insights of potential value to both researchers and policymakers keen to gain an improved understanding of how complex multi-institutional policies on paper are translated into practice.

Measuring Nutrition Governance

Defining metrics of governance is a challenge.(6) Decision making within and across governments is typically opaque, decisions usually emerge over time rather than appear fully formed as discrete events, and there are many hurdles to “accessing the many different, geographically widespread actors, individuals, groups and networks involved in policy processes.”(14) As a result, most attempts at standardizing measures of nutrition governance have used national level benchmarks based on available data, such as the presence or absence of certain kinds of policy documents, budgetary allocations, and staffing levels.(15),(16) The three recent and widely cited approaches are i) the World Health Organization’s Nutrition Governance Index, ii) the Hunger and Nutrition Commitment Index (HANCI), and iii) the Political Commitment (for Nutrition) Rapid Assessment Tool. Each is reviewed briefly below.

Nutrition Governance Index

In its global analysis of governments’ readiness to accelerate actions on nutrition, the World Health Organization chose to focus on the concepts of ‘commitment’ (willingness to act) and ‘capacity’ (readiness to act).(16) These parameters were chosen because they provide salient insight into the nature of organizational entities and the characteristics of individuals tasked with delivering improved nutrition at country level. The level of a government’s commitment was assessed in terms of political acknowledgement of the problem, the existence of relevant policies, and resource mobilization at central level (along with budgetary alignment at sub-national level).(16) Capacity was determined in relation to the skills, knowledge, satisfaction and motivation, accountability and freedom of action of individual professionals (staff) within responsible organizations. The review of 54 countries highlighted a huge number of “gaps in the design, content and implementation of policies”, while concluding that a fundamental constraint is the lack of appropriate measures or indicators that would help understand a) roles and responsibilities of individuals and organizations, b) the capacity and areas of competence required of the responsible workforce, and c) metrics of process (not just outcomes) to allow for improved nutrition governance.(17)

A total of 11 indicators were pulled together for 36 low- and middle-income countries with the highest burden of child stunting; the indicators were equally weighted, to generate a simple index from 0 to 11.(18) The “strength of nutrition governance” was arbitrarily classified as ‘weak’ for countries scoring from 0 to 6.9, ‘medium’ for those scoring from 7 to 9.9, and ‘strong’ for those scoring from 10 to 11.0. Using 2007/08 data, 10 countries were classified as having ‘strong’ nutrition governance, including Peru, Malawi and Vietnam (all countries that have made great strides in reducing the prevalence of stunting since the 1990s), while 12 had ‘weak’ nutrition governance, including Cambodia, Mali and Pakistan. Countries facing the persistent challenges of armed conflict were all in the weak category, including Afghanistan, Iraq, Yemen and the Democratic Republic of Congo (DRC).(18)

While the ranking of countries is intuitive, this approach to determining effective governance is limited by its lack of granularity; that is, its inability to differentiate across sectors of government activity or below the national (central) level of government. It is also unable to determine which facets of governance appear to be more, or less, related to policy-driven actions. For example, analysts cannot differentiate among potentially conflicting processes since countries like Cambodia and Ghana identified as having “weak nutrition governance” were still rated as “on track” to meet global nutrition goals, while a state like Burkina Faso was reported as making “no progress” towards such goals despite a ‘strong’ rating for nutrition governance. Furthermore, being based on official data collated at national level there is limited change across years.

Hunger and Nutrition Commitment Index (HANCI)

The HANCI (sometimes shortened in the literature to the Nutrition Commitment Index, or NCI) was developed to rank governments on their political commitment to tackling undernutrition while seeking to measure what governments achieve and where they fail. This index ranked 45 countries according to 22 indicators grouped under 3 themes: public expenditure, policies and programmes, and legislative agendas. Like the WHO metric of nutrition governance, HANCI decouples its indicators from outcomes; that is, it focuses on measures of ‘input’ (such as spending on nutrition) rather than on ‘outcomes’ (such as levels of stunting). Rather than allocating equal weight to all indicators, the authors give greater weights (subjectively) to aspects of governance that appear to have more relevance to nutrition than others – such as granting implementation into law of the International Code of Marketing of Breastmilk Substitutes a higher weight than simply having a dedicated line for nutrition in the national budget.(15)

The 2014 version of HANCI rated Peru, Malawi and Guatemala as the nations with greatest commitment to nutrition; Cambodia, Pakistan, Afghanistan, Yemen and the DRC all ranked poorly (as with the WHO index) as having “very low commitment.”(19) This shows close accord with the WHO rankings, but again suffers the problem of lack of granularity at sub-national level. As the authors themselves have noted, while such metrics offer insight into “the general quality of public administration in a country”, this offers little that is specific to nutrition or to the quality of the process of implementation of policies where they exist.(15)

Political Commitment (for Nutrition) Rapid Assessment Tool

The ‘tool’ developed by Fox et al. (2015) was designed not to rank countries, but to offer deeper insight into individual countries’ depth of political commitment to food security and nutrition.(20) The authors emphasize that “political commitment for food and nutrition is rarely adequately defined or empirically measured”, leading to recommended questions that could be used to assess national commitment to nutrition. These cover not just political level commitment, but also policies, organizational structures, funding for programmes, and the legal and regulatory environment. The draft tool was piloted in 10 low income countries in 2012, and revised in 2013. Some questions are yes/no, with a yes worth a score of 1; other questions allow for a range of relative responses from 1 through 10. For the latter questions (on a scale), a response of 7 or higher was allocated a 1, and for questions relating to budgets (with a 0 to 3), a response of 3 was assigned a 1. The total score possible is 51.

In the test with 10 countries, the Philippines and Columbia were among the highest rated, while Vietnam and Bangladesh fell towards the bottom of the ranking.(20) Unlike the previous two example metrics, however, this approach allows for discrimination among the various elements that make up the final score, making it more useful to any analysis of the elements of governance that are stronger or weaker. Nevertheless, it still relies mainly on an assessment of information that exists at national level that may or may not relate directly to a government’s ability to implement pro-nutrition policies and programmes.

The alternative approach proposed in this paper focuses on an empirical compilation of information derived through in-person surveys with professionals and other stakeholders holding positions and responsibilities for achieving specific nutrition goals. In this sense, the nutrition governance score is more directly based on what people know, think and do in relation to their responsibilities, from sub-national level down to field level. The next section elaborates on the approach used.

The study was conducted as part of an ongoing nationally representative prospective longitudinal study across 21 districts and three agro ecological zones of Nepal. A total of 520 participants, who included government and non-government officials, were selected purposively. The questionnaire comprised of a set of multiple interrelated Likert-scale items that were used to gather data on self-perceived governing practices within respondents’ scope of roles and responsibilities. Each participant answered one of four options: “Strongly Agree”, “Agree”, “Disagree”, and “Strongly disagree” and for some questions, “Don’t Know” or “Not Applicable”. The scale was reconstructed to “Strongly Agree”, “Agree” and “Disagree”, the latter indicating the respondent selected either “Strongly disagree” or “Disagree”. The study was approved by Tufts Social Behavioral and Educational Research Board (SBER) and Nepal Health Research Council (NHRC).

We identified six domains from the data using principal components analysis (PCA). PCA is a data reduction procedure for transforming a large number of variables into a smaller number of uncorrelated (orthogonal) factors, the principal components. These factors account for some of the variation in the data. They are ordered in such a way that the first component captures the most variation. Subsequent components are completely uncorrelated and account for the maximum variation that’s not previously accounted for. All the items in their respective domains loaded positively on the first principal component.

Factor scores were calculated to indicate each participant’s position on the retained components.

The observed items were standardized to a mean of 0 and a variance of 1. These standardized variables are then multiplied by their respective standardized scoring coefficients. The products are then summed over all the variables per domain and the sum is the value of the factor score. This was calculated in SAS using the SCORE function in the PROC FACTOR procedure.

Because the retained components account for varying levels of the total variance in the data, we accounted for this difference in their importance by using equation 01 below (21):

For example, for each individual, the unstandardized, weighted Nutrition Governance Index is calculated as:

The standardized NGI score ranges from 0 to 100 with higher scores indicating better nutrition governance.

Validity and Internal consistency

Cronbach’s alpha coefficient was calculated to determine internal consistency. Reliability tests assume that the unidimensionality assumption holds therefore these tests were conducted by their domains as opposed to across all domains. Internal consistency describes the extent to which all items in a test measure the same underlying trait (22). These estimates range from -1 to +1, with higher absolute values indicating higher reliability. It’s also a test for measurement error: as reliability increases, measurement error decreases. A criterion of 0.7 and above is generally considered acceptable.(23)

Applying the same, only two domains (Domain 1 and 2) are reliable, domains 3 and 4 are borderline reliable and 5 and 6 are unreliable (Table 1). Low alpha values imply that the domains require re-assessment to either increase the number of items or replace them altogether.

Construct validity

The construct validity for the NGI was examined using Confirmatory Factor Analysis (CFA). This technique was used to verify the factor structure imposed on the data, that is, whether the six domains obtained were a good fit. Table 2 shows the goodness of fit indices.

The c²value obtained was significant which implied that the data did not fit the model well. The Goodness of Fit test was developed as an alternative to the chi-square test. It calculates the proportion of variance accounted for by the estimated population covariance.

The Root Mean Square error of approximation (RMSEA) is a calculation based on the c²and sample size. It tells us how well the model, with unknown but optimally chosen parameter estimates would fit the populations covariance matrix (24). A value of 0 indicates perfect fit (25) with a recommended value of <= 0.06 as a cut-off for good fit. Similar to the RMSEA, a Standardized Root Mean Square Residual (SRMR) value of 0 indicates perfect fit and a cut-off of <= 0.08 is indicative of a good fit.

The null model for Bentler’s comparative fit index assumes that all latent variables are uncorrelated and compares this model to the sample covariance matrix. Values range from 0 to 1.0, with values closer to 1.0 indicating a good fit. A value >= 0.95 is recognized as the cut-off for a good fit (24).

Overall, the model is a good fit for the data. Whereas the model didn’t pass the chi-square and Bentler’s comparative fit index, it passed all the other indices thus confirming the factor structure imposed. Nevertheless, based on relatively poor reliability for some domains and failure to meet some fit indices, we recommend re-assessing the items in domains with low alpha values to improve overall model fit.

Ranking Sub-national sectors

Figure 1 summarizes performance of various groups based on their median NGI score in 2014 and 2016. The gap between the two time points intuitively reveals the extent of improvement across time; the larger the distance to the right between points of the same group, the greater the improvement in governance. Furthermore, Table 3 shows percentage distribution of respondents categorized by NGI rank.

The Health sector showed the most improvement in mean NGI score followed by the Agriculture sector. About fifty percent (50%) of respondents from the health sector were more likely to have a high score (4 or 5) compared to other sectors (OR 0.52, 95% CI 0.36 – 0.74). This is an important finding because the Health sector is at the forefront of all health related and some agriculturally relevant activities including operationalizing and advocating for maternal and child nutrition programs.

Ranking the Multi-Sector Nutrition Plan (MSNP)

The Multi-Sector Nutrition Plan (MSNP) is an initiative implemented by the Nepal government in collaboration with various international partners including UNICEF, USAID and the World Bank. They altogether aim to improve the nutritional status in Nepal, especially in mothers and children. Two districts, Jumla and Nawalparasi, were in the MSNP catchment area at the time of data collection. Only 31% of respondents exposed to MSNP initiatives were ranked in the highest scoring categories (4 and 5) compared to 41% not exposed (OR 1.57 95% CI 0.844 – 2.90). Furthermore, across time, the mean NGI score was lower for respondents from these districts. This finding serves 1) to show MSNP policy makers that there are areas of improvement to be explored and 2) to avail concerned parties a tool against which they can measure MSNP performance.

Ranking staff trained in nutrition related activities

Building local capacity by putting an emphasis on attending nutrition courses and training improved nutrition governance (OR 1.40, 95% CI 0.79-2.48). Training at the implementation level raises more awareness of the nutrition issues. Trained staff have a higher likelihood to understand the effectiveness of inter-sectoral and cross-sectoral co-ordination. Similarly, the more extensive the work experience, the higher the likelihood of performing better on the job and the greater the score on the NGI.

Discussion and Limitations of the Nutrition Governance Index

The newly created index is unique in that it assesses nutrition governance at the sub-national level, which is of great potential value to both researchers and policymakers that are keen on gaining an improved understanding of how complex multi-institutional policies on paper are translated into practice. The index assesses six different domains of nutrition governance; i) understanding nutrition and responsibilities, ii) collaboration within and across offices, iii) access to financial resources, iv) nutrition leadership, v) capacity and vi) coordination and support across and between sectors.

The tool allows for assessment of sub-national groups and individuals on how they perceive implementation of nutrition policies within their official duties. A high score on the NGI may not necessarily mean a high score on all of the domains it constitutes thus sub-national levels should be reviewed by NGI domain in order to gain better insight into their performance.

The same questionnaire was used for each respondent regardless of management roles and administrative level. Whereas respondents differed in their responsibilities, the items were broad enough to be relevant across levels yet specific enough to apply to singular roles. Table 4 below shows percentage distribution of responses at the four management levels.

An analysis of the response pattern showed that there was agreement across all administrative levels with the exception of Item 15, which inquired about access to budgetary resources. The disproportionately low percentage of respondents who strongly disagreed at the lower levels of management could be explained in two ways; 1) Their roles do not involve handling budgets so the question is irrelevant 2) They genuinely strongly disagree with the item. All the 13 respondents who opted for either the “Don’t Know” or “Not applicable” categories were from the Ward level which points us to the first explanation. A similar analysis of missing value patterns showed that Item 3 and Item 16 may not be applicable to lower levels of management, therefore rating them using these items reflects an unfair sense of judgement in their overall NGI score. Results therefore should be interpreted with caution.

The sampling strategy used for these analyses was purposive in design. Although effective in achieving meaningful results (when well executed), this strategy can be problematic when making inferences. Inferential statistics allow for generalization of results to a much larger population using sampled data. Whereas we sampled from across the three major agro-ecological regions in Nepal, participants were sampled non-probabilistically on a smaller scale within each zone thus there would have been a selection bias if perceptions of governing practices captured in this study differ from participants’ not included in the study. Consequently, the results obtained in this study may not be applicable nationwide or on a much greater scope due to limited representation.

The process of item selection was iterative in practice. The first round of data collection was qualitative because we took an exploratory approach that allowed us to gain a broad understanding of governing practices on the ground. At each subsequent round, items were refined using PCA, which resulted in replacing uncorrelated items with new, relevant items under each domain. We used the fourth round of data collection in this paper, to construct the nutrition governance indicator which poses the following limitations; 1) some of the items were not asked in previous rounds therefore the outcome of the NGI is dependent on the round of data collection 2) it may be problematic to compare trends in NGI across rounds.

As aforementioned, the first principal component accounts for the most variation but it explains only a small proportion of all the variation under each domain. Although all the items under each domain fell on a single component, any new items added may not. The use of only one component is therefore restricting as it omits a great wealth of information in the final index and implies unidimensionality where it may not exist. However, the aim was to obtain a single index representing each domain therefore this approach was necessary. In addition, since we only considered the factor with the highest variation, there won’t be significant benefit in adding higher-order factors as they tend to explain much smaller proportions of variation.

The findings from the multi-year PoSHAN policy survey showed that different groups/institutions performed differently against the index. The health sector, in particular, did well compared to six other sectors. Additional groups that performed well include respondents that were trained in nutrition-related activities and education, respondents that had over 2 years of work experience and officials from the district administrative level. We identified MSNP exposure as a target for improvement. This may be regarded as a significant finding to improve service delivery in these districts.

AGFI – Adjusted Goodness of Fit Index

CFA - Confirmatory Factor Analysis

DRC – Democratic Republic of Congo

GFI - Goodness of fit index

HANCI - Hunger and Nutrition Commitment Index

MSNP – Multisector Nutrition Plan

NGI – Nutrition Governance Index

NHRC - Nepal Health Research Council

PCA – Principal Components Analysis

PoSHAN – Policy and Science for Health, Agriculture and Nutrition

RMSEA- Root Mean Square Error of Approximation

SBER - Social Behavioral and Educational Research Board

SRMR - Standardized Root Mean Square Residual

UNICEF – United Nations International Children’s Emergency Fund

USAID – United States Agency for International Development

VDC – Village Development Committee

WHO – World Health Organization

Ethics approval and consent to participate

We obtained written informed consent in accordance with Tufts Social Behavioral and Educational Research Board (SBER) and Nepal Health Research Council (NHRC) before participants were enrolled.

Consent for publication

N/A

Availability of data and material

The datasets generated and/or analyzed during the current study are not publicly available yet but are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This study was funded under grant contract AID-OAA-L-10-00006 from the United States Agency for International Development (USAID). The funder did not have influence on the outcome or conduct of this study.

Authors' contributions

PW was a major contributor to the writing and interpretation of results, GN and RS were key contributors to the writing of the manuscript. GN performed the data analyses and consulted with RH. SG, DD and KB read and approved the final manuscript together with all co-authors.

Acknowledgements

Special thanks go to the PoSHAN data collection team and participants for their time and collaboration. We are particularly grateful to the Nutrition Innovation Lab team for their support and input over the course of several meetings, in which issues presented in this paper were zealously discussed.

Authors' information (optional)

N/A

WHO | UNICEF-WHO-The World Bank: Joint child malnutrition estimates - Levels and trends [Internet]. WHO. [cited 2018 May 16]. Available from: http://www.who.int/nutgrowthdb/estimates/en/
Hawkes C, Fanzo J. Nourishing the SDGs: Global Nutrition Report 2017. 2017;
WHO | The UN Decade of Action on Nutrition: Working together to implement the outcomes of the Second International Conference on Nutrition [Internet]. WHO. [cited 2018 May 16]. Available from: http://www.who.int/nutrition/events/2016_UNGA71_side-event_20Sept_NY/en/
Organization WH. Global nutrition targets 2025: Policy brief series. 2014;
Giroux H. Theories of reproduction and resistance in the new sociology of education: A critical analysis. Harv Educ Rev. 1983;53(3):257–293.
Gillespie S, Haddad L, Mannar V, Menon P, Nisbett N. The politics of reducing malnutrition: building commitment and accelerating progress. The Lancet. 2013 Aug 10;382(9891):552–69.
Heaver R. Strengthening Country Commitment to Human Development. Lessons from Nutrition [Internet]. [cited 2018 Aug 21]. Available from: http://documents.worldbank.org/curated/en/685551468778783254/pdf/31674.pdf
Fanzo J. Andrés Mejía Acosta Institute of Development Studies. :39.
Sunguya BF, Ong KI, Dhakal S, Mlunde LB, Shibanuma A, Yasuoka J, et al. Strong nutrition governance is a key to addressing nutrition transition in low and middle-income countries: review of countries’ nutrition policies. Nutr J. 2014 Jun 27;13:65.
Baker P, Hawkes C, Wingrove K, Demaio AR, Parkhurst J, Thow AM, et al. What drives political commitment for nutrition? A review and framework synthesis to inform the United Nations Decade of Action on Nutrition. BMJ Glob Health [Internet]. 2018 Feb 10;3(1). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5841521/
Research Institute (IFPRI) IFP. Working multisectorally in nutrition Principles, practices, and case studies [Internet]. Washington, DC: International Food Policy Research Institute; 2011 [cited 2018 May 17]. Available from: http://ebrary.ifpri.org/cdm/ref/collection/p15738coll2/id/129738
Exworthy M. Policy to tackle the social determinants of health: using conceptual models to understand the policy process. Health Policy Plan. 2008 Sep 1;23(5):318–27.
Pelletier DL, Frongillo EA, Gervais S, Hoey L, Menon P, Ngo T, et al. Nutrition agenda setting, policy formulation and implementation: lessons from the Mainstreaming Nutrition Initiative. Health Policy Plan. 2012 Jan 1;27(1):19–31.
Walt G, Shiffman J, Schneider H, Murray SF, Brugha R, Gilson L. ‘Doing’ health policy analysis: methodological and conceptual reflections and challenges. Health Policy Plan. 2008 Sep;23(5):308–17.
The Hunger and Nutrition Commitment Index (HANCI 2014): Measuring the Political Commitment to Reduce Hunger and Undernutrition in Developing Countries [Internet]. [cited 2018 May 17]. Available from: http://www.ids.ac.uk/publication/the-hunger-and-nutrition-commitment-index-hanci-2014-measuring-the-political-commitment-to-reduce-hunger-and-undernutrition-in-developing-countries
Organization WH. Landscape analysis on countries’ readiness to accelerate action in nutrition: country assessment tools. 2012;
Organization WH. Global nutrition policy review: what does it take to scale up nutrition action? 2013;
Engesveen K, Nishida C, Prudhon C, Shrimpton R. Assessing countries’ commitment to accelerate nutrition action demonstrated in PRSPs, UNDAFs and through nutrition governance. SCN News. 2009;(No.37):10–6.
HANCI Global | Hunger and Nutrition Commitment Index [Internet]. [cited 2018 May 17]. Available from: http://www.hancindex.org/hanci/#.Wv2sF6QvxhE
Fox AM, Balarajan Y, Cheng C, Reich MR. MEASURING POLITICAL COMMITMENT FOR FOOD AND NUTRITION SECURITY.
Antony GM, Visweswara Rao K. A composite index to explain variations in poverty, health, nutritional status and standard of living: Use of multivariate statistical methods. Public Health. 2007 Aug;121(8):578–87.
Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011 Jun 27;2:53–5.
Panayides P. Coefficient Alpha: Interpret With Caution. Eur J Psychol. 2013 Nov 29;9(4):687–96.
Hooper D, Coughlan J, Mullen M. Structural equation modelling: Guidelines for determining model fit. Articles. 2008;2.
Hu L, Bentler PM. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychol Methods. 1998;3(4):424.

Table 1. Items under each domain and the corresponding reliability coefficient for each domain

DOMAIN	CRONBACH'S ALPHA
DOMAIN 1: UNDERSTANDING NUTRITION AND THEIR RESPONSIBILITIES	0.83
Adequate understanding of nutrition problems to be able to implement strategies
Sufficient discussion among office colleagues on how to implement strategies
Nutrition is taken formally into consideration in annual plans and budgets
Personal responsibilities related to nutrition are clearly defined
They know when own actions have been successful or effective
Work-related decisions are based on hard data/technical evidence in nutrition
Most colleagues consider nutrition a priority for them to work on
Improving nutrition is one of the responsibilities of their sector
DOMAIN 2: COLLABORATION	0.83
There is effective collaboration across offices in addressing nutrition issues
Supervisors actively promote collaboration with other offices
There is sufficient sharing of information about nutrition plans and activities
Necessary stakeholders are included in discussions to address nutrition issues
There was demand from other ministries for their offices to collaborate with them
They effectively collaborated with their coworkers to address nutrition problems
DOMAIN 3: FINANCIAL RESOURCES	0.68
They personally have sufficient access to budgetary resources to be effective in their roles
Their office has sufficient financial resources to implement actions to meet their roles
DOMAIN 4: NUTRITION LEADERSHIP	0.65
They have a champion for nutrition in their sector
There is clear leadership on nutrition in their sector
DOMAIN 5: CAPACITY	0.6
They have personally been adequately trained to carry out their responsibilities
Their own colleagues have the right skills or training to be effective in their work
DOMAIN 6: SUPPORT	0.51
They know how to obtain any technical support for their responsibilities
They have adequate support from their supervisors for implementing their roles
They have adequate commitment from colleagues in their sector to help fulfill their roles
Sufficient non-financial resources are made available

Table 2. Goodness of fit statistics for Confirmatory Factor Analysis

Fit Summary	Goodness of fit measure	Fit estimate	Cut-off limit	Pass or Fail
Absolute Index	Fit Function	1.0866
	Chi-Square	488.958
	Pr > Chi-Square	<.0001		Fail
	Standardized RMR (SRMR)	0.0463	<0.08	Pass
	Goodness of fit index(GFI)	0.9165	>0.90	Pass
Parsimony index	Adjusted GFI(AGFI)	0.8943	>0.90	Pass
	RMSEA Estimate	0.0486	<0.06	Pass
Incremental index	Bentler Comparative Fit Index	0.9246	>=0.95	Fail

Table 3: Percentage distribution of respondent groups by the NGI ranks

	NGI RANKS(N= 520)
	1	2	3	4	5
Ministries
Agriculture	13.64	20.45	20.45	26.14	19.32
Health	12.68	15.49	22.54	23.94	25.35
Non-Health	29.68	24.2	17.35	13.7	15.07
Level of management
District	12.12	22.22	19.19	22.73	23.74
Ilaka	14.52	9.68	32.26	22.58	20.97
VDC	25.93	23.46	17.28	14.81	18.52
Ward	27.93	19.55	17.88	18.44	16.2
MSNP membership
No	19.87	20.09	19.02	20.09	20.94
Yes	21.15	19.23	28.85	19.23	11.54
Nutrition training/courses received
No	29.75	21.86	18.28	15.41	14.7
Yes	8.71	17.84	21.99	25.31	26.14
Length in current position
Less than 1 month	0	50	50	0	0
2 years or less	22.96	21.4	19.84	15.18	20.62
Over 2 years	17.24	18.39	19.92	24.9	19.54
Level of education
Bachelors	16.05	19.75	23.46	20.99	19.75
Intermediate (10+2)	24.05	20.25	21.52	18.99	15.19
Just literate (Non formal Education)	21.05	23.68	18.42	23.68	13.16
Masters	11.19	22.38	21.68	20.98	23.78
PhD	0	0	100	0	0
Primary (up to Grade 5)	29.73	18.92	16.22	18.92	16.22
Secondary (Grade 6-10)	31.73	19.23	15.38	15.38	18.27
Technical degree	10.81	10.81	18.92	27.03	32.43

Table 4. Percentage distribution of responses at the four management levels.

		DISTRICT (n = 198)			ILAKA (n = 62)			VDC ( n = 81)			WARD (n = 179)
Item	0*	0.75	1	0	0.75	1	0	0.75	1	0	0.75	1
1	9%	79%	12%	2%	87%	11%	10%	81%	9%	6%	82%	12%
2	17%	69%	14%	16%	77%	6%	20%	74%	6%	20%	69%	12%
3	21%	62%	16%	19%	71%	8%	31%	57%	9%	34%	53%	2%
4	26%	61%	12%	18%	71%	11%	33%	51%	16%	27%	60%	12%
5	13%	75%	12%	5%	77%	18%	7%	84%	9%	10%	76%	14%
6	18%	68%	15%	10%	77%	13%	20%	73%	7%	17%	66%	15%
7	33%	60%	7%	13%	76%	10%	21%	74%	5%	17%	73%	9%
8	2%	45%	53%	2%	18%	81%	0%	42%	58%	1%	49%	51%
9	22%	75%	4%	35%	55%	6%	26%	70%	2%	35%	56%	7%
10	22%	67%	11%	18%	69%	11%	16%	78%	6%	16%	77%	6%
11	13%	75%	12%	13%	71%	16%	12%	75%	12%	19%	69%	12%
12	27%	69%	3%	39%	55%	5%	30%	69%	1%	28%	64%	3%
13	17%	74%	9%	39%	58%	2%	31%	68%	1%	26%	66%	7%
14	29%	66%	3%	34%	60%	2%	31%	63%	5%	35%	54%	4%
15	21%	67%	13%	35%	63%	2%	47%	48%	5%	64%	28%	1%
16	38%	55%	7%	65%	26%	10%	60%	36%	4%	69%	22%	1%
17	64%	32%	4%	45%	50%	5%	75%	23%	1%	58%	35%	3%
18	39%	54%	8%	21%	73%	5%	42%	54%	2%	33%	60%	6%
19	35%	52%	13%	39%	53%	8%	43%	49%	7%	44%	46%	10%
20	43%	54%	3%	40%	52%	8%	41%	51%	9%	44%	51%	4%
21	3%	56%	42%	0%	63%	37%	2%	63%	35%	2%	79%	20%
22	3%	74%	23%	0%	73%	27%	9%	72%	20%	4%	74%	22%
23	1%	78%	21%	0%	73%	27%	1%	78%	21%	2%	83%	15%
24	7%	81%	12%	6%	85%	8%	11%	83%	6%	10%	84%	6%

*0=Disagree 0.75 = Agree 1 =Strongly agree

We attributed an arbitrary penalty of 0.25 to “Agree” responses in order to account for the loss of certainty. This ensured that these responses carried less weight in the PCA procedure compared to “Strongly agree” responses. The lesser degree in agreement was represented in the factor scores created resulting in more robust principal components. A larger penalty (0.5) would insinuate that a slight shift from strong agreement reduces the positive effect by half – an assumption too strong to make

Download PDF

Version 1

posted

You are reading this latest preprint version

Measuring Governance: Developing a Novel Metric for Assessing the Translation of Policies into Action in Nepal

Status:

Version 1

Abstract

Figures

Background

Methods

Results

Conclusions

Abbreviations

Declarations

References

Tables

Status:

Version 1