1. The scientificity of the method
Delphi method is a research method used to appraise or predict with the help of experts’ experience and knowledge, and a process in which the opinions of a panel of experts are collected through multiple rounds of questionnaires and put under effective control before a consensus is reached among the experts. The selection of experts and effective feedback lay the foundation of the scientific nature of the research, and the number of experts is recommended to be limited between 10-50. This research adopted Delphi method to establish a framework for the evaluation of healthcare data’s value, with 15 experts in relevant areas invited, all of whom work in areas related with computer information or health care, and 85% of whom are of senior or sub-senior titles, with degrees of or higher than master, and whose working ages higher than 10. The authority coefficient is 0.81, demonstrating that the experts were highly representative and had solid knowledge as well as rich practical experiences. The experts were also highly active, given that the response rates of both two rounds of questionnaires reached 100%, and that the criteria less agreed upon received specific adjustment advice in the advice column. The final draft of the evaluation framework was proven to be scientific, given that the average degree of agreement and the full score ratio increased while the CV decreased to a relatively low level, after the criteria less agreed upon and inconsistently scored were modified based on the results of the questionnaires.
2. The framework and its significance
Following the two rounds Delphi inquiries, this research formulated the finalized framework for the evaluation of healthcare data’s value, which includes two primary criteria, 7 secondary criteria and 21 tertiary criteria. The 2 primary criteria are “inner value” and “value in use”. “Inner value”, which includes 3 secondary criteria: “usable”, “easy to use” and “important”, means that the characteristics inherent of the data are related with the value of healthcare data. Taking one of the secondary criteria, B1 “usable”, for instance, it means that the open data can more easily create bigger value in practice if the data is complete, faithful to the reality and is compliant with national or regional standards. The other primary criterion, “value in use”, means that the value created by data open-up is related with the scenarios in which the data is used. This research summarized four major scenarios in which healthcare data could be used, including “for scientific use”, “for managers to make decisions”, “the individual’s purpose for using the data” and “for commercial use”. Taking one of the secondary criteria, B4 “for scientific use”, for instance, it means that in scientific researches, provided that all ethical requirements are met, the data open-up can create more value if the research and its fund are of high level.
Currently, hospitals and public health institutions in China have accumulated a sea of healthcare data; yet the ability to make use of these data is relatively weak and one of the reasons of that is a lack of principles and guidance pertaining to the open-up and sharing of data. To address this problem, this research established a framework for the evaluation of healthcare data’s value based on the idea of “prioritize the open-up of high-value data”, drawing on the experience of previous researches and the opinions of experts. The final framework can help the managers of data to evaluate data’s value from two aspects, namely, the characteristics and application of data, thus laying a foundation for a scientific and practical healthcare open data guidebook.
3. Limitations and future research
This research initiated an innovative research into the opening-up and sharing of healthcare data, and established a framework for the evaluation of data. Yet there are some limitations concerning this research. Firstly, this research did not assign weights to the tertiary criteria after establishing the framework, so in future researches the author will proceed to assign weights to the current criteria so as to better guide data evaluation in practice. Secondly, data’s value is not the only thing to be considered when opening up healthcare data, given that the risk and cost in the process also should not be ignored. With the basis of this research, the author will conduct further researches while aiming at establishing an open-data model for the evaluation of data from three dimensions, including value, risk and cost, so as to provide a theoretical guidance for the open data movement in the healthcare industry.