Our team diligently adhered to the standard workflow of DIN6, meticulously following each step of the process: Initiate, Develop, Implement, and Exploit (Figure 1). Throughout this entire workflow, we maintained a relentless focus on innovation, collaboration, and excellence, ensuring that each step was executed with precision and effectiveness.
1. Initiate:
We began by thoroughly understanding the problem domain and identifying opportunities for innovation. This phase involved brainstorming sessions, and stakeholder consultations to define clear objectives and strategies. The main problem in microRNA-disease databases is that they lack link to the biomarker development pipeline (clinical validity stage). This link will optimize the clinical value of these user-friendly miRNA-disease databases toward the clinical utility.
2. Develop: M3Cs Standardized Clinical Validity Framework
With a solid foundation established during the initiation phase, we proceeded to develop M3Cs standardized clinical validity framework. This stage encompassed testing to refine concepts and ensure alignment with the project goal and user requirements.
The M3Cs standardized clinical validity framework has been developed and obtained final consensus agreement from a panel of experts. The M3Cs standardized clinical validity framework consists of 4 steps:
Step 1: Evaluate the quality elements for individual studies as Yes, No, Partially, Unclear, or Not Relevant [adapted from ASCO guidelines development process7 and CPIC8]
· Sample size adequate to assess difference between compared groups, especially for negative findings9.
· Adequate methods used:
o Appropriate attainment of samples.
o Clear identification of the time points and the source of the collected samples.
o State all approaches used (profiling/discovery or candidate-miRNA identification approaches).
o Adequate platform used in the profiling/discovery stage.
o State all miRNAs screened in candidate-miRNA identification approach.
o State miRNA status as “single or panel”.
o States all genetic variants screened (if applicable).
o Adequate phenotyping or genotyping method used (if applicable).
o Alleles tested are adequate to determine “wild-type” genotype (if applicable).
o Clearly states which miRNA dysregulation were found in the study.
o Clearly states which genotypes were found in the study (if applicable).
o Defines how * alleles defined (if applicable).
· Ancestry is discussed and appropriately considered.
· Outcome definition clearly defined and measured.
· Appropriate statistics was performed.
Step 2: Rate how the individual study supports the major finding statement. [adapted from ASCO guidelines development process7 and CPIC8]
1. First, determine if the study supports the major finding statement or does not support it.
2. Second, qualify the statement (if needed) based on the quality elements listed above:
Some study quality flaws: Enough of the items in step 1 are rated “partially,” “unclear,” or “no” to introduce some uncertainty about the validity of the conclusions.
Major study quality flaws: Enough of the items in step 1 are rated “partially,” “unclear,” or “no” to introduce serious uncertainty about the validity of the conclusions.
No qualification on statement is needed if few items in step 1 are rated as “partially,” “unclear,” or “no.”
· There are six possible ratings:
o Supports the statement.
o Supports the statement but with some quality flaws.
o Supports the statement but with major quality flaws.
o Does not support the statement.
o Does not support the statement but with some quality flaws.
o Does not support the statement but with major quality flaws.
Step 3: Rate how preclinical in vitro or in vivo studies support the major finding statement:
· Supports the statement.
· Supports the statement but with some quality flaws.
· Supports the statement but with major quality flaws.
· Does not support the statement.
· Does not support the statement but with some quality flaws.
· Does not support the statement but with major quality flaws.
Step 4: Score each major finding statement based on all the evidence and studies that support the major finding using the following criteria. Scale modified slightly from Valdes et al.10 and the ASCO guideline7.
• Strong: Evidence includes consistent results from well-designed, well-conducted studies. “High confidence that the available evidence reflects the true magnitude and direction of the net effect and further research is very unlikely to change the magnitude or direction of this net effect.”
• Moderate: Evidence is sufficient to determine effects, but the strength of the evidence is limited by the number, quality, or consistency of the individual studies; generalizability to routine practice; or indirect nature of the evidence. “Further research is unlikely to alter the direction of the net effect, however it might alter the magnitude of the net effect.”
• Limited: Evidence is insufficient to assess the effects on health outcomes because of limited number or power of studies, important flaws in their design or conduct, gaps in the chain of evidence, or lack of information. “Further research may change the magnitude and/or direction of the net effect.”
3. Implement:
Upon finalizing the development phase, we transitioned into implementation, where we executed our plans and brought our DIN to life with central validation at the end of this step. It has two stages:
Stage (I): M3Cs version 2.0 Development.
M3Cs V.2.0 was developed based on the whole data from M3Cs (https://m3cs.shinyapps.io/M3Cs/). First, we downloaded data in a tabular form (studies published between 2002-2022). Second, we removed retracted papers. Third, we followed the same 3 steps of data processing in M3Cs with central validation at the end of each phase3 for studies published in 2023 as an update. In M3Cs V.2.0, we have added new features. We added data-driven in vivo 'Animal' model approach to the current two approaches in M3Cs. Data-driven in vivo 'Animal' approach included two domains: In Vivo domain and In Vivo & Drug domain. The inclusion criteria of miRNA studies in this approach were (i) Publications used animal model injected with human cell line of pediatric origin (<18 years) with verification from Cellosaurus11; (ii) Publications investigated the miRNA effect on the tumor phenotype. MiRNA studies were excluded in this approach if they were: (1) Reports, reviews or letters without primary data; (2) Non-English publications; (3) Retracted papers. We have added the two domains of data-driven in vivo 'Animal' model approach to the M3Cs V2.0 manual-based data curation tool. The presence of the three approaches in M3Cs V.2.0 (Figure 2) helps in the application of the steps of the clinical validity framework as we will discuss in the next section.
Stage (II): Integration of M3Cs Clinical Validity Framework in the clinical domain of M3Cs Version 2.0.
We utilized digital innovation, as a creative and agile problem solving that are essential for successful integration of the ‘M3Cs clinical validity framework’ in the clinical domain of M3Cs (after its update and the inclusion of the data-driven in vivo 'animal' model approach) to deliver a unique value microRNA platform ‘M3Cs V.2.0’. The assertion criteria needed to obtain a given categorization within the framework are described for each clinical validity classification in table 6 and central validation was done at the end of this phase.
Table 6. Represents ‘Clinical Validity’ summary matrix with the assertion criteria.
|
Type and Level of Evidence
|
Clinical Validity of Diagnostic miRNA
|
Clinical Validity of Prognostic miRNA
|
Clinical Validity of Predictive miRNA
|
Clinical Validity of Susceptibility/Risk miRNA
|
|
(I)- Supportive Evidence
|
Assertion Criteria
|
|
a) Strong
|
Evidence includes consistent results from well-designed, well-conducted studies. “High confidence that the available evidence reflects the true magnitude and direction of the net effect and further research is very unlikely to change the magnitude or direction of this net effect.”
|
|
b) Moderate
|
Evidence is sufficient to determine effects, but the strength of the evidence is limited by the number, quality, or consistency of the individual studies; generalizability to routine practice; or indirect nature of the evidence. “Further research is unlikely to alter the direction of the net effect, however it might alter the magnitude of the net effect.”
|
|
c) Limited
|
Evidence is insufficient to assess the effects on health outcomes because of limited number or power of studies, important flaws in their design or conduct, gaps in the chain of evidence, or lack of information. “Further research may change the magnitude and/or direction of the net effect.”
|
|
(II)- No Known Trait/Trait subtype Relationship
|
Not Reported
|
|
(III)- Contradictory Evidence
|
The presence of miRNA-level at least 2 clinical evidence OR clinical and experimental evidence with contradictory results
|
4. Exploit: Web Development
Finally, we embarked on the exploitation phase, where we strategically leverage our digital innovation to create tangible value and drive outcomes. Central validation was done at the end of this step.
As in M3Cs platform, M3Cs V.2.0 followed the same step of web development3. Shinyapps.io platform was used as a web interface for the deployment of the M3Cs V.2.0 application. Also, we have introduced a user-friendly visualization of the top miRNAs for each trait within each domain, providing researchers with an efficient means of identifying important miRNAs (Figure 3).
Limitations
Although the innovation, here, is scalable in both adding new functionality and being spread across other miRNA-disease databases considering the potential mutations in some pediatric cancer cases may affect the functionality of miRNA and subsequently the associated disease.