Out of 97 recommendations produced by CBR model which uses the logical flowchart in Fig. 3, only 92% of the recommendations generated by the CBR model which uses CBR similarity formula have similarity score ≥ 80%, and only 42% generated by the CBR model which uses CCBR similarity formula, as shown in Fig. 5. Furthermore, the model’s recommendations were validated by the 2 fertility subspecialist doctors to ensure that they match the fertility subspecialist doctors’ recommendations, as shown in Fig. 6.
The two fertility subspecialist doctors agreed that 92% of recommendations generated by the CBR model which uses CBR similarity formula had similarity score ≥ 80%, yet only 27% of these recommendations were declared valid, and 42% of the recommendations proposed by the CBR model which uses CCBR similarity formula had similarity score ≥ 80%, yet 100% of them were declared valid, as shown in in Fig. 7. The validation of fertility subspecialist doctors proves that not every one of the model’s recommendations with similarity score ≥ 80% match those that fertility subspecialist doctors would make, only recommendations generated by the CBR model which uses CCBR similarity formula are trustworthy. The excellent performance that the CBR model which uses CCBR similarity formula achieves is because of the performance of CCBR similarity formula which adopts Dekang Lin’s[63] similarity theory, making it more precise in calculating the similarity score.
The validation from the fertility subspecialist doctors proves that CBR model which uses CCBR similarity formula has the best performance and information in Fig. 8 is the result of performance measurement using Confusion Matrix.
The excellent performance that the CBR model which uses CCBR similarity formula achieves is also shown in the result of validation by the fertility subspecialist doctors who suggests that out of the recommendations with similarity score < 80% (information in Fig. 5), 82% of them are declared valid, as shown in Fig. 9. However, any recommendation with similarity score < 80% cannot be used since this research finds that the highest similarity score for invalid result from the CBR model which uses CCBR similarity formula is 79.7%.
In the CBR model, to deal with the recommendations with similarity score < 80%, the fertility subspecialist doctors need to go to revision stage to validate the recommendations. The reality is that fertility subspecialist doctors have already had high workload[1, 8, 11, 12], and this makes them unable to immediately revise the CBR model’s recommendations. On the other hand, if IVF program patients wait for the reply for too long, it might increase their anxiety level[1, 6] and adversely affect the IVF program they are participating in[1, 4, 5, 15–17]. To handle this, the CBR model was modified by combining CBR model and RBR model[43–46], following the logical flowchart in Fig. 4. The two fertility subspecialist doctors were still involved to validate the model’s recommendations and the performance was measured using Confusion Matrix, as shown in Fig. 10.
The result of Confusion Matrix measurement proves that the combination between CBR model and RBR model can improve the CBR model’s performance. The RBR model’s success to improve the CBR model’s performance is because the Rule-Based Reasoning properly adopts the way the 2 experienced fertility subspecialist doctors work in dealing with IVF program patient’s complaints and uses CCBR similarity formula. The highest improvement of accuracy score is shown by the combination of CBR model which uses CCBR similarity formula and RBR model at 47% with the precision score staying at 100% (Fig. 8) or without any increase (0%). The improved accuracy score and the high precision score proves that the combination between CBR model which uses CCBR similarity formula and RBR model can actually help the fertility subspecialist doctors deal with the revision stage of CBR model and accurately reply the IVF program patient’s anxious questions.
Considering some factors, such as: >78% Indonesians actively using mobile phone to access the Internet, fertility subspecialist doctor-IVF program patient ratio still below WHO’s standard[14] and the excellent performance of the combination between CBR model (using CCBR formula) and RBR model, this research recommends an architecture of IVF program patient’s anxious question replier smart system as shown in Fig. 11. Based on the logical flowchart in Fig. 2, an algorithm of combination between CBR model (using CCBR similarity formula) and RBR model which complements the information of architecture of IVF program patient’s anxious question replier smart system as shown in Fig. 12 is built.
The logical model applied to build the smart system software is the combination between CBR model (using CCBR similarity formula and applying the standard score of ≥ 80% for recommendation accuracy level) and RBR model. The smart system software model uses the hybrid model (web and mobile applications), hence the smart system users (fertility subspecialist doctors, IVF program patients, and hospital medical staff) can access the smart system software via the Internet network connected to many devices they use, such as computer, notebook or smartphone. To build the database and web technologies in the smart system, it is recommended to use opensource software which has some strengths, such as: being used by many programmers, having high reliability and good quality[65].