The RSSD methodology was tested by the respondents (software developers) in 10 projects. The project timeline is shown in
Table 3. On average, the proportion of each phase of RSSD is shown in Fig. 3. The evaluation and maintenance phase was the largest part of the project, reaching 42.1% of the total project, while the development phase was the second largest (35.09%).
Table 3
RSSD Implementation Result on the Software Development Timeline
Case #
|
Dev #
|
Software Name
|
Language
|
Total Days
|
Development Duration (% of total days)
|
Meet *
|
Plan & Pre-Ev
|
Design
|
Develop
|
Test
|
Eval. & Maint.**
|
C1
|
D1
|
Academic information system
|
PHP + JS
|
194
|
7.7 (2)
|
4.6
|
5.2
|
35.1
|
5.7
|
41.8 (6)
|
C2
|
D1
|
Data analytics for eCommerce
|
PHP + JS
|
202
|
8.9 (3)
|
5
|
6.4
|
30.2
|
4
|
45.5 (4)
|
C3
|
D2
|
Clinic information system
|
PHP + JS (Laravel)
|
129
|
5.4(2)
|
6.2
|
5.4
|
34.9
|
3.9
|
44.2 (5)
|
C4
|
D2
|
Dental clinic system
|
PHP + JS (Laravel)
|
115
|
8.7 (2)
|
7
|
5.2
|
29.6
|
5.2
|
44.3 (4)
|
C5
|
D3
|
Casual game
|
Unity C#
|
130
|
5.4(2)
|
6.9
|
5.4
|
26.9
|
9.2
|
46.2 (3)
|
C6
|
D4
|
Online poker game
|
Unity C#
|
256
|
5.5 (2)
|
4.3
|
2.7
|
31.3
|
6.3
|
50 (10)
|
C7
|
D5
|
Point of sales (metal industry)
|
PHP + JS
|
175
|
0.6(1)
|
4.6
|
4.6
|
38.9
|
5.1
|
46.3 (6)
|
C8
|
D5
|
Point of sales (cafe)
|
PHP + JS
|
154
|
4.5(2)
|
5.2
|
5.8
|
46.8
|
6.5
|
31.2 (3)
|
C9
|
D6
|
Health mobile app
|
HTML + JS (Cordova)
|
199
|
10.6 (5)
|
6
|
5.5
|
40.7
|
5
|
32.2 (4)
|
C10
|
D7
|
Attendance system
|
Android & Web
|
178
|
9 (3)
|
4.5
|
5.6
|
36.5
|
5.1
|
39.3 (7)
|
|
|
|
Average
|
173.2
|
6.63%
|
5.43%
|
5.18%
|
35.09%
|
5.6%
|
42.10%
|
* Meet phase: Duration (number of meetings). The Meet and Plan phases were sometimes prolonged due to the stakeholder's availability. |
** Evaluation and Maintenance phase: Duration (number of meetings): |
-
The evaluation phase starts from the first time the final draft was handed over to the stakeholder
-
The maintenance phase is based on 30 days cut-off period, where if there is no updates from the stakeholders, the (major) maintenance period is considered over.
In addition, a post-interview was also conducted to measure the success of RSSD. The survey questions are listed in Table 4, while the result is listed in Table 5. Each question was answered using the Likert scale (1–5). The overall score of the post-interview was 4.19/5.0, which means that the respondents agreed that RSSD was helpful. The question Po1 scores 4.30/5.0, which means RSSD helped in streamlining the development process. The results for each phase are discussed below.
Table 4
Post-Interview Questions (Quantitative)
#
|
Dev. Phase
|
Interview Question
|
Scale*
|
Po1
|
All
|
RSSD methodology helps in streamlining the development process, while maintaining the robustness and scalability of the software
|
1–5
|
Po2
|
Meet
|
Drawing sketches and showing them to the stakeholders were useful
|
1–5
|
Po3
|
Meet
|
The initial meeting went smoothly
|
1–5
|
Po4
|
Plan and Pre-Evaluation
|
The plan phase went smoothly as all information was collected thoroughly during the initial meeting
|
1–5
|
Po5
|
Plan and Pre-Evaluation
|
Identifying the small and big chunk of codes was useful
|
1–5
|
Po6
|
Design
|
Drawing only several diagrams were useful for better planning
|
1–5
|
Po7
|
Design
|
Making code structures before the development were useful
|
1–5
|
Po8
|
Develop
|
The development went smoothly because there was a careful planning
|
1–5
|
Po9
|
Develop
|
The difficulties during the development can be handled well
|
1–5
|
Po10
|
Test
|
Testing the codes with different test cases was useful to ensure stability
|
1–5
|
Po11
|
Test
|
Optimizing codes, such as minimizing loops and arrays, was useful
|
1–5
|
Po12
|
Evaluate
|
The presentation to the stakeholders went smoothly
|
1–5
|
Po13
|
Evaluate
|
The evaluate phase went smoothly, with minimal reworks needed
|
1–5
|
Po14
|
Maintenance
|
Changes during the maintenance phase can be handled well as there were careful plannings beforehand
|
1–5
|
* Likert Scale 1–5: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree |
Table 5. Post-Interview Results
6.1. Meet Phase
Questions Po2 and Po3 are related to the Meet phase, which produced scores of 4.1 and 4.6 out of 5.0, respectively. Respondent D7 stated that drawing sketches (Po2) were sometimes not doable when the meeting time is short.
6.2. Plan and Pre-Evaluation Phase
The questions Po4 and Po5 are related to the Plan Phase, which produced scores of 4.2 and 4.0 out of 5.0, respectively. Based on Po4, the respondents agreed that the Plan phase was smooth. Respondent D7 highlighted that identifying the big and small chunks during this phase (Po5) is useful. Respondent D4 stated that project C6 was smooth during the initial meeting. However, there were disagreements about the online system during the Plan phase as there were overlooked aspects during the meeting. It should've been better planned.
6.3. Design Phase
For the design phase, based on Po6 and Po7, the respondents generally agreed that creating diagrams and code structures was helpful in the later phases. As stated by D1:
"…For developing information systems, especially, I found it useful to create the code structure before implementing it, because big projects tend to be messier the longer it goes…" (D1)
Respondents D2 and D3 highlighted that creating diagrams and structure was a bit complicated at first, especially for a medium project for solo developers; however, they were eventually useful:
"…As a solo developer, I don't usually create diagrams. However, making some important diagrams (sequence, in my case) helped me to plan my app more systematically, which in turn decreases complications in the end…" (D2)
"…The systematic subphases in RSSD really helped me to develop my game more quickly. The initial subphases, such as creating structure and diagrams look complicated for solo development, but it helped me to focus on the codes…" (D3)
6.4. Develop Phase
Based on questions Po8 and Po9, respondents generally agreed that the Develop phase was smooth and difficulties can be handled well. Respondent D4 highlighted that the code structure that was made during the Design Phase was helpful, especially in difficult projects:
"…The meeting was smooth, even though I encountered a lot of difficulties during the implementation, especially on the synchronization processes. During the design phase, I planned by making codes structure as suggested. The structure and the diagrams serve as guides as the codes grow tremendously…" (D4)
Respondents D5 and D6 also highlighted that the Develop phase was smooth, particularly to develop the software systematically:
"…During the development, creating the code structure helps during the integration…" (D5)
"…As mainly information systems developers, the detailed subphases of RSSD are very practical. It prevents hassle towards the end of the project. The project is my master thesis, and RSSD has helped me develop the app in a systematic way…" (D6)
6.5. Test Phase
The respondents generally agreed that the Test Phase was smooth, based on Po10 and Po11. However, the average score for the Test Phase is the lowest among all the phases, which is 4.0. Some respondents find it useful to do code testing and optimization to prevent mess during the maintenance:
"…The methodology has made me realize the importance of code optimization before the amount of data in the system becomes much larger. I found an inefficient loop in my system that causes the student's grade saving process to be 70% slower. I wish I knew this earlier…" (D1)
"…Testing each method individually helped me to quickly identify wrong loops, excessive array usage, inefficient SQL queries, vulnerabilities, etc…" (D7)
"…I also constantly made code optimizations as the number of users in the game grows, especially for the socket system and queries, which reduced the database and server load significantly…" (D4)
In contrast, respondent D3 stated that code optimization might not be needed for projects that have to be completed quickly, and respondent D5 prefer not to optimize the code because there were only a few users using the software:
"…As for the code optimization part, I did that for the enemy AI code that slightly reduced the CPU usage. Even though such optimization might be more critical for information systems or bigger games, as opposed to small games with a quick development cycle, I think it helps to reduce the complexity of codes, especially for revisits…" (D3)
"…During the test phase, I successfully tested the methods using different test cases. However, for optimization, I only did some SQL query optimization, because I think the app will still work fine with just a few concurrent users…" (D5)
6.6. Evaluate and Maintenance Phase
Based on Po12, Po13, and Po14, the Evaluate and Maintenance phases went smoothly with minimal re-works needed due to the careful planning in the earlier phases. Project C6 was a difficult project with the longest project duration (256 days) and the longest Evaluate and Maintenance Phases (50% of the total project). However, RSSD helped to easier project navigation:
"…The evaluate and maintenance phases were still difficult simply because the app is huge with a lot of users involved, which produced a lot of varieties. Still, the plans from the earlier phases have helped me to navigate the project…" (D4)