1. ARMADILLO Case Study
In this section, the case study of the ARMADILLO mission is presented with the application of the methodologies: DoD, risk management for CubeSats by Yassi and Sousa (2017)15, and RTF, to analyze the risks in probability versus impact matrices.
The CubeSat ARMADILLO (Atmosphere Related Measurements and Detection of submILLimeter Objects) was a 3U CubeSat developed by the SDL (Satellite Design Laboratory) of the University of Austin in Texas (USA).
The development of the satellite began in 2012, and the launch took place in 2019. The mission's primary objectives were: to characterize sub-millimeter-level dust and debris in-situ in LEO orbit13.
Table 1 presents the identified risks for the ARMADILLO mission, with a description of the risk, the risk type rating, the risk TAG, and the assigned probability and impact values for each risk.
Table 1
Identified risks and qualitative risk analysis for the ARMADILLO mission.13.
Risk Description
|
Risk Type
|
TAG
|
Probability
|
Impact
|
Risk of non-delivery of the engineering model for the competition flight committee review
|
Schedule
|
SCH
|
3,98
|
3,59
|
Failure to acquire data from the scientific mission in orbit
|
Payload
|
PAY
|
4,00
|
3,00
|
Being unable to communicate with the satellite
|
Spacecraft
|
SP-1
|
2,19
|
3,07
|
Being unable to acquire satellite data
|
Spacecraft
|
SP-2
|
2,05
|
3,14
|
Inability to achieve industrial quality standards
|
Spacecraft
|
SP-3
|
1,09
|
2,09
|
Loss of human knowledge and experience
|
Personnel
|
PER
|
3,95
|
2,56
|
Mission to have a substantial increase in cost to be able to be taken forward
|
Cost
|
COST
|
3,00
|
3,50
|
1.1. Probability and Impact Matrix - DoD Risk Management
With the application of the DoD methodology16 for the ARMADILLO mission, we obtained the probability versus impact matrix shown in Fig. 1:
In the DoD model of the seven risks identified in the ARMADILLO mission, one was in the acceptable zone (SP-3), five were in the mitigable zone (PER, PAY, COST, SP-1, and SP-2), and one was in the zone to be avoided (SCH).
1.2. Probability and Impact Matrix - Methodology for Technical Risk Management for Projects with CubeSats (Yassi and Sousa, 2017)15
This methodology uses the concept of risk level, presented in topic 3. By applying this methodology to the ARMADILLO mission, table 2 is obtained:
Table 2
Identification, qualitative analysis of risks and risk level for the ARMADILLO mission of the Yassi and Sousa methodology13, 15
Risk Description
|
Risk Type
|
TAG
|
Probability
|
Impact
|
Risk Level
|
Risk of non-delivery of the engineering model for the competition flight committee review
|
Schedule
|
SCH
|
3,98
|
3,59
|
3
|
Failure to acquire data from the scientific mission in orbit
|
Payload
|
PAY
|
4,00
|
3,00
|
2
|
Being unable to communicate with the satellite
|
Spacecraft
|
SP-1
|
2,19
|
3,07
|
1
|
Being unable to acquire satellite data
|
Spacecraft
|
SP-2
|
2,05
|
3,14
|
1
|
Inability to achieve industrial quality standards
|
Spacecraft
|
SP-3
|
1,09
|
2,09
|
1
|
Loss of human knowledge and experience
|
Personnel
|
PER
|
3,95
|
2,56
|
2
|
Mission to have a substantial increase in cost to be able to be taken forward
|
Cost
|
COST
|
3,00
|
3,50
|
2
|
With the help of this table, it is possible to generate the probability versus impact matrix represented in Fig. 2
In this methodology of the seven risks identified in the ARMADILLO mission, one was in the low-risk zone (SP-3), five were in the medium risk zone (PER, PAY, COST, SP-1, and SP-2), and one was in the zone and high risk (SCH).
1.3. RTF Methodology
The criteria for determining the tolerance factors for each type of risk identified were developed according to the criteria described below to use the RTF methodology.
Schedule risks: the smaller the schedule flexibility, the smaller the tolerance factor should be; similarly the more significant the flexibility, the greater the factor.
Payload Risks: These are risks related to the mission payload. A payload composed of COTS components and with extensive flight heritage should have a small risk tolerance, as well as a payload that does not use COTS and has no flight heritage, should have greater risk tolerance.
Spacecraft Risks are risks related to the satellite platform used in the mission. For example, a satellite composed of subsystems that use COTS and with broad flight heritage should have a small risk tolerance, just as a satellite that does not use COTS and does not have flight heritage should have a greater tolerance to risk.
Personnel risks: the experience and training of the development team are also influencing factors for risk tolerance. The greater the training and experience, the lower the tolerance factor; the less training and experience, the greater the factor.
Cost risks: concerns the availability of financial resources to accomplish the mission. The lower the availability of resources, the lower the tolerance factor; the more significant the availability, the greater the tolerance factor.
Note that the values described herein are indicative and may vary with each organization's ability to tolerate a given risk. If an identified risk applies to more than one category, the most severe RTF must be considered.
Then, the parameters were applied to Table 1, forming Table 3, which, together with the previous risk information, also presents the RTF values obtained for each type of risk in the mission. Therefore, the risks were classified in the probability versus impact matrices and illustrated in Figs. 3 and 4.
Table 3
Identification and classification of risks following the RTF approach, ARMADILLO mission13
Risk Description
|
Risk Type
|
TAG
|
Probability
|
Impact
|
RTF
|
Risk of non-delivery of the engineering model for the competition flight committee review
|
Schedule
|
SCH
|
3,98
|
3,59
|
3
|
Failure to acquire data from the scientific mission in orbit
|
Payload
|
PAY
|
4,00
|
3,00
|
5
|
Being unable to communicate with the satellite
|
Spacecraft
|
SP-1
|
2,19
|
3,07
|
5
|
Being unable to acquire satellite data
|
Spacecraft
|
SP-2
|
2,05
|
3,14
|
5
|
Inability to achieve industrial quality standards
|
Spacecraft
|
SP-3
|
1,09
|
2,09
|
5
|
Loss of human knowledge and experience
|
Personnel
|
PER
|
3,95
|
2,56
|
3
|
Mission to have a substantial increase in cost to be able to be taken forward
|
Cost
|
COST
|
3,00
|
3,50
|
3
|
As two RTF values were obtained, two matrices were used, one for each determined RTF and its respective risks. These matrices can be seen in Figs. 3 and 4.
Using the RTF methodology, of the seven risks identified in the ARMADILLO mission, three were in the acceptance zone (SP-1, SP-2, and SP-3), three were in the mitigation zone (PAY, PER, COST), and one of the risks was in the zone to be avoided (SCH).
1.4. Analysis of probability versus impact matrices for different methodologies - ARMADILLO case study
By analyzing the probability versus impact matrices and the qualitative risks that should be dealt with in each methodology, it is possible to visualize the optimization of the risk management process proposed by the RTF methodology. Table 4illustrates the qualitative risks that will make up the risk mitigation plans.
Table 4
Comparisons between methodologies for the ARMADILLO mission
COMPARATIVE - ARMADILLO Mission
|
Approach
|
Acceptable Risks
|
Risks that require mitigation
|
Risks that must be avoided
|
DoD
|
1
|
5
|
1
|
Yassi & Sousa
|
1
|
5
|
1
|
RTF
|
3
|
3
|
1
|
The DoD methodology and Yassi and Sousa obtained the same treatment for the risks. The RTF methodology managed to have an increase in the number of acceptable risks by 200% (with the addition of two of them), and a 40% reduction in mitigable risks (with the decrease of two of them).
This result demonstrates that the approach can identify and classify risks and provide a scenario that is more by the organization's capabilities to accept, mitigate or avoid risk, with a smaller number of risks to be dealt with.
2. CONASAT Case Studies
This topic presents the case study of the CONASAT mission applying the methodologies: DoD16, for risk management for Yassi and Sousa's CubeSats (2017)15; and RTF, to highlight the risks identified in the probability versus impact matrix.
The CONASAT mission began in 2010 with a public notice issued by the Brazilian Space Agency (AEB). The mission's primary objective is to replace the current space segment of the SBCDA (Brazilian Environmental Data Collection System), which has been in orbit since the 1990s.
The project is in an advanced stage of development, already having the first satellite that will make up the constellation (flight and engineering model).
For the CONASAT mission, seven risks were also identified: risk of budget unavailability (TAG – COST); turnover of project members (TAG – PER); technological immaturity of the payload (TAG – PAY); discontinuity of the project (TAG – PROJ); inability to acquire satellite data (TAG – SP-1); inability to communicate between the satellite and the payload (TAG – SP-2); inability to send commands to the satellite (TAG – SP-3);
Table 5 presents the risks identified for the CONASAT mission, with a description of the risk, the risk type classification, the risk TAG, and the assigned probability and impact values for each risk.
Table 5
Identified risks and qualitative risk analysis for the CONASAT mission
Risk Description
|
Risk Type
|
TAG
|
Probability
|
Impacto
|
Risk of budget unavailability
|
Cost
|
COST
|
3,87
|
4,01
|
Project member turnover
|
Personnel
|
PER
|
3,86
|
3,72
|
Technological immaturity of the payload
|
Payload
|
PAY
|
2,19
|
3,88
|
Project discontinuity
|
Schedule
|
PROJ
|
3,57
|
3,92
|
Inability to acquire satellite data
|
Spacecraft
|
SP-1
|
1,52
|
3,78
|
Inability to communicate between satellite and payload
|
Spacecraft
|
SP-2
|
2,22
|
3,56
|
Inability to send commands to the satellite
|
Spacecraft
|
SP-3
|
1,09
|
3,50
|
2.1. DoD Risk Management Application
With the application of the DoD methodology16 for the CONASAT mission, the following probability versus impact matrix for the mission was obtained, as shown in Fig. 5:
In the DoD model, of the seven risks identified for the CONASAT mission, two were in the acceptable zone (SP-1 and SP-3), two were in the mitigable zone (PAY and SP-2), and three were in the avoidable zone (PER, PROJ, and COST).
2.2. Methodology for Technical Risk Management for Projects with CubeSats (Yassi and Sousa)
In this methodology, the concept of risk level, presented in topic 3, is used. When applying the methodology to the CONASAT mission, Table 6is obtained:
Table 6
Risk level mission CONASAT Methodology for Technical Risk Management for Projects with CubeSats Yassi and Sousa methodology
Risk Description
|
Risk Type
|
TAG
|
Probability
|
Impact
|
Risk Level
|
Risk of budget unavailability
|
Cost
|
COST
|
3,87
|
4,01
|
3
|
Project member turnover
|
Personnel
|
PER
|
3,86
|
3,72
|
3
|
Technological immaturity of the payload
|
Payload
|
PAY
|
2,19
|
3,88
|
2
|
Project discontinuity
|
Schedule
|
PROJ
|
3,57
|
3,92
|
3
|
Inability to acquire satellite data
|
Spacecraft
|
SP-1
|
1,52
|
3,78
|
1
|
Inability to communicate between satellite and payload
|
Spacecraft
|
SP-2
|
2,22
|
3,56
|
2
|
Inability to send commands to the satellite
|
Spacecraft
|
SP-3
|
1,09
|
3,50
|
1
|
With this table, it is possible to generate the probability versus impact matrix represented in Fig. 6:
In this methodology, of the seven risks identified in the CONASAT mission, two were in the low-risk zone (SP-1 and SP-3), two were in the medium risk zone (SP-2 and PAY), and three were in the high-risk zone (PER, COST, and PROJ).
2.3. RTF Methodology
After defining the parameters for determining the RTF, they were applied to Table 5, giving rise to Table 7, which, together with the information previously presented on risks, also shows the RTF values obtained for each type of risk of the mission. Finally, the risks were classified in probability versus impact matrices and illustrated in Figs. 7, 8, and 9.
Table 7
Identification and classification of risks following the RTF approach, CONASAT mission
Risk Description
|
Risk Type
|
TAG
|
Probability
|
Impact
|
RTF
|
Risk of budget unavailability
|
Cost
|
COST
|
3,87
|
4,01
|
4
|
Project member turnover
|
Personnel
|
PER
|
3,86
|
3,72
|
3
|
Technological immaturity of the payload
|
Payload
|
PAY
|
2,19
|
3,88
|
3
|
Project discontinuity
|
Schedule
|
PROJ
|
3,57
|
3,92
|
5
|
Inability to acquire satellite data
|
Spacecraft
|
SP-1
|
1,52
|
3,78
|
3
|
Inability to communicate between satellite and payload
|
Spacecraft
|
SP-2
|
2,22
|
3,56
|
3
|
Inability to send commands to the satellite
|
Spacecraft
|
SP-3
|
1,09
|
3,50
|
3
|
As more than one RTF value was obtained, three matrices were used; one for each RTF determined and its respective risks. These matrices can be seen in Figs. 7, 8, and 9.
Using the RTF methodology, of the seven risks identified in the CONASAT mission, two were in the acceptance zone (SP-1 and SP-3), four were in the mitigation zone (COST, PROJT, SP-2, and PAY), and only one of the risks was in the avoidable zone (PER).
2.4. Analysis of different methodologies
Analyzing the probability versus impact matrices and the quantitative risks to be dealt with in each methodology makes it possible to visualize the optimization of the risk management process proposed by the RTF methodology. Table 8 illustrates the number of risks that each methodology will deal with.
Table 8
Comparisons between methodologies for the CONASAT mission
COMPARATIVE - CONASAT Mission
|
Approach
|
Approach
|
Approach
|
Approach
|
DoD
|
2
|
2
|
3
|
Yassi & Sousa
|
2
|
2
|
3
|
RTF
|
2
|
4
|
1
|
The DoD methodology and Yassi and Sousa obtained the same treatment for risks. The RTF methodology managed to have an increase in the number of mitigable risks by 100% (with the addition of two of them), and a decrease in 67% of the risks to be avoided (with the decrease of two of them).
This result demonstrates that the approach can identify and classify risks and provide a scenario that is more following the organization's capabilities to accept, mitigate or avoid risk, with a smaller number of risks to be dealt with.