4.1. Emphasis on Service Quality Dimensions
A total of 577,414 tweets were included in the analysis. About 67% of these tweets are related to conventional rail (CR) and 33% to high-speed rail (HSR). These values are very close to the actual CR and HSR passenger rates, showing that the machine learning-based technique used for document classification has a high estimation success[1]. As presented in Figure 3, the number tweets decreased between 2014 and 2017 but increased again as of 2018.
Much emphasis on a service quality dimension in social media content indicates users attached higher importance. Tables 4 and 5 show how much each service quality dimension has been emphasized in social media messages over the years.
Reliability is the most mentioned service dimension in social media messages for both CR and HSR, and its share in total mentions has been increasing over the years. Its’ quite high ratio in the total emphasis (49% and 54%, respectively) indicates that it is the most vital service dimension for citizens. This finding supports the previous study on Turkish users, which revealed that the timeliness of train movements is the most important service dimension (Seçilmiş et al., 2011). Reliability is followed by connection, where its share in HSR is slightly higher (15% and 17%, respectively). Assurance, empathy, and tangibles have similar rates in both systems. While the rate of assurance is 11%, tangibles remained at just 2%. This is also the lowest value among all service dimensions. In addition, HSR passengers gave a little more attention to comfort in their social media posts (8% and 11%, respectively).
In addition to these similarities, significant differences were also detected between the priorities of CR and HSR passengers. The most striking difference was observed in the responsiveness. Responsiveness was emphasized in only 2% of HSR-related messages, while this rate rises to 12% in CR. Similarly, only 3% of HSR passengers posted about safety&security, compared to 8% in CR. This shows that responsiveness and safety&security issues are discussed more by CR passengers. On the other hand, HSR passengers care more about convenience (12%), compared to 7% in CR.
Overall, these findings are important as they exhibit the similarities and differences of the most discussed service dimensions by CR and HSR passengers. While there are no significant differences between assurance, empathy, reliability, tangibles, connection, and comfort, CR passengers share more messages about responsiveness and safety&security. HSR passengers, however, discussed more about convenience. This finding indicates which service dimensions gain more importance in CR and HSR systems.
Table 4. Emphasis on conventional rail
|
2011
|
2012
|
2013
|
2014
|
2015
|
2016
|
2017
|
2018
|
2019
|
2020
|
2021
|
Overall
|
Assurance
|
7%
|
8%
|
11%
|
9%
|
10%
|
11%
|
11%
|
14%
|
13%
|
11%
|
11%
|
11%
|
Empathy
|
7%
|
5%
|
4%
|
5%
|
6%
|
8%
|
6%
|
6%
|
6%
|
4%
|
4%
|
5%
|
Reliability
|
45%
|
48%
|
53%
|
54%
|
49%
|
46%
|
44%
|
42%
|
46%
|
50%
|
56%
|
49%
|
Responsiveness
|
11%
|
10%
|
7%
|
8%
|
11%
|
18%
|
17%
|
20%
|
10%
|
10%
|
9%
|
12%
|
Tangibles
|
2%
|
1%
|
1%
|
2%
|
2%
|
1%
|
2%
|
3%
|
3%
|
3%
|
3%
|
2%
|
Comfort
|
10%
|
8%
|
7%
|
10%
|
9%
|
10%
|
10%
|
8%
|
10%
|
8%
|
7%
|
8%
|
Connection
|
19%
|
19%
|
14%
|
12%
|
13%
|
11%
|
13%
|
14%
|
16%
|
18%
|
15%
|
15%
|
Convenience
|
10%
|
8%
|
7%
|
7%
|
8%
|
7%
|
8%
|
7%
|
9%
|
8%
|
6%
|
7%
|
Safety&Security
|
8%
|
7%
|
7%
|
7%
|
8%
|
5%
|
6%
|
8%
|
11%
|
8%
|
9%
|
8%
|
Note: Due to the multi-labeling approach, the sum of the ratios is more than 100%.
Table 5. Emphasis on high-speed rail
|
2011
|
2012
|
2013
|
2014
|
2015
|
2016
|
2017
|
2018
|
2019
|
2020
|
2021
|
Overall
|
Assurance
|
6%
|
7%
|
10%
|
13%
|
11%
|
12%
|
14%
|
15%
|
13%
|
10%
|
8%
|
11%
|
Empathy
|
5%
|
4%
|
3%
|
5%
|
5%
|
7%
|
5%
|
5%
|
6%
|
4%
|
4%
|
5%
|
Reliability
|
53%
|
51%
|
49%
|
53%
|
49%
|
50%
|
51%
|
51%
|
53%
|
56%
|
65%
|
54%
|
Responsiveness
|
2%
|
2%
|
1%
|
1%
|
2%
|
2%
|
2%
|
3%
|
2%
|
2%
|
2%
|
2%
|
Tangibles
|
2%
|
2%
|
1%
|
2%
|
3%
|
3%
|
2%
|
3%
|
3%
|
2%
|
2%
|
2%
|
Comfort
|
11%
|
9%
|
9%
|
9%
|
14%
|
12%
|
11%
|
12%
|
12%
|
10%
|
9%
|
11%
|
Connection
|
17%
|
22%
|
24%
|
17%
|
15%
|
15%
|
17%
|
16%
|
16%
|
18%
|
14%
|
17%
|
Convenience
|
13%
|
16%
|
14%
|
14%
|
13%
|
14%
|
14%
|
13%
|
11%
|
10%
|
8%
|
12%
|
Safety&Security
|
4%
|
2%
|
2%
|
3%
|
3%
|
4%
|
3%
|
5%
|
4%
|
4%
|
3%
|
3%
|
Note: Due to the multi-labeling approach, the sum of the ratios is more than 100%.
Users' reactions (retweets, likes, and quotes) to service dimensions support emphasis rates. Table 6 shows the ratio of total retweets, likes, and quotes for each service quality dimension. It is interesting to note that the rates of the retweet, like, and quotes received by the service quality dimensions are largely similar to the rate of their emphasis on social media contents, both for CR and HSR systems.
Overall, reliability creates the highest social impact and it is followed by connection and assurance while empathy and tangibles are the least. This means that passengers respond more to messages about timing, the accuracy of reservations, access, parking availability, and connectivity with other modes of transport, and less to empathy, the appearance of staff, and physical equipment. The most striking finding here is users’ responses to responsiveness posts. Compared to HSR users, CR passengers responded significantly more to prompt service, availability and willingness of staff in handling their requests, suggestions, and complaints. HSR users are more responsive to comfort and convenience posts, while CR passengers are more concerned with safety&security. This clearly reveals the difference in expectations of CR and HSR users and the safety concerns of CR passengers.
Table 6. Social impacts of each service quality dimension
|
Retweet
|
Like
|
Quotes
|
|
CR
|
HSR
|
CR
|
HSR
|
CR
|
HSR
|
Assurance
|
13%
|
12%
|
11%
|
9%
|
14%
|
11%
|
Empathy
|
5%
|
5%
|
5%
|
5%
|
6%
|
5%
|
Reliability
|
41%
|
49%
|
50%
|
56%
|
41%
|
49%
|
Responsiveness
|
15%
|
2%
|
11%
|
2%
|
13%
|
3%
|
Tangibles
|
3%
|
3%
|
3%
|
2%
|
4%
|
3%
|
Comfort
|
8%
|
12%
|
9%
|
12%
|
7%
|
10%
|
Connection
|
19%
|
21%
|
15%
|
16%
|
20%
|
24%
|
Convenience
|
7%
|
11%
|
7%
|
12%
|
8%
|
9%
|
Safety&Security
|
10%
|
5%
|
9%
|
4%
|
10%
|
5%
|
Readers should recall that the values discussed in this section do not show the users’ satisfaction with the service dimensions but how much each dimension has been discussed on social media. The level of satisfaction for each dimension will be measured by sentiment analysis in the next section.
4.2. Level of Service Quality
Tables 7 and 8 display the positivity rates of tweets for CR and HSR services, respectively. The results are quite striking and support the view that Twitter content mostly tends to be negative (Collins et al., 2013; Yang and Anwar, 2016; Haghighi et al., 2018; El-Diraby et al. 2019; Whatman, 2018). As Schweitzer (2014) discussed, Twitter comments about transport services are mostly more negative compared to other public services. HSR received much higher satisfaction rates, but what is striking is the extremely low satisfaction rate of the conventional rail. Compared to the study conducted on the British railway system (Mogaji and Erkan, 2019), we found that the user satisfaction reflected in social media is extremely low in the Turkish conventional rail system (19% on average). This finding clearly indicates that users are mostly dissatisfied with the quality of conventional rail services. On the other side, half of the tweets about HSR are positive and contain expressions of satisfaction. The superiority of HSR is an expected result as it offers services with newer, faster, and more comfortable equipment and a higher level of security at the stations. Thus, the clear segregation of the results into the CR and HSR scores demonstrates the robustness of the analysis results.
Although the satisfaction rate of HSR is much higher than CR, the other half of the social media comments regarding HSR contain negative judgments. Nevertheless, a relative interpretation of the results would provide important insights into which service dimensions are better or worse. Despite the large difference in satisfaction rates, some similarities were found between the two rail systems. Both CR and HSR received the highest rate of positive comments from the comfort dimension (28% and 56%, respectively), and comfort satisfaction tends to increase over the years. Empathy, convenience, reliability, connection, and safety&security demonstrated a higher level of service than assurance and tangibles in both systems. Tangibles and convenience satisfaction tended to increase in both CR and HSR. Empathy satisfaction increased from 14% to 22% in CR but decreased from 55% to 51% in HSR over the observation period. While HSR has improved the reliability, it has experienced a decline in assurance satisfaction. Finally, responsiveness has the lowest satisfaction rate among all service dimensions for both CR and HSR (9% and 30%, respectively). This finding strongly supports the previous studies on Turkish rail passengers, where responsiveness was the least satisfied service dimension in CR (Poyraz et al. 2004; Seçilmiş et al. 2011) and HSR (Altan and Ediz, 2016). This demonstrates the inability of staff to respond to users' requests and the unwillingness of existing staff to assist passengers. Fortunately, compared to the first year of the observation period, the satisfaction rate for this service dimension has improved significantly.
Table 7. Level of satisfaction in conventional rail
|
2011
|
2012
|
2013
|
2014
|
2015
|
2016
|
2017
|
2018
|
2019
|
2020
|
2021
|
Overall
|
Assurance
|
16%
|
28%
|
13%
|
15%
|
18%
|
18%
|
23%
|
12%
|
14%
|
18%
|
17%
|
17%
|
Empathy
|
14%
|
18%
|
23%
|
27%
|
22%
|
26%
|
24%
|
20%
|
22%
|
23%
|
22%
|
22%
|
Reliability
|
21%
|
22%
|
27%
|
20%
|
20%
|
24%
|
24%
|
19%
|
21%
|
21%
|
22%
|
22%
|
Responsiveness
|
4%
|
8%
|
10%
|
7%
|
6%
|
4%
|
6%
|
6%
|
16%
|
14%
|
17%
|
9%
|
Tangibles
|
11%
|
17%
|
16%
|
11%
|
29%
|
17%
|
15%
|
14%
|
15%
|
16%
|
17%
|
16%
|
Comfort
|
19%
|
21%
|
33%
|
38%
|
32%
|
26%
|
30%
|
28%
|
24%
|
26%
|
27%
|
28%
|
Connection
|
17%
|
17%
|
20%
|
19%
|
19%
|
19%
|
22%
|
12%
|
14%
|
15%
|
18%
|
17%
|
Convenience
|
15%
|
17%
|
17%
|
15%
|
17%
|
16%
|
22%
|
21%
|
21%
|
17%
|
19%
|
18%
|
Safety&Security
|
13%
|
17%
|
12%
|
15%
|
27%
|
23%
|
21%
|
15%
|
16%
|
18%
|
21%
|
18%
|
Table 8. Level of satisfaction in high-speed rail
|
2011
|
2012
|
2013
|
2014
|
2015
|
2016
|
2017
|
2018
|
2019
|
2020
|
2021
|
Overall
|
Assurance
|
47%
|
42%
|
46%
|
56%
|
48%
|
41%
|
46%
|
42%
|
41%
|
40%
|
40%
|
44%
|
Empathy
|
55%
|
49%
|
47%
|
51%
|
54%
|
69%
|
45%
|
49%
|
49%
|
54%
|
51%
|
52%
|
Reliability
|
49%
|
49%
|
50%
|
51%
|
53%
|
54%
|
53%
|
52%
|
47%
|
51%
|
52%
|
51%
|
Responsiveness
|
27%
|
30%
|
21%
|
26%
|
29%
|
20%
|
25%
|
42%
|
34%
|
39%
|
42%
|
30%
|
Tangibles
|
36%
|
42%
|
44%
|
42%
|
56%
|
51%
|
40%
|
35%
|
38%
|
36%
|
42%
|
42%
|
Comfort
|
50%
|
50%
|
55%
|
55%
|
45%
|
67%
|
56%
|
60%
|
55%
|
58%
|
60%
|
56%
|
Connection
|
44%
|
54%
|
60%
|
47%
|
47%
|
46%
|
55%
|
46%
|
44%
|
44%
|
44%
|
48%
|
Convenience
|
47%
|
69%
|
62%
|
47%
|
50%
|
53%
|
46%
|
54%
|
45%
|
49%
|
50%
|
52%
|
Safety&Security
|
47%
|
46%
|
41%
|
42%
|
48%
|
47%
|
44%
|
43%
|
45%
|
44%
|
47%
|
45%
|
Figure 4 presents the differences between the overall satisfaction levels of CR and HSR. While HSR provided superior services in all dimensions, it outperformed CR particularly in convenience (34%), connection (31%), and empathy (30%). This indicates that the most distinguishing feature of HSR compared to CR is the ease of booking, travel information access, boarding-alighting, and support for disabled users. It also offers better connectivity in terms of access, parking, and integration with other modes of transport. Moreover, in HSR services, user needs are clearly better understood and due attention is paid to the interests of users. On the other hand, the most similar service offered by both systems is responsiveness (21%), which indicates HSR made less difference in the staff's approach to the user.
4.3. Service Quality Improvement
The importance-performance analysis introduced by Martilla and James (1977) is widely used to visualize the relationship between the importance attributed to service dimensions and the perceived service level, and to identify prior improvement areas. In this simple method, the perceived service level and the importance of service dimensions are displayed in a two-dimensional graph, including mean importance and performance ratings. Thus, service dimensions are divided into one of four quadrants according to its performance and perceived service quality: high importance – high satisfaction (keep up the good work), low importance – high satisfaction (possible overkill), low importance – low satisfaction (low priority), and high importance – low satisfaction (concentrate here) (Martilla and James, 1977).
In the case of rail services, these quadrants refer to the following policy recommendations for decision-makers:
- Quadrant 1 – Keep up the good work: Rail passengers attach great importance to these service dimensions and are satisfied with the level of service provided. This success must be maintained and improved to compete with other modes of transport and enhance passenger loyalty.
- Quadrant 2 – Possible overkill: Passengers are satisfied with the level of service offered, but these services are of little importance to them. Nevertheless, successful practices must be sustained in the public services.
- Quadrant 3 – Low priority: Passengers are not satisfied with the level of service offered, but they do not worry much about it. If resources are limited, the focus should be placed on other services at Quadrants I and IV. Since the rail service is provided by the state and is a public service in Turkey, these services should not be ignored and ways of improvement should be sought without worsening the priority services.
- Quadrant 4 – Concentrate here: Rail passengers attach great importance to these services, but perceived satisfaction is low. This quadrant includes the service dimensions that need the most urgent improvement, as they can make a major contribution to the user satisfaction and competitiveness of the rail system.
This study determined the importance of service dimensions according to their frequency in social media content. That is, we assumed that much emphasis on a service quality dimension indicates higher importance (see Tables 4 and 5). The level of satisfaction was measured by sentiment analysis and obtained from Tables 7 and 8. In fact, it is clear that the satisfaction levels of all services are very low and they all need improvement, especially in the CR system. Nevertheless, the importance-performance structure provides important insights to decision-makers about which services they should prioritize.
Figures 5 and 6 display the importance-satisfaction of the CR and HSR systems, respectively, and visualize the change from 2011 to 2021. In both figures, the importance value of reliability is excluded because it deviates excessively from the others. It was observed in the figures that the average satisfaction of both systems increased at the end of the observation period and the satisfaction lines moved upward. In particular, CR satisfaction exhibited a significant improvement from 14% to 20%.
In the CR system, reliability and connection took place in Quadrant I in 2011. Reliability managed to stay in the first quadrant in 2021, thanks to its high importance and increased satisfaction. Passengers care about reliability and satisfaction level is relatively high, and its’ continuity should be ensured. However, the connection shifted to Quadrant IV in 2021. Although there was a slight increase in its’ satisfaction, this increase remained below the general average. Moreover, its’ importance has also decreased. Therefore, the connection is positioned in the fourth quarter, which indicates prior improvement areas. This implies that the connection is of higher importance to users (despite the recent decrease) but the perceived performance is low. Due to its high importance, improvements in connection will make more sense for users and may contribute more to the competitiveness of conventional railways.
Satisfaction levels of empathy and comfort increased by 8% in 2021 and both service dimensions remained in Quadrant II. The operator put much effort into developing empathy and comfort, and offered a relatively higher level of service, but users do not perceive this as an important service dimension. Convenience is positioned in Quadrant II in 2021 despite small increase in satisfaction rate.
Tangibles, assurance, responsiveness, and safety&security have fallen to Quadrant III in 2011, which refers to low satisfaction and low importance. It was observed that these four service dimensions showed great improvement in 2021. In particular, the satisfaction rate of responsiveness has made a great leap from 4% to 17%. In the same period, tangibles improved by 6% and safety&security by 8%. Thus, safety&security is positioned in Quadrant II in 2021. ASR, on the other hand, has shown a horizontal development, that is, the importance attributed to it has increased. The conclusion to be drawn from this is not that operator should ignore these services and focus on the prior ones listed in Quadrants I and IV. TCDD should be expected to maintain and improve these services as a service-oriented public institution. Indeed, the fact that the safety&security is less discussed on social media does not mean that this dimension can be neglected. But, it is clear that passengers care more about reliability and connection than the appearance of personnel, station, and trains (tangibles), prompt response, and the staff attitudes. Thus, decision-makers should ensure that resources are allocated primarily to services that are more important to users.
Similar to CR, reliability remained important in the HSR system and was positioned in Quadrant I. HSR passengers attach more importance to this service and are relatively satisfied. Convenience was included in Quadrant I in 2011, but its importance to users has decreased significantly in recent years. It shifted to Quadrant II with a horizontal movement in 2021. Due to the possible improvements made in travel information sharing, ticketing, and reservation, the issue may have been less discussed on social media recently and occupied less place on the agenda of the passengers. On the other hand, the connection is still located in Quadrant IV and has been identified as a priority dimension to focus on, although it has been less discussed on social media recently. Decreased service satisfaction of assurance and stability of safety&security caused a regression from Quadrant II to Quadrant III. Tangibles and responsiveness remained in Quadrant III, despite the rising service quality. In particular, responsiveness satisfaction achieved a significant improvement from 27% to 42%. During the analysis period, an increase in the service quality of comfort and a decrease in empathy were observed. Yet both are positioned in Quadrant II in 2021.
[1] The proportion of CR and HSR passengers was 66.7% and 33.3% in 2018, 68.2% and 31.8% in 2019, respectively (TCDD, 2020).