i. Dimension Analysis:
The first step in the validation process comprises of dimension analysis which involves the identification of clusters as is being done using EFA in the classical approach. The analysis was done using Exploratory Graph Analysis, which deals with the identification of the number of clusters (factors) in the network by maximizing the number of connections within a set of nodes, while minimizing the connections from the same set of nodes to the other set of nodes through community detection algorithms. The nodes represent the items in the network, while the connections between nodes are called edges, which represent the regularized partial correlations between the nodes after conditioning on all the other nodes in the network. Thus the communities represent the stable and coherent sub-network within the overall network. The Exploratory Graph Analysis was done using EGAnet function in the EGAnet R Package. After importing the data set, the function was run on R, which yielded a network comprising of 5 clusters/dimensions as against to that of a 6-dimension structure revealed in the original scale. However, the item numbers 17, 18 and 19 of dimension 4(Exposure) loaded on dimension 1 (Information Seeking). Thus the dimension analysis using EGA in the R, revealed the five communities of the scale, with node numbers 17, 18 and 19 getting merged with first cluster/dimension. These three items pertained to the ‘Exposure’ cluster/dimension in the original scale and got merged with ‘Information Seeking’ cluster/dimension. The EGA trial first results are shown in Fig. 1 below.
The first EGA run was followed by a 2nd EGA run after the three items (17, 18 and 19) showing split loading were dropped. The network analysis plot obtained after the second run showed a clear 5 community/cluster/factor structure including Information Seeking (7 nodes), Connection (6 nodes), Entertainment (3 nodes), Social Influence (5 nodes) and Coordination (3 nodes).
The network plot emerging from the 2nd EGA run is plotted below and shows a clear 5 factor structure as shown below in Fig. 2.
The above Final Network Plot showing dimensions extracted using EGA. The color of the nodes represents the dimensions and the thickness of the lines represent the partial correlations (green = positive; red = negative).
The final network plot clearly shows the extraction of 5 Factors with each factor showing an item structure in the form of nodes. Exploratory Graph analysis adapts to the inclusion of the Louvain community detection algorithm (Blondel, Guillaume, Lambiotte, & Lefebvre, 2008), which has demonstrated better performance in identifying dimensions (Christensen, 2020). However, it is pertinent to mention that we estimated the dimensions of the network using both the Walktrap algorithm as well as the Louvain algorithm. The resultant network using both the community detection algorithms yielded similar cluster structure.
Estimates of Node Strength/Network Loading
The network loading represents each nodes contribution to the production of a coherent dimension (cluster) in the network (Christensen and Golino,2020). Node strength, the sum of the connections of the node are roughly redundant with CFA factor loadings (Hallqruist et al, 2019). The network loadings hence represent a very complex structure that lies in between the saturated structure (EFA) on one side and the simple structure (CFA) on the other side. We employed the net.loads function to compute the standardized node strength of each node for each cluster in the emergent network. The network loadings of different items against each dimension is shown in the table below:
Table 1: Estimates of Network Loading
1 2 3 4 5
|
IG5 0.348427946 0.041569928 0.000000000 -0.007705642 0.016502279
|
IG4 0.342863634 0.005215451 0.020533423 -0.008055649 0.000000000
|
IG2 0.296657603 0.048167879 0.004512860 0.007029490 0.000000000
|
IG7 0.295232606 0.095341051 0.000000000 -0.030677915 0.000000000
|
IG6 0.288815034 0.055825368 0.023939042 0.006545624 0.042207328
|
IG1 0.267295469 0.009567528 0.016281317 -0.003998253 0.001178831
|
IG3 0.249064636 0.035481865 0.001431533 0.000000000 0.029503034
|
IG12 0.000000000 0.348803673 0.052655221 0.045178989 0.032174002
|
IG11 0.011283141 0.294641592 0.000000000 0.046560871 0.066242271
|
IG8 0.027337040 0.254321106 0.006296971 0.005872836 0.095969600
|
IG13 0.000000000 0.245255002 0.134724410 0.055960125 0.009608938
|
IG9 0.110254422 0.229345714 0.082200694 -0.012709969 0.000000000
|
IG10 0.129041751 0.198324019 0.000000000 -0.003371526 0.079710044
|
IG15 0.003017661 0.030633499 0.425687457 0.062562903 0.000000000
|
IG14 0.010886997 0.109857053 0.379536952 0.005366771 0.031711517
|
IG16 0.030695103 0.052780000 0.376227716 0.024616052 0.014249413
|
IG22 -0.011506116 0.033899971 0.000775617 0.414319293 0.039021570
|
IG21 -0.015400878 0.045500232 0.047919349 0.401969920 0.003345238
|
IG24 -0.016303995 0.041725601 0.000000000 0.352945717 0.040801909
|
IG23 0.000000000 0.000000000 0.000000000 0.319070165 0.103102794
|
IG20 0.011630128 0.031152162 0.069876119 0.161813392 0.048496110
|
IG26 0.004744402 0.009074997 0.000000000 0.071578084 0.468349948
|
IG25 0.056876107 0.078897575 0.001267113 0.081251168 0.373567494
|
IG27 0.000000000 0.116920416 0.046113362 0.036068320 0.246433355
|
The network loadings, like that of factor loadings in EFA represent how well an item belonging to a particular factor/dimension is related to it. While analyzing the network loadings matrix, shown above, it was observed that the network loadings significantly differ for different items. In network psychometrics, these represent partial correlation values and hence the values are actually significant. According to Christensen and Golino (2022), the network loading thresholds are: 0.15 = Small network loading, 0.25 Moderate network loading, 0.35 = Large network loading.
As per the results shown above, the perusal of network loadings shows that the items contained within their respective dimensions, show moderate to large network loadings, thereby we can say that these nodes represent their clusters quite well. However, a few items show small network loadings (Item No’s 2,3 & 6 of Connection dimension, Item-1 of Social Influence dimension. However, it should be borne into mind that the network loadings do not necessarily correlate to the interpretation of factor loadings and actually represent, the contribution of a particular node in leading to its respective cluster. Hence the items showing small network loadings need not to be necessarily removed. By this interpretation, the nodes have been considered, and the final network yielded after the 2nd Trial of EGA showing 5 cluster and 24 node structure has been retained.
ii. Confirmatory Factor Analysis
To confirm the above obtained factor structure, we used computation of fit measures using lavaan package. The factor structure shown in Fig. 3 below was revealed.
Table 2
Goodness of Fit Estimates of the Ordinal Confirmatory Factor Analysis
|
CFI
(For Interval Data)
|
Robust
CFI
(For Ordinal Data)
|
TLI
(For Interval Data)
|
Robust
TLI
(For Ordinal Data)
|
RMSEA
(For Interval Data)
|
Robust
RMSEA
(For Ordinal Data)
|
SRMR
(For Interval Data)
|
SRMR
Bentler
(For Ordinal Data)
|
Benchmark
|
> 0.95
|
> 0.95
|
> 0.95
|
> 0.95
|
< 0.05
|
< 0.05
|
< 0.08
|
< 0.08
|
Study Estimates
|
0.982
|
0.887
|
0.980
|
0.871
|
0.086
|
0.088
|
0.074
|
0.064
|
The result estimates yielded in the study as shown in the above table fall well within the respective benchmarks as per the goodness of fit standards set by Hu and Bentler (1999). Thus the results confirm the 5-dimensional network structure of the internet gratification scale as yielded in the Exploratory Graph Analysis.
Estimation of Regularized Network
The regularized network estimation requires installation of bootnet package. The results of the regularized network is shown in Fig. 4 as under.
As it is clear from the edge weights, the nodes of all the five clusters show more relationships within and less relation with the nodes of the other dimensions. As clearly evident, nodes 1,2,3,4,5,6 and 7 belonging to Dimesnion-1 of Internet Gratification scale are much interrelated as compared to their interrelation with the nodes of the dimension-2. Similarly, the nodes of the dimension 2 including nodes 8, 9, 10, 11, 12 and 13 also much interrelation in between as compared to the nodes 14, 15 and 16 of dimension 3 and so on is the case with all the nodes of the rest of the dimensions in the scale.
iii. Internal Consistency Analysis/Structural Consistency Analysis
In the classical approaches, the commonly used measure for the assessment of unidimensionality and internal consistency is Cronbach’s alpha. It has been one of the most pervasive methods for establishing internal consistency of the scale (Mcneish, 2018). However, despite this pervasiveness, there is a sort of ambiguity on the understanding of internal consistency, which measures the extent of the items in a dimension to which they are interrelated and the homogeneity, denoting the set of items having a common cause. With this perspective in mind, the current study employed the measurement of Structural Consistency, which is a measure of the extent to which the items in a dimension are homogenous and are interrelated, given the multidimensional structure of the tool under Network Psychometrics (Christensen, 2019). The same has been performed using Bootstrap Exploratory Graph Analysis using the function bootEGA. We approached the structural consistency analysis with the installation of bootEGA package. The following results were yielded after the dimension stability function.
Table 3
Structural Consistency Values (Dimension Wise)
Dimension
|
Structural Consistency
|
1
|
1.000
|
2
|
0.880
|
3
|
0.990
|
4
|
0.988
|
5
|
0.998
|
As evident from the above table, and Fig. 5, four dimensions have nearly perfect structural consistency, whereas 1 dimension has got very near to perfect structural consistency. The above values can range from 0 to 1 and correspond to the proportion of times that each empirically derived dimension is exactly recovered from the replicate bootstrap samples. A look into the data reveals that each dimension shows a higher structural consistency ranging from 0.9 to 1. Thus the dimension structure extracted using EGA has high structural consistency as per the original EGA results.
Item Stability within Dimensions
In most simple terms, the item stability refers to the proportion of the times, each of the item is retained in its empirically derived dimension across the replicated samples. The table below shows the item-wise stability within dimensions/clusters.
Table.4 Item Stability
Item/Dimension
|
Dim1 Dim2 Dim3 Dim4 Dim5 Dim6
|
IG1
|
1.000 0.000 0.00 0.000 0.000 0.000
|
IG2
|
1.000 0.000 0.00 0.000 0.000 0.000
|
IG3
|
1.000 0.000 0.00 0.000 0.000 0.000
|
IG4
|
1.000 0.000 0.00 0.000 0.000 0.000
|
IG5
|
1.000 0.000 0.00 0.000 0.000 0.000
|
IG6
|
1.000 0.000 0.00 0.000 0.000 0.000
|
IG7
|
1.000 0.000 0.00 0.000 0.000 0.000
|
IG8
|
0.004 0.992 0.00 0.000 0.004 0.000
|
IG9
|
0.020 0.976 0.00 0.000 0.004 0.000
|
IG10
|
0.020 0.976 0.00 0.000 0.004 0.000
|
IG11
|
0.000 0.904 0.00 0.000 0.000 0.096
|
IG12
|
0.000 0.904 0.00 0.000 0.000 0.096
|
IG13
|
0.000 0.904 0.00 0.000 0.000 0.096
|
IG14
|
0.000 0.004 0.99 0.000 0.000 0.006
|
IG15
|
0.000 0.004 0.99 0.000 0.000 0.006
|
IG16
|
0.000 0.004 0.99 0.000 0.000 0.006
|
IG20
|
0.000 0.006 0.00 0.988 0.004 0.002
|
IG21
|
0.000 0.000 0.00 1.000 0.000 0.000
|
IG22
|
0.000 0.000 0.00 1.000 0.000 0.000
|
IG23
|
0.000 0.000 0.00 1.000 0.000 0.000
|
IG24
|
0.000 0.000 0.00 1.000 0.000 0.000
|
IG25
|
0.000 0.000 0.00 0.002 0.998 0.000
|
IG26
|
0.000 0.000 0.00 0.002 0.998 0.000
|
IG27
|
0.000 0.000 0.00 0.002 0.998 0.000
|
It is clear from the above table that the nodes 1 to 7 are loading in cluster/dimension 1 in the replicate sample 100 percent times, while as nodes 8 to 13 are loading with cluster 2, with a proportion of 90 to 99% times. Similarly, nodes, 14, 15 and 16 are loading with cluster 3 with a proportion of 99% times, nodes20, 21, 22, 23 and 24 are loading with cluster 4 with a proportion of 99 to 100% times and nodes 24, 25 and 26 loading with cluster 5 with a proportion of 99% times in the bootstrapped sample. Hence all these items are highly stable. Thus it can be concluded from the table that all the nodes are showing consistent loading with their original cluster/dimension.
Estimation of Centrality Indices
As clear from the centrality indices in Fig. 6 above, the nodes 24 and 25 has the highest strength of 0.9 and 1 respectively with highest node 25 having the highest expected influence of 1. The strength of the nodes is in the range of 0.6 to 1 and the expected influence of the nodes is also in the range of 0.6 to 1. The strength centrality denotes number of direct connections of a node in a network. Expected influence measure the sum of all the edges extending from a given node in a network. Robinaugh, Millner, & McNally (2016). The closeness is determined by the path length, which implies the number of edges it takes to move from one node to another. The betweenness shows the relative number of shortest paths passing through the respective nodes in the network. It is revealed that nodes 9 and 10 has the highest betweenness and closeness among all the other nodes in the network. The perusal of the values of all the centrality indices including the strength, expected influence, betweenness and closeness clearly indicate that the values are fairly effective. It is pertinent to mention here that the nodes 24 and 25 score highest in strength, while as node 25 scored highest in expected influence. Among the betweenness and closeness, the nodes 9 and 10 showed the highest scores.
Estimation of the Accuracy of the Edges of Internet Gratification Network Structure
It is clear from the Fig. 7 above that all the edge weights for almost all the nodes in the network are close to their bootstrap mean, thereby implying that the values fall within the acceptable level of accuracy.
Estimation of Stability of the Edges of Internet Gratification Network
The bootstrapped replication was conducted for 500 times, the correlation with respect to edges stability between original and reduced sampled cases was found to be stable even after the sampled cases were reduced to 40% of the original sample of 662 which implies that the order of edge weights between nodes as displayed in Fig. 8 remained intact even when the sample size is reduced to 265 samples. The Correlation Stability Coefficient revealed between 0.5 to 1 is well within the acceptable benchmark of 0.5 (Epskamp, Borsboom and Fried, 2018)