CCECGP: causal consistency model of edge–cloud collaborative based on grouping protocol

At present, most causal consistency models based on cloud storage can no longer meet the needs of delay-sensitive applications. Moreover, the overhead of data synchronization between replicas is too high. This paper proposes a causal consistency model of edge–cloud collaborative based on grouping protocol. The model is based on the edge–cloud collaboration architecture, partitions cloud data centers and groups edge nodes by distributed hash tables, and stores a subset of the complete data set in nodes located at the edge of the network, thereby realizing partial geo-replication in edge–cloud collaboration environment. At the same time, we design a group synchronization algorithm called Imp_Paxos, so that the update only needs to be synchronized to the main group, which reduces the visibility delay of the update and decreases the data synchronization overhead. Besides, a sort timestamp is proposed in this paper, which generates different timestamps according to the type of update to track causality, keeping the amount of metadata managed in a relatively stable and low state. Therefore, the proposed model reduces the overhead of metadata for system management and improves throughput quantity of system. Experiments show that our model performs well in terms of throughput, operation latency, and update visibility latency compared with existing causal consistency models.


Introduction
As the continuous expansion of the network scale, the rapid increase of Internet devices, and the explosive growth of data, only relying on cloud storage and cloud computing cannot meet the client's demand for high-performance and realtime services [1]. The communication distance between the client and the cloud data center is so far, which inevitably leads to high latency, and cannot adapt the requirements of delay-sensitive applications such as automatic driving and multiplayer online games [2]. In addition, all data in the traditional cloud storage architecture are centrally managed and calculated by the central cloud, resulting in huge network bandwidth overhead.
Therefore, the edge-cloud collaborative architecture has become a feasible solution [3]. According to the white paper report on edge computing and cloud computing collaboration released by the Edge Computing Industry Alliance (ECC) and the Industrial Internet Industry Alliance (AII) in November 2018, edge computing is a distributed open platform that integrates network, computing, storage, and application core capabilities, and locates on the network edge side near objects or data sources [4]. The relationship between edge computing and cloud computing is not a substitute, but a complementary synergy. The close collaboration between edge computing and cloud computing can better meet and match various demand scenarios, thus amplifying the application value of edge computing and cloud computing. Edge computing is closer to the execution unit, and it is also the collection and preliminary processing unit of high-value data required by the cloud, which can better support cloud applications. At the same time, the business rules or models optimized by cloud computing through big data analysis can be distributed to the edge side, so that edge computing can run based on new business rules or models [5]. The edge end of the edge cloud collaborative architecture adopts the distributed multi-copy storage mode to ensure the high fault tolerance and high availability of the system. How to ensure data consistency between replicas is an important problem that must be solved [6].
Data consistency is divided into strong consistency and weak consistency [7]. Although strong consistency has simple semantics, it will cause phenomena such as high latency and intolerance of network partitions. Weak consistency does not require the timeliness of data updates, so it is difficult to ensure data freshness [8]. Causal consistency is a protocol between strong consistency and weak consistency [9]. It can not only solve the problems of high latency and intolerance of network partitions of strong consistency, but also avoid client anomalies caused by stale reads of weak consistency.
Most of the existing causal consistency schemes are based on cloud storage architectures and employ geographic replication strategies. Geographic replication strategies are divided into full geographic replication strategies and partial geographic replication strategy [10]. Systems based on full geo-replication such as GentleRain [11] use scalar timestamp based on physical clocks to track causal dependencies. And GentleRain also sets a global stable time (GST) to judge whether updates are visible. However, the client needs to wait until the GST of the remote data center to be larger than the local timestamp when querying for the updates, resulting in an increase in the visible latency of updates in the system. Okapi [12] proposes a global stabilization strategy that combines a general stability vector (GSV) with a hybrid logical clock to reduce the update visible delay. However, as the number of data centers increases, the overhead of maintaining system metadata is too large. CausalSpartan [13] proposed a data center stability vector (DSV) to track causality more accurately while using hybrid logic clocks. Nonetheless, the update needs to be synchronized to all replicas, so that the update visibility delay is still high. In 2019, Nejati et al. [14] improved the CausalSpartan scheme, proposed strict time causal consistency STCC, and used the distributed user operation table (DUOT), which further reduced the update delay and improved the system throughput. At the same time, STCC cuts down the probability of stale reads. Since a full geo-replication system needs to synchronize updates to all replicas before they become visible, it will inevitably lead to a high latency of update visibility. Recent systems have paid more attention to partial geo-replication strategies [15]. Partial geographic replication policies require that updates only need to be synchronized to the data centers that stores a common subsets. Systems based on partial geo-replication strategy, such as Saturn [16], use a tree topology and dissemination method that separates data and metadata, which decreases the overhead of metadata management. In 2018, Tariq et al. [17] proposed the Karma model, whose dynamic ring bonding mechanism is able to guarantee full availability under a single availability zone failure and a simple network partition mode. PaRiS proposed by Kristina et al. [18] tracks causal dependencies by a new protocol called universal stability time (UST), which reduces read latency and update propagation cost.
However, the metadata storage and management overhead of the traditional consistency model based on cloud storage are high, and the client has a high communication delay when communicating with a distant cloud data center, which cannot satisfy many low-latency-demand applications. Therefore, the consistent storage scheme of edge-cloud collaboration has received concern and attention. In 2018, Harshit Gupta et al. [19] proposed FogStore as an event-based key-value storage system that exploits the contextual relevance of events (data) in context-aware applications, and provides clients with different consistency guarantees. PathStore proposed by SH Mortazavi et al. [20] is a hierarchical storage system whose updates are propagated through a hierarchy in the background, providing eventual consistency. In 2020, Karim Sonbol et al. [21] proposed a decentralized edge storage system EdgeKV, which provides fast and reliable data storage, realizing data replication with strong consistency guarantees. Through location transparency and interface-based design, EdgeKV can scale with heterogeneous systems of edge nodes. Gesto [22] is a causal consistency model of edge-cloud storage that combines high throughput and low migration latency, reducing metadata overhead for system management, and client migration latency by using multi-part timestamp. In 2021, MI Naas et al. [23] proposed two strategies to manage IoT data replication and consistency in fog infrastructure. Their strategy selects the correct replica number and its location for each data to reduce data access latency and replica synchronization costs. The above research shows that the traditional cloud computing distributed storage architecture has gradually developed toward edge computing.
Most of the distributed models previously applied to the edge computing environment choose strong consistency or eventual consistency, but problems such as high update delay caused by strong consistency and stale reads caused by eventual consistency cannot be well adapted to the edge computing environment. In addition, in the causal consistency model Gesto, based on edge cloud storage, uses the synchronization algorithm Multi-Paxos between cloudlets. The implementation efficiency of this protocol is fixed, so it will soon reach the upper limit of bandwidth, leading to a decline in system performance and an increase in communication overhead.
In this regard, we propose CCECGP (Causal Consistent Model of Edge-Cloud Collaborative Based on Grouping Protocol), which groups the complete dataset through a distributed hash table, and decentralizes storage subsets in edge nodes to implement partial geo-replication. In addition, a grouping synchronization algorithm Imp_Paxos and a sort timestamp are also designed, which declines the visible update delay, reduces the system management metadata and communication overhead, thereby increases the throughput. What's more, it also reduces the probability of stale reads. Therefore, it can better adapt to the edge storage environment and low latency demand application scenarios. The main contributions of this paper are as follows: (1) The cloud data center is partitioned by using the distributed hash table, and the complete data set is divided into multiple subsets strictly according to the distributed hash table and stored in the edge nodes, which realizes the highly scalable partial geo-replication and improved system fault tolerance. (2) We design a group synchronization algorithm Imp_Paxos. By selecting the highefficiency nodes in the edge group as the main group, the update visibility delay and the probability of stale reads are reduced, and the data synchronization overhead between replicas is decreased. (3) A sort timestamp is proposed. CCECGP can generate two different timestamps according to the update type to track the causality of local and remote updates. The timestamp is independent of the number of edge groups and the number of nodes in the group. As a result, the metadata management overhead of the system can be reduced and the throughput is improved.
CCECGP is compared with two causal consistency models, STCC and Gesto. STCC is a causal consistency model based on cloud storage and full geo-replication, and Gesto is a causal consistency model based on edge-cloud storage and partial geo-replication. The experimental results show that the proposed model's edge-cloud collaboration architecture and partial geo-replication strategies are superior to STCC, Imp_Paxos algorithm and sort timestamp are superior to Gesto's Multi-Paxos algorithm and multipart timestamp.

3
CCECGP: causal consistency model of edge-cloud collaborative… 2 System design 2.1 System model CCECGP stipulates that the distributed key-value storage system is divided into two layers. First is the location of the cloud data center called the cloud layer. All edge nodes constitute the another part of the system called the edge layer. The system model is shown in Fig. 1.
In the edge layer, the edge nodes distributed at the edge of the network are closer to the client than the cloud, which can reduce the high communication delay between the client and the cloud data center. The storage and computing tasks of the cloud data center are dispersed to edge nodes, lessening the communication and storage overhead of the system. Firstly, some edge nodes with nearer geographical location are divided into an edge group, and each edge group runs the Imp_Paxos group synchronization algorithm. Then, the Imp_Paxos algorithm divides several high-efficiency nodes in the edge group into a group, which is called the main group, and the remaining nodes in the edge group are called the secondary group. The number of edge groups is determined by the number of subsets that the full dataset is divided into. All nodes in the group store replicas of the exact same state (key-value pairs) to A cloud data center has n partitions. The complete data set is divided into n subsets by distributed hash table, which are stored in n partitions, respectively. And the subsets in each partition are strictly assigned to an edge group to ensure the data subsets are different from each other between groups. Therefore, partial geo-replication is accomplished. The hash value (location information) of each edge group is stored in the cloud data center as a directory, which makes clients' migration more easier.

Distributed hash table
Distributed hash table (DHT) is a distributed storage method that stores information uniquely identified by key values on multiple nodes according to a certain protocol, which can effectively avoid "centralized." The entire network is paralyzed by a single failure of a server (such as a tracker), reducing the risk of all data being corrupted [24]. Distributed hash tables usually use a consistent hash function to perform uniform operations on nodes and keys. There are many techniques for implementing DHT, and this paper adopts the Chord algorithm. Chord guarantees consistent hashing by mapping nodes and keys to the same space. In order to ensure non-repetitive hashing, SHA-1 [25] is chosen as the hash function. In theory, the nodes on the Chord ring should be randomly distributed, but if the number of nodes is not large, it will lead to uneven distribution of nodes, so consider adding virtual nodes to increase the balance.
As shown in Fig. 2, it is a Chord ring, and the nodes mapped on the ring are called node. The node before each node is called the predecessor, and the subsequent node is called the successor. The first precursor is called the direct precursor and the first successor is called the direct successor. Each node maintains a pointer table, as shown in Table 1. The table length is m (m is a digit, 160 in Chord). Item ith in the table stores the n + 2 i−1 mod2 m successors (1 ≤ i ≤ m) of node n, that is, the The modulus is taken because the successors of the last node are the beginning nodes in the ring structure. For example, the next node of the largest node is defined as the first node. Each node also maintains a precursor table and a successor table to quickly locate the predecessor and successor of the node.
Clockwise along the Chord ring, the key is stored on the first node of the hash (Node) ≥ hash (key), and the node is called the successor of the key. Therefore, for a given key, finding its node is to find its successor. The search process is as follows (assuming that the search occurs on node n): ① Check whether the hash of the key falls between node n and its direct successors. If the search is completed, the direct successors of n are found; ② In the pointer table of n, find the successor of n that is closest to the hash (Key) and less than the hash (Key). This node is also the closest predecessor to the Key in the pointer table, and forward the search request to this node; ③ Repeat the previous two steps until the node corresponding to the Key is found.

Sort timestamp
Sort timestamp (sort_ts) is a vector timestamp based on hybrid logical clocks that can accommodate clock skew and workload skew. Sort timestamp generates two different timestamps based on the type of update. One is an edge group timestamp (ets), which is only valid in an edge group to track local updates. The other is sent to other remote edge groups for tracking remote updates, called global timestamp (gts). Both two timestamps consist of a < type, src, clock > tuple, where type represents the update type, including local update or remote update. src is the source field, pointing to the source copy that generated the update. clock denotes the clock field, that is, the real time when the timestamp was generated. The src of the sort timestamp always points to the original replica node that initiated the update.
The client tracks the causal dependencies of the latest local update by ets, which is only valid in that edge group. ets is generated by the replica that initiated the update and can only be compared with ets generated by other replicas in the same edge group. When the replica detects that the update type is remote update, it will generate a global timestamp gts for the update operation and send the request to the cloud data center. After returning the location of the target node from the cloud data center, the client will perform migration operation. Clients use gts to track causal dependencies for remote updates. gts allows updates generated in replicas of different edge groups to be sorted.
CCECGP generates a sort timestamp (ets, gts) for each update, and the client also maintains a sort timestamp for sending to the target replica when connecting. Using a single global timestamp (gts) to track all remote causal dependencies prevents clients from storing different edge group timestamps (ets) for many replicas in an edge group. The server combines all remote updates in a single entry could keep the amount of metadata small, reducing overhead. CCECGP can track local updates more accurately than remote updates. So we expects the client to perform more local operations.

Imp_paxos algorithm
In the causal consistency model Gesto based on Multi-Paxos algorithm, each edge group is distributed with n edge nodes, and each node provides services for multiple clients. When a client sends a request to an edge node, the Multi-Paxos protocol requires the node to which the client is currently connected to send the requested content to all other nodes. The request is considered complete when more than half of the nodes reply. Since the execution efficiency of the protocol is fixed, the upper limit of the bandwidth is quickly reached, resulting in a degraded system performance. In addition, Gesto's read operation does not need any synchronization. Although this will reduce the performance of the read operation, it will also increase the probability of stale reads. To solve the above problems, we propose the Imp_ Paxos algorithm in this paper.
Imp_Paxos algorithm performs in each edge group. Imp_Paxos algorithm will randomly select one node as the efficiency calculator at set intervals (record as Δt). All other nodes need to send their own indicator parameters to the calculator node every Δt time. The calculator node filters out several efficient nodes in the edge group by calculating four indicators and divides them into the main group, while other nodes in the group are classified into the secondary group. That means the edge group will reselect the main group every Δt time. This method of dynamic selection of the main group can ensure that the main group is always composed of the most efficient nodes and avoid the problem that the efficiency value of each node in the edge group changes when the delay increases due to the movement of the edge node or the data access frequency changes.
For the request sent by the client, the Imp_Paxos algorithm requires that the current node only needs to synchronize the request to the main group nodes in the edge group. After the synchronization of the main group node is completed, the request is considered to be completed. When bandwidth resources are sufficient, the main group synchronizes the updates to the secondary group nodes. Therefore, the system can complete the request in a shorter time with a smaller data transmission volume, thereby saving bandwidth, reducing synchronization overhead, and improving system performance.

Algorithm definition
Since the nodes are distributed in different environments, the processing power and communication quality of the request are different. Therefore, the main group can be selected based on the difference of the nodes. During data replication process, latency directly affects user experience, so the average transmission delay is selected as the first indicator. The communication environment, communication capability and communication distance of nodes will affect the transmission delay. Since the communication environment of nodes in the same edge group is similar, and the communication capability of different nodes is influenced by too many factors. In order to reduce the computing resource cost of the calculator node, we abstract the average transmission delay as the communication distance between two nodes, because the farther the distance is, the higher the delay is. The processing capacity of the node is also very important. At the same time, the more requests a node handles, the stronger its processing capability. Therefore, the node processing capacity is used as the second indicator. In addition, the frequency of client access to the data is also different. Since data that is accessed frequently has a greater impact on bandwidth usage, the frequency of data access is used as the third indicator. The three indicators for selecting major groupings are defined as follows.

Definition 1 Average transmission delay.
In time Δt, we assume that there is no movement between the two nodes, so the distance is fixed. The average transmission delay of a single node is denoted by t i . It is defined as the ratio of the total transmission delay of this node and all other nodes to the total number of nodes N. Definition 2 Node processing capacity.
The processing capacity of a single node is denoted by p i . It is the ratio of the number of requests processed to the total number of client requests in time Δt.

Definition 3 Data access frequency.
The access frequency of data in a single node is denoted by f i . It refers to the ratio of the number of accesses to node data to the total number of accesses over Δt time. (1) We expect to select several nodes with low latency, high node processing capacity and high data access frequency as the main group. The Imp_Paxos algorithm defines the above three indicators for each node and integrates the three indicators to obtain an efficiency measurement indicator e i , which is used to judge the priority of the nodes to form the main group. Defined as follows.
The efficiency measurement e i of a single node is the ratio of the data access frequency of the node to the average transmission delay and waiting latency, where β is a system parameter.
Since the synchronization of nodes in the main group means the synchronization of the edge group is completed, and the efficiency measurement of all nodes in the edge group is different, the nodes in the main group must be sufficiently representative and the main group must be unique. The Imp_Paxos algorithm requires that the sum of the efficiency measurement of the nodes in the main group must be greater than half of the sum of all nodes to ensure that two main groups do not appear at the same time. In order to reduce the request response time, the algorithm also requires that the number of nodes in the main group does not exceed half of the number of all nodes.

Optimization objective function
Under the above requirements, multiple node sets that meet the conditions can be obtained, so it is necessary to construct an optimization objective function to find the best main group, which satisfies high efficiency and has a small number of nodes. By balancing the efficiency index and the number of main groups, the function is constructed as follows: where e is a vector consisting of node efficiency measurement. j is a vector which reflects whether the node belongs to the main group (j = 0 or 1, "1" means the node is in the main group, "0" means the node is outside the main group). The second half is the percentage of nodes requesting to be synced, where α is a system parameter.

Data synchronization strategy
When a client initiates update operations to an edge node, the Imp_Paxos algorithm performs the following synchronization strategy.

Synchronization strategy within edge groups
The client first writes the new value of the corresponding key to the currently connected edge node, and the current node sends the update to the nodes in the main group. After the nodes in the main group are synchronized, the update is considered complete. When the bandwidth resources are sufficient, the main group sends the latest value of the key to other nodes to complete the synchronization of the entire edge group and records the synchronization time t. This can avoid the waste of resources caused by frequent synchronization of the main group to other nodes when the same key is updated multiple times in a short period of time. As a result, the overhead is reduced.

Synchronization strategy of edge group to cloud data center
The Imp_Paxos algorithm requires all edge groups to periodically upload data to the cloud data center for synchronization. Assuming that the edge group directly sends the data updated each time to the cloud, the cloud data center will waste a lot of bandwidth and computing resources if multiple consecutive requests are for the update of the same data. Therefore, it is necessary to judge the uploaded data: Whether the data have been modified within time T. If the data have not been modified, the data are directly synchronized to the cloud. If the data have been modified, it is judged whether the time of uploading to the cloud last time exceeds T. If it exceeds T, upload the data to the cloud. Otherwise cached in the edge group, waiting for new updates. If the data are updated frequently within a short period of time, only the latest data will be uploaded to the cloud. T is dynamically set to the maximum value of the synchronization completion time t of all edge groups, that is,

CCECGP protocol
CCECGP is a causal consistent communication protocol that runs stably in a distributed storage system. The design of the protocol is based on the edge-cloud collaborative architecture. The data stored in the cloud data center is distributed and stored in the small data center at the edge of the network by the distributed hash table.
Expanded cloud storage services to bring data closer to the client. The definitions of the symbols used in the protocol are shown in Table 2.

Client operation
The algorithm execution process is as follows. The client first sends read and write requests to the node closest to its geographical location (called the preferred node). If the preferred node stores corresponding data items, it will first request synchronization from the nodes in the main group. Then return the result to the client when the synchronization is completed. If the preferred node does not store the corresponding data item, the client will migrate and change the preferred node according to the received reply.

Read operation
Clients do not need to provide their client-side sort_ts when performing read operations. As shown in the READ algorithm in Algorithm 1, the client merges its current client_sort_ts with the returned timestamp. If the update is generated in the same node, the largest of the client edge group timestamp and the edge group timestamp in the same group is selected for merging.

Update operation
The remote update operation flow is shown in Fig. 2. The client first creates a unique identifier for each update operation. Then sends the update request to the preferred node along with its client_sort_ts, where the client sort timestamp contains the update type. The client performs an update or migration operation according to the reply from the preferred node, and merges it with the timestamp returned by the preferred node (as shown in the UPDATE algorithm in Algorithm 1). The execution steps are as follows: ① The client generates the update operation identifier update_id. ② Sends clent_sort_ts and update_id to the preferred node. ③ The preferred node sends the update request to the cloud. ④ The cloud data center finds the location of the edge group where the corresponding key is located according to the distributed hash table. ⑤ Cloud data center returns the location information to the preferred node. ⑥ The preferred node returns the location information to the client. ⑦ The client performs the migration operation according to the location information and changes the preferred node to the target node. ⑧ The target node synchronizes to the main group after the update is completed. ⑨ The main group completes the update synchronization. ⑩ The main group synchronizes updates to other nodes in the edge group and cloud according to the data synchronization strategy of the Imp_Paxos algorithm (Fig. 3).

Migrate operation
Clients perform read and write operations by connecting to their closest preferred node. But if the preferred node does not store the data item requested by the client or the client is geographically closer to another node, the client will perform migration operation (as shown by the MIGRATE algorithm in Algorithm 2). That is, the process of changing from a currently connected node to another connected node. The preferred node first sends the remote read/write request to the cloud data center. Then, the cloud data center finds the location of the edge group where the requested key is located according to the distributed hash table and sends the location of the target node back to the preferred node. Finally, the preferred node returns the location of the target node to the client.
When the client receives the location of the target node from the preferred node, it first sends a connection request to the target node. After receiving the reply from the target node, client will change the preferred node to the target node and merge the returned sort timestamp with the client's own sort timestamp to complete the migration.

Connect operation
In order to perform the migration operation, the client needs to send a connection request to the target node (as shown by the CONNECT algorithm in Algorithm 2). The parameters of this request include the client's sort timestamp and preferred node information. In order to successfully migrate to the target node, the client must wait for the node to apply all past operations. The target node keeps track of the latest updates it applied from all remote groups (NewestUpdate) and the maximum global timestamp (MaxCloudTS) within the local edge group. As long as MaxCloudTS and NewestUpdate [cloud] are, respectively, greater than or equal to the gts and ets of the client's sort timestamp, the connection request can be satisfied.
The target node generates a new sort timestamp. Since the client has not observed any update of the target node at this time, the edge group timestamp part of the new sort timestamp is set to 0, and the global timestamp part is set to the latest update observed by the client (that is, keep the maximum value of gts from the latest update applied by the target node from this edge group and the last operation the client observed from other nodes). Finally, target node returns this new sort timestamp to the client.

Edge node operation
As shown in Algorithm 3, the node processes the request by returning the stored value and returns it to the client along with the corresponding timestamp.

Respond to read operations
When the preferred node receives a local read request from the client, the preferred node forwards the request to the main group. Then, the main group synchronizes the latest value of the request key to the preferred node, and the preferred node returns the value to the client. If the node does not cache the target key, which means it is a remote read request, the preferred node will send the read request to the cloud data center. After receiving the reply from the cloud data center, the preferred node returns the location of the target node to the client.

Respond to write operations
The preferred node performs operations according to the update type as follows (as shown in the UPDATE algorithm in Algorithm 3). When a local update is received and the current node stores the target key, it is directly updated and synchronized to the main group, otherwise return an error (that is, the current node has not stored the target key). When a remote update is received, the node will create a new sort timestamp for the update operation, where the global timestamp part is preserved and a new edge group timestamp part is generated with a clock higher than the most recent operation recorded by the client. So the new timestamp is guaranteed to be greater than any relevant timestamp the client has observed, and this is also the key to guaranteeing causal consistency when the client migrates. Then, the node sends the write request to the cloud, and after receiving the reply from the cloud data center, the target node location is sent to the client for client migration.

Imp_paxos algorithm
The Imp_Paxos algorithm runs in edge groups as shown in Algorithm 4, where EG represents the edge group. The Imp_Paxos algorithm first obtains three indicators from all nodes in the edge group and calculates the efficiency metric e i and the main group identifier j i . Then Imp_Paxos generates a vector e composed of efficiency measures and a vector j composed of main group identifiers, finally select the optimal main group by optimizing the objective function Opt.

Cloud data center operations
As shown in Algorithm 5, when the cloud data center receives the remote request sent by the preferred replica, the cloud calculates the location of the remote target edge group according to the distributed hash table and sends the location back to the preferred replica.

3 5 Evaluation
This section evaluates the performance of CCECGP through simulation experiments. CCECGP is compared with STCC and Gesto in the same experimental environment. The advantages of CCECGP are shown by analyzing and comparing the effects of different parameters on performance.

Experimental environment
Unless extra specified, the default experimental parameters are set as follows: For CCECGP, the cloud data center sets 3 partitions (that is, 3 servers), which are evenly distributed in the distributed hash table, corresponding to 3 edge groups. Each edge group contains 6 edge nodes. For Gesto set up 3 brokers and 18 cloudlets. For STCC, 3 data centers are set up, and each data center has 6 partitions (i.e., 6 servers). The initial number of records in all systems is 3000, the number of operations is 3000, the length of the data field is 10 bytes, and the read/write ratio is 90:10. Each experiment is repeated 3 times, and the running time of each experiment is about 5 min (Table 3).
Based on the above experimental environment and settings, this section tests and compares CCECGP, Gesto and STCC in terms of throughput, operation response time, etc. The test results are as follows.

Throughput
Throughput refers to the number of key-value pairs updated by the model per unit time, representing the speed at which the client updates key-value pairs. It is an important indicator for measuring system performance. This experiment evaluates the performance of system throughput by increasing the number of edge nodes, changing the workload, and increasing the number of clients. Figure 4 depicts the effect of the number of edge nodes on the system throughput. When the number of edge nodes increases from 0 to 18, the throughput continues to rise until it reaches the maximum value when there are about 10 edge nodes, which is basically in line with Amdahl's law (when the read/write ratio is 90:10 (that is, when the parallel computing ratio is 0.9, the theoretical maximum speedup ratio is about 10). When continue to increase the number of nodes, the system cannot process all requests completely in parallel, so the system becomes bottlenecked.
Since STCC is based on a full geo-replication strategy, each update needs to synchronize replication messages to all replicas. As a result, the increase in the number of edge nodes will lead to a significant increase in the overhead of managing metadata. On the other hand, CCECGP and Gesto are based on partial geo-replication, so the throughput is higher than STCC. Gesto is based on the Paxos algorithm and needs to be synchronized to the cloud and more than half of the replicas for each update. While CCECGP uses the Imp_Paxos algorithm, each update only needs to be synchronized to the nodes in the main group, so CCECGP shows higher throughput than Gesto. The experimental results show that under the same conditions, CCECGP has an average 33.15% higher throughput than STCC and 17.83% higher throughput than Gesto. Figure 5 shows the effect of simulated workload on system throughput by varying the read/write ratio, which is set to the default value above. The read/write ratio is set to 25:75, 50:50, 75:25, 90:10, 99:1. As the ratio of read operations increases, the throughput of the three models shows an upward trend. Since STCC is based on a full geo-replication strategy, more metadata is stored in the DC, which leads to lower throughput. As can be seen from the figure, when the read/write ratio is greater than 50:50, the throughput of CCECGP is slightly lower than that of Gesto, because the read operation of CCECGP needs to synchronize from the main group to ensure that the latest data are read. The read operation of Gesto does not require any synchronization. The replica directly processes the read operation by returning the value of the corresponding key. Therefore, as the proportion of read operations increases, the throughput of Gesto is improved more significantly.
In this experiment, the read/write ratio is fixed at 90:10, and the number of clients is continuously increased to achieve the peak throughput of the system. The results are shown in Fig. 6. Due to STCC needs to maintain the distributed user operation table (DUOT), the massive overhead caused by maintaining metadata seriously affects its throughput as the number of clients increases. In contrast, both the multipart timestamps used by Gesto and the categorical timestamps used by CCECGP can keep metadata to a relatively stable and small order of amount. Compared with the Paxos algorithm used by Gesto, the Imp_Paxos algorithm used by CCECGP can reduce the resources and overhead required for synchronization between edge nodes, save bandwidth, and improve system throughput.

Operation delay
In this section, we test the effect of the number of edge nodes on the latency of read operations, the visible latency of updates, as well as the visible latency of updates under different numbers of clients. The experimental results are, respectively, shown in Figs. 7a, b, and 8.
As can be seen from Fig. 7a, b, the read latency of Gesto is lower than that of CCECGP and STCC. The reason is that when STCC performs the read operation, the client needs to access the DUOT, and the read operation of CCECGP needs to synchronize from the main group firstly. However, the read operation of Gesto Platform DKVF [26] does not need any synchronization. Therefore, Gesto has the lowest read latency. CCECGP's update visible latency is lower than Gesto and STCC, because STCC is based on full geo-replication, and each update needs to be synchronized to all replicas. Each update of Gesto needs to implement the data synchronization strategy according to Paxos algorithm, and Gesto uses the method of separate transmission of data and metadata, which is easy to cause the delay of update waiting. The update operation of CCECGP follows the Imp_Paxos algorithm and only needs to synchronize updates to the nodes in the main group, so it has a lower update visible delay. The results show that the average read latency of CCECGP is 27.89% lower than that of STCC and 5.52% higher than that of Gesto, while the average update latency of CCECGP is 19.52% lower than that of Gesto and 28.29% lower than that of STCC. This experiment simulates different network environments by changing the number of clients, in which the client read/write ratio is set to 50:50 to reduce the impact of client operation types on model performance. The update visibility delay of the system is shown in Fig. 7. As the number of clients increases, the update visibility delay for all three models increases. When the number of clients exceeds 100, it is considered that the network environment is congested. At this time, the update visibility delay growth rate of Gesto and STCC models is greater than that of CCECGP. The reason is that for each update operation, the Imp_Paxos algorithm requires that the update only needs to be synchronized to the master group node, so the number of replica nodes that CCECGP needs to synchronize is less than that of Gesto and STCC. Therefore, CCECGP has a lower update visible delay.

Data staleness
Data staleness refers to the probability that a client accesses stale data when accessing certain data in the system. By comparing data staleness, the consistency level of the system can be obtained. In general, the higher the consistency level, the higher the system reliability. Figure 9 depicts the comparison of data staleness when the three models perform read operations. According to the Imp_Paxos algorithm, CCECGP needs to request synchronization from the main group every time it performs a read operation to ensure data freshness. STCC read operations require clients to access DUOT, which can also maintain a low data staleness. Due to the read operation of Gesto does not require any synchronization. Although the latency of the read operation is reduced, the probability of reading stale data is increased.

Conclusion
In this paper, we propose a partial geo-replication causal consistency model CCECGP based on the grouping protocol, which is applied to the edge-cloud collaborative storage architecture. The model partitions the cloud data center through a distributed hash table and groups and stores a subset of the complete data set in the edge nodes according to the distributed hash table, thereby realizing partial geographic replication. By extending the cloud storage architecture to the edge-cloud collaborative storage architecture, the data are brought closer to the client, which effectively reduces the client read and write latency and improves the system throughput. This paper also designs a group and synchronize algorithm Imp_Paxos, which dynamically selects the high efficient nodes in the edge group as the main group by calculating four indicators, thereby decreasing the update visibility delay and data staleness, saving bandwidth and reducing synchronization overhead between replicas. CCECGP also generates and maintains a sort timestamp according to the type of client update operation, which keeps the metadata at a relatively stable and low level, therefore reduces the overhead caused by managing a large amount of metadata. The experimental results show that by using the Imp_Paxos algorithm and sort timestamps in the edge-cloud collaborative storage environment, CCECGP achieves the reduction of read and write operation delay, the lessening of update visible delay and the increase of system throughput.
Compared with the traditional causal consistency model based on cloud storage, CCECGP improves the system performance to a certain extent by combining edge storage, using the Imp_Paxos algorithm and sort timestamps. However, the application scenarios of this model are still limited. How to design the structure and communication mode between cloud data centers to expand the application scope of CCECGP, and how to further reduce the read operation delay while ensuring data freshness will be the focus of future work.
Author contribution JT and HJ conceived the scheme, HJ conducted the experiment and wrote the manuscript. JT, HJ and WB reviewed the manuscript.