Scheduling computations with provably low synchronization overheads

We present a Work Stealing scheduling algorithm that provably avoids most synchronization overheads by keeping processors’ deques entirely private by default and only exposing work when requested by thieves. This is the first paper that obtains bounds on the synchronization overheads that are (essentially) independent of the total amount of work, thus corresponding to a great improvement, in both algorithm design and theory, over state-of-the-art Work Stealing algorithms. Consider any computation with work T1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{1}$$\end{document} and critical-path length T∞\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\infty }$$\end{document} executed by P processors using our scheduler. Our analysis shows that the expected execution time is OT1P+T∞\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O\left( \frac{T_{1}}{P} + T_{\infty }\right) $$\end{document}, and the expected synchronization overheads incurred during the execution are at most OCCAS+CMFencePT∞\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O\left( \left( C_{\mathrm{CAS}} + C_{\mathrm{MFence}}\right) PT_{\infty }\right) $$\end{document}, where CCAS\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_{\mathrm{CAS}}$$\end{document} and CMFence\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_{\mathrm{MFence}}$$\end{document}, respectively, denote the maximum cost of executing a Compare-And-Swap instruction and a Memory Fence instruction.

Abstract Work Stealing has been a very successful algorithm for scheduling parallel computations, and is known to achieve high performances even for computations exhibiting fine-grained parallelism.In Work Stealing, each processor owns a deque that uses to keep track of its assigned work.To ensure proper load balancing, each processor's deque is accessible not only to its owner but also to other processors that may be searching for work.However, due to the concurrent nature of deques, it has been proven that even when processors operate locally on their deque, synchronization is required.Moreover, many studies have found that synchronization related overheads often account for a significant portion of the total execution time.For that reason, many efforts have been carried out to reduce the synchronization overheads incurred by traditional schedulers.
In this paper we present and analyze a variant of Work Stealing that avoids most synchronization overheads by keeping processors' deques entirely private by default, and only exposing work when requested by thieves.Consider any computation with work T 1 and critical-path length T ∞ executed by P processors using our scheduler.Our analysis shows that the expected execution time is O T1 P + T ∞ , and the expected synchronization overheads incurred during the execution are at most O ((C CAS + C M F ence ) P T ∞ ), where C CAS and C M F ence respectively denote the maximum cost of executing a Compare-And-Swap instruction and a Memory Fence instruction.The second bound corresponds to an order of magnitude improvement over state-of-the-art Work Stealing algorithms that use concurrent deques.

Introduction
Multithreading is a fundamental mechanism to leverage the parallel design of multiprocessor chips.However, the performance of multithreaded computations is tightly bound to the efficiency of the underlying thread scheduler, responsible for keeping processors busy.
For some time now, the Work Stealing algorithm is one of the most popular for scheduling multithreaded computations.In Work Stealing, each worker (usually referred to as processor ) owns a double-ended queue (deque) of threads ready to execute.This deque is locally manipulated as a stack, similarly to a sequential execution: processors push and pop threads from the bottom side of their deque when, respectively, a new thread is spawned and the execution of the current thread concludes.Additionally, whenever a pop operation finds the local deque empty, the processor becomes a thief and starts targeting other processors -called its victims -uniformly at random, with the purpose of stealing a thread from the top of their deques.
As shown by Blumofe et al. in [11], Work Stealing is provably efficient for scheduling multithreaded computations.However, due to the concurrent nature of processors' deques, the use of appropriate synchronization mechanisms is required to ensure correctness [9].Work Stealing schedulers, effectively employed in systems such as Cilk [10,25], X10 [15], Intel TBB [23], Wool [20], Java Fork/Join [24], Hood [13], Pasl [3], among others, require the use of such synchronization mechanisms every time a processor adds or retrieves a thread from a deque.Consequently, even when processors operate locally on their deques, they incur expensive synchronization overheads that, in most cases, are unnecessary.
The first provably efficient Work Stealing algorithm, proposed by Blumofe et al. [11], assumed that all the steal attempts targeting each deque were serialized, and only ensured the success of at most one such attempt per time step.The idea was materialized in the Cilk run-time system [10] via a blocking synchronization protocol named THE.Although Cilk is known to be extremely efficient, Frigo et al. found that the overheads introduced by the THE protocol could easily get to more than half of the total execution time [21].Subsequent non-blocking versions of Work Stealing [8,12] mitigated part of these overheads by replacing the THE protocol with a nonblocking one that resorts to Compare-And-Swap (CAS) and Memory Fence (MFence) instructions.Even so, the synchronization costs imposed by these operations are not negligible.For instance, Morrison et al. tuned Cilk by removing a single MFence instruction (one that was executed whenever a processor tried to take work from its deque) and found that this single Memory Fence could account for as much as 25% of the total execution time [28].Unfortunately, it has been proved, by Attiya et al. in [9], that it is impossible to eliminate all synchronization from the implementation of concurrent deques, while maintaining correctness.Indirectly, this result implied the im-possibility of eliminating all synchronization from Work Stealing algorithms that use concurrent deques.
Various proposals have been made with the goal of eliminating synchronization for local deque accesses, by making deques partly or even entirely private [2,26,28,31,32,22,18].Although the use of private deques eliminates synchronization for local accesses, it introduces new overheads that are not present in Work Stealing with concurrent deques.Consequently, and even though steals are usually rare, efforts have been placed in the use of partially private deques, known as Split Deques, or SpDeques for short, to avoid the drawbacks of using entirely private deques [17,18,26,28,31,32].Like private deques, SpDeques avoid synchronization for local accesses by keeping their bottom part private, accessible only to their owner.However, SpDeques do not introduce extra overheads since, as in Work Stealing with concurrent deques, idle processors can take work directly from other processors' SpDeques without requiring their intervention.The earliest work to use SpDeques for reducing synchronization in Work Stealing was, to the best of our knowledge, by Dinan et al. in [18].In their paper, the authors showcased the practical advantages of using SpDeques in large scale distributed settings, by comparing the performance of their algorithm when using SpDeques versus when using concurrent deques.The advantages of SpDeques were clear and, since then, numerous studies have further explored their use, obtaining favorable empirical results [26,28,31,32].Nevertheless, it is still unknown whether the performance gains of using SpDeques can be reflected in theory, and whether Work Stealing with SpDeques enjoys the same guarantees of Work Stealing with concurrent deques.
In this paper we propose yet another variant of the Work Stealing algorithm, baptized Low Cost Work Stealing, that avoids most synchronization overheads by leveraging SpDeques to obtain the following four distinctive features: 1. Busy processors only expose work to be stolen (by placing it in the public part of the SpDeque) after being targeted by one or more steal attempts, which are issued by thieves trying to load balance.This allows processors to work locally on their work-queue without requiring any synchronization, imposing it only when load balancing may be required.
2. Work exposure requests are attended in constant time.This is crucial to keep the algorithm's execution time bounds a factor away from optimal.The requirement may be achieved by periodically checking for requests or by implementing an asynchronous notification mechanism.In this paper, for the sake of simplicity, we only focus on the former.
3. Processors only expose one thread of their local work at a time, contrasting with the "expose-half " approach of others, such as [32].Only so, synchronization for local operation can be eliminated when load balancing is only sporadically required.
4. All interactions between processors are completely asynchronous, making our algorithm viable for multiprogrammed environments.
The combination of these four features is the key for eliminating most of the unnecessary synchronization overheads that are present in Work Stealing implementations that use fully concurrent deques while maintaining the same theoretical guarantees.Our analysis shows that for a P -processor execution of a computation with total work T 1 and critical-path length (i.e.span) T ∞ , the expected total synchronization costs of Low Cost Work Stealing are at most O ((C CAS + C M F ence ) P T ∞ ), where C CAS and C M F ence respectively denote the synchronization costs incurred by the execution of a CAS and MFence instructions.This bound is tight and implies that for several classes of computations the use of SpDeques allows to reduce synchronization by an almost exponential factor when compared with conventional Work Stealing algorithms that use concurrent deques.Additionally, we prove that the expected execution time of Low Cost Work Stealing is O T 1 P + T ∞ , and briefly explain why the expected total communication of Low Cost Work Stealing is a constant factor away from the expected total communication of Work Stealing with concurrent deques, which in turn is known to be asymptotically optimal [34,11].These results justify the folk wisdom that SpDeques allow to effectively reduce synchronization while maintaining the same theoretical guarantees of Work Stealing with concurrent deques.
The rest of the paper is organized as follows: Section 2 introduces some preliminary concepts about how a computation is modeled; Sections 3 and 4, respectively, present and analyze the Low Cost Work Stealing algorithm; Section 5 compares the synchronization costs of our algorithm against Work Stealing algorithms that use concurrent deques; Section 6 compares our work against the current state-of-the-art, and finally; Section 7 draws our final conclusions.

Preliminaries
Like in much previous work [11,1,30,8,7,4,5,29,2], we model a computation as a dag (i.e. a direct acyclic graph) G = (V, E), where each node v ∈ V corresponds to an instruction, and each edge (µ 1 , µ 2 ) ∈ E denotes an ordering between two instructions (meaning µ 2 can only be executed after µ 1 ).Nodes with in-degree of 0 are referred to as roots, while nodes with out-degree of 0 are called sinks.Equivalently to Arora et al in [8], we make two assumptions related with the structure of computations.Let G denote a computation's dag:  The total number of nodes within a dag is expressed by T 1 and the length of a longest directed path (i.e. the critical-path length) by T ∞ .A node is ready if all its ancestors have been executed, implying that all the ordering constraints of E are satisfied.When a node becomes ready we say that it was enabled ; to ensure correction only ready nodes can be executed.The assignment of a node µ to a processor p means that µ will be the next node p executes.Finally, a computation's execution can be partitioned into discrete time steps, such that at each step, every processor executes an instruction.

Low Cost Work Stealing
In Low Cost Work Stealing, each processor owns a Lock Free SpDeque, instead of a typical concurrent deque.An SpDeque (illustrated in Figure 1) is simply a deque that is split into two parts: a private part and a public part.The public part lies in the top of the SpDeque whereas the private corresponds to the rest of the SpDeque.To avoid synchronization for local operations, only the owner of an SpDeque is allowed to access its private part.Furthermore, by default busy processors operate on the private part of their SpDeque, pushing and popping ready nodes as necessary.In fact, a busy processor only attempts to fetch work from the public part of its SpDeque if the private part is empty.In such situation, if the processor's attempt succeeds (i.e. if the public part of the processor's SpDeque is not empty), the obtained node becomes the processor's new assigned node.However, if the public part of the processor's SpDeque is also empty, the processor becomes a thief and begins a stealing phase.During stealing phases, thieves target victims uniformly at random and attempt to steal work from the top of their SpDeques.To keep the private part of SpDeques entirely private, steal attempts are only allowed to access the public part.Thus, when a thief attempts to steal work from a victim's SpDeque whose public part is empty (illustrated in Figure 1a), the steal attempt simply fails and the thief does not obtain work.In such case, the thief then updates a victim's flag (referred to as the targeted flag) to (asynchronously) notify the victim that the public part of its SpDeque is empty (more on this ahead).When the owner of the SpDeque realizes it was notified (by checking the value of its targeted flag), it tries to transfer a node from the private part of its SpDeque to the public part.If the private part is not empty then a node is transferred, in which case we say that the transferred node became stealable (illustrated in Figure 1b).

The Lock-Free Split Deque
We now present the specification of an SpDeque object, along with its associated relaxed semantics.Being the behavior of SpDeques similar to the behavior of concurrent deques, the SpDeque's relaxed semantics are comparable to the relaxed deque semantics presented in [8].An SpDeque object meeting the relaxed semantics supports five methods: push -Pushes a node into the bottom of the SpDeque's private part.
pop -Removes and returns a node from the bottom of the SpDeque's private part, if that part is not empty.Otherwise, returns the special value race.
updateBottom -Transfers the topmost node from the private part of the SpDeque into the bottom of the public part, and does not return a value.The invocation has no effect if the private part of the SpDeque is empty.
popBottom -Removes and returns the bottom-most node from the public part of the SpDeque.If the SpDeque is empty, the invocation has no effect and empty is returned.
popTop -Attempts to remove and return the topmost node from the public part of the SpDeque.If the public part is empty, the invocation has no effect and the value empty is returned.If the invocation aborts, it has no effect and the value abort is returned.
An SpDeque implementation is constant-time iff any invocation to each of these methods takes at most a constant number of steps to return.Say that a set of invocations to an SpDeque's methods meets the relaxed semantics iff there is a set of linearization times for the corresponding non-aborting invocations such that: 1. Every non-aborting invocation's linearization time lies within the beginning and completion times of the respective invocation; 2. No linearization times associated with distinct non-aborting invocations coincide; 3. The return values for the non-aborting invocations are consistent with a serial execution of the methods in the order given by the linearization times of the corresponding non-aborting invocations; and 4. For each aborted popTop invocation x to an SpDeque d, there exists another invocation removing the topmost item from d whose linearization time falls between the beginning and completion times of invocation x.

The Low Cost Work Stealing Algorithm
Algorithm 1 depicts the specification of the Low Cost Work Stealing algorithm.Each processor owns an SpDeque that uses to store its attached nodes and, additionally, owns a targeted flag that stores a Boolean value.This flag is used to implement an asynchronous notification mechanism that allows thieves to request their victims to expose work, allowing it to be stolen.Even though, in practice, the notification mechanism our algorithm can be implemented using signals, to perform a correct analysis of the algorithm's synchronization overheads all the possible sources of such overheads must be explicit, for which reason we chose to embed a simple notification mechanism into the algorithm's specification.Although the targeted flag of each processor can be simultaneously accessed by multiple processors, to ensure the algorithm's correctness it suffices to guarantee that no read nor write operation to a processor's targeted flag is cached.Before a computation's execution begins, every processor sets its assigned node to none and its targeted flag to false.To start the execution, one of the processors gets the root node assigned.
As we will see, the behavior of Low Cost Work Stealing is similar to the original Work Stealing algorithm.Consider some processor p working on a computation scheduled by Low Cost Work Stealing, and some iteration of the scheduling loop that p executes (corresponding to lines 2 to 23 of Algorithm 1).First, p reads the value of its targeted flag to check if it has been notified by some thief.If p's targeted flag is set to true (i.e. if p was notified), the processor tries to make a node stealable, by invoking updateBottom to its SpDeque.After that, and regardless of that invocation's outcome, p resets its targeted flag back to false.The subsequent behavior of p depends on whether it has an assigned node.
• If p has an assigned node, p executes the node.From this execution, either zero, one or two nodes can be enabled.

Zero nodes enabled
The processor tries to fetch the bottommost node stored in its SpDeque.To that end, p first tries to fetch a node from the bottom of its SpDequeś private part (line 15

A Split Deque Implementation
Algorithm 2 depicts a possible implementation of the lock-free SpDeque, based on the deque's implementation given in [8].As illustrated in Figure 1, each SpDeque object has four instance variables: entries -an array of ready nodes.
privateBottom -the index below the bottommost node of the SpDeque.
officialBottom -the index below the bottommost node of the SpDeque's public part.
age -composed of two fields: top, which corresponds to the index right below the topmost node of the SpDeque's public part; and tag, which is only used to ensure correction (avoiding the famous ABA problem).
Upon creation, the instance variables privateBottom, officialBottom, and both fields of the instance variable age are set to 0.
We say that a set of invocations is good if and only if the methods push, pop, updateBottom and popBottom are never invoked concurrently.For Low Cost Work Stealing, as only the owner of each SpDeque can invoke these methods, it is easy to deduce that all sets of invocations issued by the algorithm are good.Furthermore, we claim that the implementation depicted in Algorithm 2 is constant-time and meets the relaxed semantics (defined in Section 3.1) on any good set of invocations.However, even though all methods are composed by a small number of instructions and none includes a loop, proving this claim is not a straightforward task because all possible execution interleaves have to be considered.Moreover, as the main focus of this study is not related with programs' verification, the proof of this claim falls out of the scope of this paper.Yet, we remark that the proposed implementation is a simple extension of the deque implementation presented in [8], which has been proven in [14] to be a correct implementation, meeting the relaxed deque semantics on any set of invocations made by the Work Stealing algorithm.For this reason, throughout this paper we assume that for any set of invocations issued by the Low Cost Work Stealing algorithm, the relaxed semantics is always satisfied.Proof.Any invocation to the pop method only reads from two publicly accessible fields of the SpDeque's state, namely officialBottom (line 8) and entries (line 12).However, due to a data dependency, no re-ordering between these read operations may occur, and so, no MFence instructions are required.
The dag of a computation is dynamically unfolded during its execution.If the execution of a node u enables another node u , then (u, u ) is an enabling edge and refer to node u as the designated parent of u .Refer to the tree formed by the enabling edges of a particular execution of a dag by enabling tree, and denote the depth of a node u within this tree by d (u).Define the weight of u as w (u) = T ∞ − d (u).Similarly to [8], our analysis is made in an a posteriori fashion, allowing us to refer to the enabling tree generated by a computation's execution.
The following corollary is a direct consequence of the standard properties of deques (a full proof can be found in the appendix (Lemma A.1)).
Corollary 3.3.Let v 1 , . . ., v k denote the nodes stored in some processor p's SpDeque, ordered from the bottom of the SpDeque to the top, at some moment during the execution of Low Cost Work Stealing.Moreover, let v 0 denote p's assigned node (if any).Then, we have w

Analysis
In this section we obtain bounds on the expected execution time of computations using Low Cost Work Stealing, and on the expected synchronization overheads incurred by the scheduler.The analysis we make follows the same overall idea as the one given in [8].Due to space restrictions, it is not possible to include all the details of the proofs in the paper.These can be found in a companion technical report available from1 .Before advancing any further, we introduce a few more essential definitions.
Define a scheduling iteration as a sequence of instructions executed by a processor corresponding to a particular iteration of the scheduling loop (lines 2 to 23 of Algorithm 1).Thus, the full sequence of instructions executed by each processor during a computation's execution can be partitioned into scheduling iterations.As in [8], we introduce the concept of milestone: an instruction within the sequence executed by a processor is a milestone iff it corresponds to a node's execution (line 8) or to the return of a call to WorkMigration (line 31).Taking into account the definition of a scheduling iteration it is clear that any scheduling iteration of the algorithm includes a milestone.Refer to iterations whose milestone corresponds to a node's execution as busy iterations, and refer to the remainder as idle iterations.As one might note, if a processor has an assigned node at the beginning of an iteration's execution, the iteration is a busy one, and, otherwise, the iteration is an idle one.By observing the scheduling loop (lines 2 to 23 of Algorithm 1), and taking into account that the SpDeque's implementation is constant time, it is clear that any scheduling iteration is composed of a constant number of instructions.It then follows that any processor executes at most a constant number of instructions between two consecutive milestones.Throughout the analysis, let C denote a constant that is large enough to guarantee that any sequence of instructions executed by a processor with length at least C includes a milestone.
We can now bound the execution time of a computation depending on the number of idle iterations that take place during that computation's execution.The proof of the following result can be found in the appendix (Lemma A.3), and is a trivial variant of [8, Lemma 5], but considering the Low Cost Work Stealing algorithm.

Now, we prove that the synchronization overheads incurred by Low Cost
Work Stealing only depend on the number of idle iterations that take place during a computation's execution (proofs in appendix (Lemmas A.4 and A.5, respectively)).Lemma 4.2.Consider a processor p executing a busy iteration such that p's targeted flag is set to false when p checks it at the beginning of the iteration.If the execution of p's assigned node enables one or more nodes, or, if the private part of p's SpDeque is not empty, then, no MFence instruction is required during the execution of the iteration.

Bounds on the expected number of idle iterations
The rest of the analysis focuses on bounding the number of idle iterations that take place during a computation's execution, and follows the same general arguments as the analysis presented in [8].
We say that a node u is stealable if u is stored in the public part of some processor's SpDeque.Furthermore, we denote the set of ready nodes at some step i by R i .Consider any node u ∈ R i .The potential associated with u at step i is denoted by φ i (u) and is defined as otherwise The total potential at step i, denoted by Φ i , corresponds to the sum of potentials of all the nodes that are ready at that step: The following lemma is a formalization of the arguments already given in [8], but considering the potential function we present (proof in appendix (Lemma A.6)).
Lemma 4.4.Consider some node u, ready at step i during the execution of a computation.
1.If u gets assigned to a processor at that step, the potential drops by at least 3 4 φ i (u).
2. If u becomes stealable at that step, the potential drops by at least 3 4 φ i (u).
3. If u was already assigned to a processor and gets executed at that step i, the potential drops by at least 47 64 φ i (u).
For the remainder of the analysis, we make use of a few more definitions, first introduced in [8].We denote the set of ready nodes attached to some processor p (i.e. the ready nodes in p's SpDeque together with the node it has assigned, if any) at the beginning of some step i by R i (p).Furthermore, we define the total potential associated with p at step i as the sum of the potentials of each of the nodes that is attached to p at the beginning of that step For each step i, we partition the processors into two sets, D i and A i , where the first is the set of all processors whose SpDeque is not empty at the beginning of step i while the second is the set of all other processors (i.e. the set of all processors whose SpDeque is empty at the beginning of that step).Thus, the potential of any step i, Φ i , is composed by the potential associated with each of these two partitions The following lemma is a direct consequence of Corollary 3.3 and of the potential function's properties (proof in appendix (Lemma A.7)).Lemma 4.5.Consider any step i and any processor p ∈ D i .The top-most node u in p's SpDeque contributes at least 4  5 of the potential associated with p.That is, we have With this, we now show that if a processor p is targeted by a steal attempt, then p's potential decreases by a constant factor (proof in appendix (Lemma A.8)).
Lemma 4.6.Suppose a thief processor p chooses a processor q ∈ D i as its victim at some step j, such that j ≥ i (i.e. a steal attempt of p targeting q occurs at step j).Then, at step j + 2C, the potential decreased by at least The next lemma is a trivial generalization of the original result presented in [8, Balls and Weighted Bins] (proof in appendix (Lemma A.9)).Lemma 4.7 (Balls and Weighted Bins).Suppose we are given at least B balls, and exactly B bins.Each of the balls is tossed independently and uniformly at random into one of the B bins, where for i = 1, . . ., B, bin i has a weight W i .The total weight is W = B i=1 W i .For each bin i, we define the random variable X i as W i if some ball lands in bin i 0 otherwise and define the random variable X as X = B i=1 X i .Then, for any β in the range 0 < β < 1, we have The following result states that for each P idle iterations that take place, with constant probability, the total potential drops by a constant factor.The result is a consequence of Lemmas 4.6 and 4.7 (proof in appendix (Lemma A.10)).
Lemma 4.8.Consider any step i and any later step j such that at least P idle iterations occur from i (inclusive) to j (exclusive).Then, we have Following Lemma 4.8, we are able to bound the expected number of idle iterations that take place during a computation's execution using the Low Cost Work Stealing algorithm (proof in appendix (Lemma A.11)).Lemma 4.9.Consider any computation with work T 1 and critical-path length T ∞ being executed by Low Cost Work Stealing using P processors.The expected number of idle iterations is at most O (P T ∞ ).Moreover, with probability at least 1−ε the number of idle iterations is at most O T ∞ + ln

Comparison with Work Stealing
To get a better understanding of the importance of avoiding synchronization for local deque accesses, we now compare the synchronization costs of our algorithm against conventional Work Stealing algorithms that use concurrent deques.To that end, we developed a simulator that, given a computation's dag, executes it, monitoring not only the number of CAS and MFence instructions executed but also the number times that thieves requested other processors to expose work.In this section we use the term notification to refer to when a thief sets another processor's targeted flag to true, requesting it to expose work.For this comparison, we consider two distinct classes of dags.Dags of the first class essentially correspond to trees of instructions where every non-leaf instruction forks two other instructions, and whose depth is given by an argument that is passed to the simulator.Dags of the second class are intended to simulate unbalanced computations.To that end, we use the argument passed to the simulator as the total depth of the dag, and make the depth between each two consecutive fork instructions follow an exponential distribution with parameter λ = 0.05.The first class of dags corresponds to computations with balanced parallelism (e.g.Fibonacci, Parallel-For, etc) whist the second corresponds to the ones with unbalanced parallelism (e.g.Graph Searches).From Figure 2a, it is clear that while for Work Stealing with concurrent deques the number of synchronization operations grows linearly with the total amount of work (and thus exponentially increases with the span of the dag), for Low Cost Work Stealing the number of synchronization operations and notifications scales linearly with the span of the computation.Thus, even if the costs of handling notifications (i.e. of exposing work) were a thousand times greater than the cost of executing CAS or MFence instructions, for dags with fork-span of at least ≈ 20, our algorithm would incur in less synchronization overheads.In practice, computations with a fork-span ≥ 20 are very common, especially among fine-grained parallelism.Unfortunately, due to the limitations that come with using a simulator, we have not been able to benchmark dags with a fork-span greater than 25.Yet, we remark that the trend is obvious and confirms that the use of SpDeques allows to avoid most of the synchronization that is present in conventional Work Stealing algorithms.Figure 2b reinforces our insight, showing that even for computations exhibiting irregular parallelism, Low Cost Work Stealing is able to avoid most of the synchronization present in Work Stealing algorithms that use concurrent deques.Finally, Figure 2c shows that, while the synchronization costs for Work Stealing are always extremely high, even for single processor executions, for Low Cost Work Stealing these costs only grow linearly with the number of processors used and, for a single processor execution, synchronization is negligible.
From a more theoretical perspective, note that, by taking into account our assumptions regarding computations' structure, we can create computations for which T 1 = O 2 T∞ (which correspond to dags of the first class).Since for such computations the number of deque accesses is directly proportional to the total amount of work (T 1 ), our result shows that the use of SpDeques allows to reduce by almost an exponential factor the synchronization present in conventional Work Stealing algorithms.

Related work
Many efforts have been carried out towards reducing and even eliminating the expensive synchronization overheads of state-of-the-art Work Stealing schedulers.
Michael et al propose an idempotent version of Work Stealing that eliminates the overheads of synchronization by relaxing the semantics of work queues [27].Concretely, rather than using the conventional exactly-once semantics, the authors present several concurrent data structures only satisfying at-least-once semantics.By using these structures, processors no longer have to incur in expensive synchronization overheads when operating locally, which, as the authors show, can be extremely beneficial.Unfortunately this strategy suffers from two limitations: on one hand, for non-idempotent computations the synchronization overheads are moved to the computation itself, as it is necessary to ensure correctness; on the other, for idempotent computations these semantics can lead to situations where some computationally heavy tasks are executed more than once, limiting the scheduler's performance.
Hiraishi et al. suggested that processors should behave as in a sequential execution, thus eliminating all overheads inherent from parallelism [22].Under their scheme, deques are kept entirely private and processors only permit parallelism when an idle processor requests work.Upon such request, the busy processor backtracks to the last point where it could have spawned a task, spawns the task, offers it to the requesting processor, and, finally, proceeds with the execution.Since work requests are usually rare, the gains of eliminating synchronization for local operation greatly surpasses the extra overheads caused by using entirely private deques, which for this algorithm come from backtracking.This remark is key and motivated the efforts for reducing synchronization overheads to focus on the elimination of synchronization for local operation.
Acar et al. presented and studied two Work Stealing algorithmssender-and receiver-initiated -that avoid synchronization by making deques entirely private to each processor [2].In addition to promising empirical results, their theoretical analysis showed that the expected execution time for both algorithms can be competitive with Work Stealing algorithms that use concurrent deques.For the sender-initiated algorithm, busy processors now have to periodically search for idle ones, and thus synchronization is replaced by communication.Regarding the receiver-initiated algorithm the trade-off is more subtle: since deques are kept entirely private to each processor, when an idle processor requests work from a busy one, it has to perform a handshake with the busy processor.For multiprogrammed environments these handshakes can be harmful if, for example, the busy processor -in this case, the busy system thread -is not currently scheduled by the Operating System.Additionally, in this approach, most of the load balancing burdens are transferred from the thief to the busy processors, who, to handle a work request, have to halt their work, pop a thread from the top of their deque and send it to the requesting processor.Our approach is different, as we follow Work Stealing's strategy of keeping the burden of load balancing entirely on the thief.In this process, thieves do not wait for their victims; when they target a processor with no publicly visible work, they simply notify the latter of the steal attempt and proceed to the next victim.
The first use of split queues for avoiding unnecessary synchronization was, to the best of our knowledge, by Endo et al. in [19].In this study, the authors presented an implementation of a scalable garbage collector system that, by using clever load balancing techniques and split queues to avoid unnecessary synchronization overheads, achieves high performances even for large scale machines.Later, and as already mentioned, Dinan et al. studied Work Stealing under a distributed environment and proposed the use of SpDeques to avoid synchronization for local deque accesses [17,18].More recently, Dijk et al. studied the effectiveness of SpDeques on shared memory environments [32,33].We remark that our algorithm has some similarities with Dijk et al.'s, since thieves also use flags to inform busy processors that they should add work to the public part of their deques, allowing it to be stolen.However, in their algorithm, and in contrast with our approach, busy processors only check for requests each time they access their deque.As a consequence, the scheduler is not appropriate for generic computations because the frequency at which busy processors may permit load balancing depends on the structure of the computation.In particular, for computations whose workload is mostly composed by large sequential threads, processors will rarely check for the necessity of load balancing, giving a very small room for parallelism.Finally, when a busy processor realizes it was targeted by a steal attempt, it makes at least half of its work to become public.Even though this strategy is useful to mitigate the limitation we noted above, it severely increases the synchronization costs incurred by the algorithm.For instance, by making half of their work public, busy processors have to incur extra synchronization overheads for each time they have to fetch work from the non-private part of their deques.
Morrison et al. studied alternative designs to the synchronization pro-tocols used by Work Stealing schedulers, considering the architectures of modern TSO processors [28].The authors found that, by taking into account the actual implementation of microprocessors, MFence instructions can be fully eliminated while maintaining correctness.In their algorithm, thieves can only steal work from a victim if such work is stored at a safe distance, far enough from the bottom of the victim's deque to avoid any data race.This safe distance is computed a priori, taking into account the size of the microprocessor's internal store buffer to determine the minimum safe distance to avoid any possible conflicts.With this strategy, not only thieves can asynchronously take work from their victims, but processors can also access their deques locally without requiring any synchronization.Unfortunately, their scheme suffers from a limitation similar to Dijk et al.'s, as the bottommost threads within a processor's deque cannot to be stolen, meaning their strategy is not appropriate for generic computations.Nevertheless, the authors' showed that this technique (similar to using SpDeques) substantially outperforms Work Stealing algorithms that use concurrent deques.Lifflander et al. studied the execution of iterative over-decomposed applications [26] and proposed, among others, a message-based retentive Work Stealing algorithm adapted for the execution of iterative workloads on large scale distributed settings.To avoid synchronization overheads and improve the overall performance of the scheduler, their Work Stealing algorithm uses SpDeques.Their evaluation showed that Work Stealing (with SpDeques) is a practical algorithm even for systems with as much as 163.940 processors, for which their Work Stealing algorithm achieved a remarkable result of more than 91% of efficiency.A distinction between the Work Stealing algorithm proposed by Lifflander et al. and Low Cost Work Stealing is that our algorithm keeps processors' deques entirely private by default.As we will discuss in Section 7, this fact is key for guaranteeing that a 1-Processor execution of our algorithm requires no synchronization.
Tzannes et al. proposed a scheduling algorithm where each processor keeps all of its work entirely private, except for the topmost node that is kept stored in a shared cell [31].However, since the algorithm always ensures that the topmost node is shared, it does not behave appropriately for various types of computations in which processors access the topmost nodes of their deques often.As mentioned in [2], a similar limitation has been identified for the Chase-Lev Deque [16].

Conclusion
In this paper we studied a Work Stealing algorithm that uses SpDeques to reduce synchronization overheads.Whereas traditional Work Stealing algorithms require synchronization for every time processors access deques, in our proposal, the use of synchronization is avoided as much as possible.
By default, busy processors operate locally on their deques without any synchronization, resembling a sequential execution.However, in order to avoid transferring the load balancing burdens to busy processors, thieves may request their victims to make some of their work public, permitting direct steals.
We showed that the expected execution time of our algorithm is asymptotically optimal, and that the expected total synchronization of the algorithm is O (P T ∞ (C CAS + C M F ence )).To justify the tightness of our bounds, we recall that, for Low Cost Work Stealing, the expected number of (successful and unsuccessful) steal attempts is O (P T ∞ ).Thus, by noting that the public part of an SpDeque is essentially a concurrent deque, and, by taking into account the impossibility of eliminating all synchronization from the implementation of concurrent deques while maintaining their correction (see [9]), we conclude that the synchronization bounds we have obtained for Low Cost Work Stealing are tight.
In addition to an asymptotically optimal expected execution time, we claim that the expected total communication of Low Cost Work Stealing is the same as for the Work Stealing algorithm given in [11], for which, in turn, the expected total communication is known to be existentially optimal [11,34].Despite being trivial to prove this claim after having obtained bounds on the expected number of steal attempts, such proof falls out of the scope of this paper as it requires the introduction of various extra definitions (such as the notion of threads) and additional assumptions on computations' structure.Nevertheless, we note that by using the exact same reasoning made in the proof of [11,Theorem 14] we could obtain the proof of our claim, meaning that the expected total communication of Low Cost Work Stealing is also asymptotically optimal.
To conclude, and as already discussed in Section 5, for several types of computations, the synchronization overheads of conventional Work Stealing algorithms grow linearly with both the number of steal attempts and total amount of work.Taking into account our assumptions regarding computations' structure, it is possible to create computations for which T 1 = O 2 T∞ .As a matter of fact, for numerous classes of parallel computations, the total amount of work indeed increases exponentially with the span of the computation.With this, it is straightforward to conclude that our algorithm reduces, by orders of magnitude, the synchronization overheads when executing such computations, when compared with traditional Work Stealing algorithms that use fully concurrent deques.

A Appendix: Proofs
Due to space constraints, most of the proofs of our claims are placed in this appendix.Nevertheless, if this paper is accepted, we will publish a full version of this paper, with proofs, in a freely accessible on-line repository.
The following lemma is crucial for the performance analysis of Low Cost Work Stealing.An analogous result has already been proved for concurrent deques (see [8,Lemma 3]).For the sake of completion we present its proof, which is a simple transcription of original proof of [8, Lemma 3], adapted for SpDeques.
Lemma A.1 (Structural Lemma for SpDeques).Let v 1 , . . ., v k denote the nodes stored in some processor p's SpDeque, ordered from the bottom of the SpDeque to the top, at some point in the linearized execution of Low Cost Work Stealing.Moreover, let v 0 denote p's assigned node (if any), and for i = 0, . . ., k let u i denote the designated parent of v i in the enabling tree.Then, for i = 1, . . ., k, u i is an ancestor of u i−1 in the enabling tree, and despite v 0 and v 1 may have the same designated parent (i.e.u 0 = u 1 ), for i = 2, 3, . . ., k, u i−1 = u i (i.e. the ancestor relationship is proper).
Proof.Fix a particular SpDeque.The SpDeque state and assigned node only change when the assigned node is executed or a thief performs a successful steal.We prove the claim by induction on the number of assigned-node executions and steals since the SpDeque was last empty.In the base case, if the SpDeque is empty, then the claim holds vacuously.We now assume that the claim holds before a given assigned-node execution or successful steal, and we will show that it holds after.Specifically, before the assignednode execution or successful steal, let v 0 denote the assigned node; let k denote the number of nodes in the SpDeque; let v 1 , . . ., v k denote the nodes in the SpDeque ordered from the bottom to top; and for i = 0, . . ., k, let u i denote the designated parent of v i .We assume that either k = 0, or for i = 1, . . ., k, node u i is an ancestor of u i−1 in the enabling tree, with the ancestor relationship being proper, except possibly for the case i = 1.After the assigned-node execution or successful steal, let v 0 denote the assigned node; let k denote the number of nodes in the SpDeque; let v 1 , . . ., v k denote the nodes in the SpDeque ordered from bottom to top; and for i = 1, . . ., k , let u i denote the designated parent of v i .We now show that either k = 0, or for i = 1, . . ., k , node u i is an ancestor of u i−1 in the enabling tree, with the ancestor relationship being proper, except possibly for the case i = 1.
Consider the execution of the assigned node v 0 by the owner.
If the execution of v 0 enables 0 children, then the owner pops the bottommost node off its SpDeque and makes that node its new assigned node.If k = 0, then the SpDeque is empty; the owner does not get a new assigned node; and k = 0.If k > 0, then the bottommost node v 1 is popped and becomes the new assigned node, and We now rename the nodes as follows.For i = 0, . . ., k , we set v i = v i+1 and u i = u i+1 .We now observe that for i = 1, . . ., k , node u i is a proper ancestor of u i−1 in the enabling tree.
If the execution of v 0 enables 1 child x, then x becomes the new assigned node; the designated parent of x is v 0 ; and k = k.If k = 0, then k = 0. Otherwise, we can rename the nodes as follows.We set v 0 = x; we set u 0 = v 0 ; and for i = 1, . . ., k , we set v i = v i and u i = u i .We now observe that for i = 1, . . ., k , node u i is a proper ancestor of u i−1 in the enabling tree.That u 1 is a proper ancestor of u 0 in the enabling tree follows from the fact that (u 0 , v 0 ) is an enabling edge.
In the most interesting case, the execution of the assigned node v 0 enables 2 children x and y, with x being pushed onto the bottom of the SpDeque and y becoming the new assigned node.In this case, (v 0 , x) and (v 0 , y) are both enabling edges, and k = k + 1.We now rename the nodes as follows.We set v 0 = y; we set u 0 = v 0 ; we set v 1 = x; we set u 1 = v 0 ; and for i = 2, . . ., k , we set v i = v i−1 and u i = u i−1 .We now observe that u 1 = u 0 , and for i = 2, . . ., k , node u i is a proper ancestor of u i−1 in the enabling tree.That u 2 is a proper ancestor of u 1 in the enabling tree follows from the fact that (u 0 , v 0 ) is an enabling edge.
Finally, we consider a successful steal by a thief.In this case, the thief pops the topmost node v k off the SpDeque, so k = k − 1.If k = 1, then k = 0. Otherwise, we can rename the nodes as follows.For i = 0, . . ., k , we set v i = v i and u i = u i .We now observe that for i = 1, . . ., k , node u i is an ancestor of u i−1 in the enabling tree, with the ancestor relationship being proper, except possibly for the case i = 1.
Corollary A.2.If v 0 , v 1 , . . ., v k are as defined in the statement of Lemma A.1, then we have w We are now able to bound the execution time of a computation depending on the number of idle iterations that take place during that computation's execution.The following result is a trivial variant of [8, Lemma 5] but considering the Low Cost Work Stealing algorithm, and is only added for the sake of completion.Proof.Consider two buckets to which we add tokens during the computation's execution: the busy bucket and the idle bucket.At the end of each iteration, every processor places a token into one of these buckets.If a processor executed a node during the iteration, it places a token into the busy bucket, and otherwise, it places a token into the idle bucket.Since we have P processors, for each C consecutive steps, at least P tokens are placed into the buckets.
Because, by definition, the computation has T 1 nodes, there will be exactly T 1 tokens in the busy bucket when the computation's execution ends.Moreover, as I denotes the number of idle iterations, it also corresponds to the number of tokens in the idle bucket when the computation's execution ends.Thus, exactly T 1 + I tokens are collected during the computation's execution.Taking into account that for each C consecutive steps at least P tokens are placed into the buckets, we conclude the number of steps required to collect all the tokens is at most C. T 1 P + I P .After collecting all the T 1 tokens, the computation's execution terminates, implying the execution time is at most O T 1 P + I P .
Lemma A.4.Consider a processor p executing a busy iteration such that p's targeted flag is set to false when p checks it at the beginning of the iteration.
If the execution of p's assigned node enables one or more nodes, or, if the private part of p's SpDeque is not empty, then, no MFence instruction is required during the execution of the iteration.
Proof.Consider the Low Cost Work Stealing algorithm, depicted in Algorithm 1.The first action p takes for the execution of that iteration is checking the value of its targeted flag (line 3).Since, by the statement of this lemma p's targeted flag is set to false at the moment when p checks the flag's value, p does not enter the then branch of the if statement.Moreover, as a consequence of the conditional statement of line 3, there is a control dependency that does not allow the instructions succeeding the conditional expression to be reordered with the evaluation of the condition, implying no MFence instruction is required until this point.After that, p checks if it has an assigned node.Again, since the next action p takes depends on its currently assigned node, there is a control dependency from the instruction where p checks if it has a currently assigned node to both branches of the if statement.Thus, no instruction reordering between the evaluation of the condition and any of the instructions succeeding that evaluation can be made, implying no MFence instruction is required until here.
Because we assumed p was executing a busy iteration, by the definition of a busy iteration, p must have an assigned node.Hence, p executes its assigned node.Since the next action p takes depends on the outcome of that node's execution, there is a control dependency between the execution of p's assigned node and the execution of the sequence of instructions corresponding to each of the possible outcomes.Hence, no instruction reordering can be made, implying no MFence instruction is necessary until this point.
From that node's execution, three outcomes are possible: 0 nodes enabled In this case, p invokes the pop method to its own SpDeque.
By Lemma 3.2, the invocation does not require the execution of a MFence instruction.Furthermore, the next instruction (line 16) has a data dependency on the value of p's assigned node, for which reason it cannot be reordered with the invocation of the pop method and so no MFence instruction is required.
Since we have assumed that the private part of p's SpDeque was not empty, it is trivial to conclude that the pop invocation returns a node, which immediately becomes p's new assigned node.Thus, after having a new node assigned p takes no further action during the iteration, meaning no MFence instruction was required for the execution of the iteration in this situation.
1 node enabled In this case the enabled node becomes p's new assigned node.Next, p checks the number of nodes that were enabled.The assignment of one of the enabled nodes and the instruction where p checks the number of nodes enabled can be reordered.Fortunately, because enabled is a local variable that is solely accessed by p, there is no harm for a parallel execution if the instructions are reordered and so no MFence instruction is required for this case as well.Because p enabled a single node it takes no further action during the iteration, implying the lemma holds in this situation as well.
2 nodes enabled Finally, for this case one of the enabled nodes becomes p's new assigned node.Using the same reasoning as for the case where a single node was enabled, we conclude that no MFence instruction is required at least until the evaluation of the conditional statement.Because p enabled two nodes, p enters the conditional expression and pushes the node it did not assign into the bottom of its SpDeque, by invoking the push method.Since there is a control dependency between the execution of this instruction and the evaluation of the condition, no instruction reordering is allowed.Thus, no MFence instruction is required between these two instructions.
Finally, Lemma 3.1 states that an invocation to the push method does not require a MFence instruction to be executed.Because after the invocation p takes no further action during the iteration, we deduce the lemma holds, concluding its proof.
The following lemma is a consequence of Lemma 4.2 (corresponding to Lemma A.4 of the appendix) and states that the number of CAS and MFence instructions executed during a computation's execution using Low Cost Work Stealing only depends on the number of idle iterations and processors.
Lemma A.5.Consider any computation being executed by the Low Cost Work Stealing algorithm, using P processors.The number of CAS and MFence instructions executed by processors during the computation's execution is at most O (I + P ), where I denotes the total number of idle iterations executed by processors.
Proof.By observing Algorithms 1 and 2, it is easy to see that only invocations to popBottom or popTop methods can lead to the execution of CAS instructions.Furthermore, both these methods are invoked at most once per scheduling iteration, and, for both, at most one CAS instruction is executed per invocation.Since processors only invoke the popTop method when executing idle iterations, the number of CAS instructions caused by invocations to popTop is O (I).On the other hand, processors only invoke the popBottom method during busy iterations where the private part of their SpDeque is empty and the execution of their currently assigned node does not enable any new nodes.Let p denote some processor executing one such iteration.From p's invocation to the popBottom method two outcomes are possible: A node is returned In this case the public part of p's SpDeque was not empty implying p had previously transferred a node from the private part of its SpDeque to the public part.By observing Algorithm 1 it is easy to deduce that p only makes these node transfers if some thief had previously set p's targeted flag to true.Moreover, because after transferring the node p immediately sets its targeted flag back to false, the number of times p makes such node transfers is at most the number of times it is targeted by a steal attempt.Taking into account that processors only make steal attempts during the execution of idle iterations, and make exactly one steal attempt for each such iteration, exactly I steal attempts take place during a computation's execution.As such, the number of CAS instructions executed in situations like this one is at most O (I).
Empty is returned In this case p will not have an assigned node at the end of the scheduling iteration's execution.Thus, after p finishes executing the iteration two scenarios may occur: p executes an idle iteration For this case we can create a mapping from idle iterations to each busy iteration that precedes an idle iteration, implying there can be at most O (I) such iterations.With this, it is trivial to conclude that the number of CAS instructions executed by Low Cost Work Stealing for situations equivalent to this one is at most O (I).
The computation's execution terminates Because there are exactly P processors, at most P scheduling iterations can precede the end of a computation's execution.Consequently, the number of CAS instructions executed for scenarios equivalent to this one is at most O (P ).
Summing up all the possible scenarios, the number of CAS instructions executed by Low Cost Work Stealing is at most O (I + P ).
We now turn to the number of MFence instructions executed during a computation's execution.To that end, we first bound the number of scheduling iterations that can contain MFence instructions.Consider any scheduling iteration s during a computation's execution, and let p denote the processor that executed the iteration.Iteration s was either an idle or a busy iteration.
s is an idle iteration By definition, at most I iterations are idle, implying there are O (I) such iterations that could contain MFence instructions.
s is a busy iteration When p checks its targeted flag, one of the two following situations arises: targeted is true By observing Algorithm 1 we conclude that such a situation can only occur if another processor q has set p's targeted to true, which can only occur if q was executing an idle iteration.
After executing the conditional statement, p resets its targeted flag back to false.Thus, the total number of busy iterations where a processor has its flag set to targeted is at most I, because each such iteration can be mapped by an idle iteration.
Consequently, the number of iterations similar to this one is at most O (I).
targeted is false As p is executing a busy iteration, it will execute the node it has assigned.From that node's execution, either 0, 1 or 2 other nodes can be enabled.
More than 0 nodes are enabled Lemma 4.2 (corresponding to Lemma A.4 of the appendix) implies that no MFence instruction is executed in this case.0 nodes are enabled In this case, p cannot immediately assign a new node, because it did not enable any.By Algorithm 1, p will then invoke the pop method to its own SpDeque.With this, one of two possible situations arises: SpDeque's private part is not empty As a consequence of Lemma 4.2 (corresponding to Lemma A.4 of the appendix), no MFence instruction is executed in this case.SpDeque's private part is empty In this case, by observing Algorithm 2 we conclude that the invocation returns the special value race, implying p will make an invocation to popBottom still during that same iteration.From that invocation, two outcomes are possible: A node is returned In this situation, p assigns the node.By observing Algorithm 1 it is trivial to conclude that this scenario only arises if some processor previously set p's targeted flag to true.As a consequence, p transfered a node from the private part of its SpDeque to the public part.Again, using the same reasoning as for the case where p's targeted flag is set to true, we conclude the number of such iterations is at most O (I).
empty is returned After p finishes executing the current scheduling iteration s, two scenarios may occur: p executes an idle iteration It is trivial to deduce that we can create a mapping from idle iterations to each iteration satisfying the same conditions as s.
Thus, there can be at most O (I) such iterations.The computation's execution terminates Since there are exactly P processors, at most P scheduling iterations can precede the end of a computation's execution.Consequently, there are at most P scheduling iteration similar to s.Now, we sum up all the scheduling iterations that may contain MFence instructions.Accounting with all possible scenarios it follows that at most O (I + P ) scheduling iterations may contain MFence instructions.Since any scheduling iteration is composed by at most C instructions, at most C MFence instructions can be executed per iteration, implying the number of MFence instructions executed during a computation's execution is at most O (I + P ).
The following lemma is a formalization of the arguments already given in [8], but considering the potential function we present.
Lemma A.6.Consider some node u, ready at step i during the execution of a computation.
1.If u gets assigned to a processor at that step, the potential drops by at least 3 4 φ i (u).
2. If u becomes stealable at that step, the potential drops by at least 3 4 φ i (u).
3. If u was already assigned to a processor and gets executed at that step i, the potential drops by at least 47 64 φ i (u).
Proof.Regarding the first claim, if u was stealable the potential decreases from 4 3w(u)−1 to 4 3w(u)−2 .Otherwise, the potential decreases from 4 3w(u) to enabled node.Since both x and y have u as their designated parent in the enabling tree, we have As such, the potential decreases by 47 64 φ i (u), concluding the proof of the lemma.
The following lemma is a direct consequence of Corollary 3.3 (corresponding to Corollary A.2 of the appendix) and of the potential function's properties.The result is a variant of [8, Top-Heavy Deques], considering SpDeques instead of the conventional fully concurrent deques, and our potential function, instead of the original.
Lemma A.7. Consider any step i and any processor p ∈ D i .The top-most node u in p's SpDeque contributes at least 4  5 of the potential associated with p.That is, we have Proof.This lemma follows from Corollary 3.3 (corresponding to Corollary A.2 of the appendix).We prove it by induction on the number of nodes within p's SpDeque.
Base case As the base case, consider that p's SpDeque contains a single node u.The processor itself can either have or not an assigned node.
For the second scenario, we have φ i (u) = Φ i (p).Regarding the first case, let x denote p's assigned node.Corollary 3.3 implies that w (u) ≥ w (x).It follows Thus, if p's SpDeque contains a single node we have Φ i (q) ≤ 5 4 φ i (u).
Induction step Consider that p's SpDeque now contains n nodes, where n ≥ 2, and let u, x denote the topmost and second topmost nodes, respectively, within the SpDeque.For the purpose of induction, let us assume the lemma holds for all the first n − 1 nodes (i.e.without accounting with u): concluding the proof of the lemma.
The following result is a consequence of Lemma 4.5 (corresponding to Lemma A.7 of the appendix).
Lemma A.8. Suppose a thief processor p chooses a processor q ∈ D i as its victim at some step j, such that j ≥ i (i.e. a steal attempt of p targeting q occurs at step j).Then, at step j + 2C, the potential decreased by at least 3 5 Φ i (q) due to either the assignment of the topmost node in q's SpDeque, or for making the topmost node of q's SpDeque become stealable.
Proof.Let u denote the topmost node of q's SpDeque at the beginning of step i.We first prove that u either gets assigned or becomes stealable.
Three possible scenarios may take place due to p's steal attempt targeting q's SpDeque.
The invocation returns a node If p stole u, then, u gets assigned to p.
Otherwise, some other processor removed u before p did, implying u got assigned to that other processor.
The invocation aborts Since the SpDeque implementation meets the relaxed semantics on any good set of invocations, and because the Low Cost Work Stealing algorithm only makes good sets of invocations, we conclude that some other processor successfully removed a topmost node from q's SpDeque during the aborted steal attempt made by p.
If the removed node was u, u gets assigned to a processor (that may either be q, or, some other thief that successfully stole u).Otherwise, u must have been previously stolen by a thief or popped by q, and thus became assigned to some processor.
The invocation returns empty This situation can only occur if either q's SpDeque is completely empty, or if there is no node in the public part of q's SpDeque.
• For the first case, since q ∈ D i , some processor must have successfully removed u from q's SpDeque.Consequently, u was assigned to a processor.
• If there was no node in the public part of q's SpDeque, p sets q's targeted flag to true in a later step j .Recall that, for each C consecutive instructions executed by a processor, at least one corresponds to a milestone.It follows that j ≤ j + C. Furthermore, by observing Algorithm 1, we conclude that q will make and complete an invocation to updateBottom of its SpDeque in one of the C steps succeeding step j .Thus if q's SpDeque's private part is not empty, a node will become stealable.From that invocation, only two possible situations can take place: No node becomes stealable In this case, the private part of q's SpDeque was empty, implying some processor (either q or some thief) assigned u.A node becomes stealable If the node that became stealable as the result of the invocation was not u, then either u was assigned by a processor (that could have been q or some thief), or u had already been transfered to the public part of q's SpDeque as a consequence of another thief's steal attempt that also returned empty, implying that either u became assigned, or it became stealable.Otherwise, the node that became stealable as a result of the updateBottom's invocation was u.Thus, in any case, u either gets assigned to a processor or becomes stealable.
With this, we conclude that u either became assigned or became stealable until step j + 2C.From Lemma 4.5 (corresponding to Lemma A.7 of the appendix), we have Furthermore, Lemma 4.4 (corresponding to Lemma A.6 of the appendix) proves that if u gets assigned the potential decreases by at least 3 4 φ i (u) , and if u becomes stealable the potential also decreases by at least 3 4 φ i (u) .
Because u is either assigned or becomes stealable in any case, we conclude the potential associated with q at step j + 2C has decreased by at least 4 5 The following lemma is trivial a generalization of the original result presented in [8,Balls and Weighted Bins].The only difference between the two results is the assumption of having at least B balls, rather than exactly B balls.Its proof is only presented for the sake of completion, and is (trivially) adapted from the proof of [8, Balls and Weighted Bins].
Lemma A.9 (Balls and Weighted Bins).Suppose we are given at least B balls, and exactly B bins.Each of the balls is tossed independently and uniformly at random into one of the B bins, where for i = 1, . . ., B, bin i has a weight W i .The total weight is W = B i=1 W i .For each bin i, we define the random variable X i as W i if some ball lands in bin i 0 otherwise and define the random variable X as X = B i=1 X i .Then, for any β in the range 0 < β < 1, we have Proof.Consider the random variable W i − X i taking the value of W i when no ball lands in bin i and 0 otherwise, and let B denote the total number of balls that are tossed.It follows From the linearity of expectation, we have From Markov's Inequality it follows The following result states that for each P idle iterations that take place, with constant probability the potential drops by a constant factor.An analogous lemma was originally presented in [8, Lemma 8] for the non-blocking Work Stealing algorithm.The result is a consequence of Lemmas 4.7 and 4.6 (corresponding to Lemmas A.9 and A.8 of the appendix, respectively) and its proof follows the same traits as the one presented in that study.
Lemma A.10.Consider any step i and any later step j such that at least P idle iterations occur from i (inclusive) to j (exclusive).Then, we have Proof.By Lemma 4.6 (corresponding to Lemma A.8 of the appendix) we know that for each processor p ∈ D i that is targeted by a steal attempt, the potential drops by at least 3 5 Φ i (p), at most 2C steps after being targeted.When executing an idle iteration, a processor plays the role of a thief attempting to steal work from some victim.Thus, since P idle iterations occur from step i (inclusive) to step j (exclusive), at least P steal attempts take place during that same interval.We can think of each such steal attempt as a ball toss of Lemma 4.7 (corresponding to Lemma A.9 of the appendix).
For each processor p in D i , we assign it a weight and for each other processor p in A i , we assign it a weight W p = 0.
Clearly, the weights sum to Using β = 1 2 in Lemma 4.7 (Lemma A.9 of the appendix) it follows that with probability at least Finally, we bound the expected number of idle iterations that take place during a computation's execution using the Low Cost Work Stealing algorithm.The result follows from Lemma 4.8 (corresponding to Lemma A.10 of the appendix) and is proved using similar arguments as the ones used in the proof of [8,Theorem 9].The presented proof corresponds to an adaptation of the one originally presented for the just mentioned Theorem.Proof.To analyze the number of idle iterations, we break the execution into phases, each composed by Θ (P ) idle iterations.Then, we prove that, with constant probability, a phase leads the potential to drop by a constant factor.A computation's execution begins when the root gets assigned to a processor.By definition, the weight of the root is T ∞ , implying the potential at the beginning of a computation's execution starts at Φ 0 = 4 3T∞−2 .Furthermore, it is straightforward to deduce that the potential is 0 after (and only after) a computation's execution terminates.We use these facts to bound the expected number of phases needed to decrease the potential down to 0. The first phase starts at step t 1 = 1, and ends at the first step t 1 such that, at least P idle iterations took place during the interval [t 1 , t 1 − 2C].The second phase starts at step t 2 = t 1 + 1, and so on.
Consider two consecutive phases starting at steps i and j respectively.We now prove that .
Recall that we can partition the potential as Since, from the beginning of each phase and until its last 2C steps, at least P idle iterations take place, then, by Lemma 4.8 (corresponding to Lemma A.10 of the appendix) it follows .
Now, we have to prove the potential also drops by a constant fraction of Φ i (A i ).Consider some processor p ∈ A i : • If p does not have an assigned node, then • Otherwise, if p has an assigned node u at step i, then, Noting that each phase has more than C steps, then, p executes u before the next phase begins (i.e.before step j).Thus, the potential drops by at least 47 64 φ i (u) during that phase.Cumulatively, for each p ∈ A i , it follows Thus, no matter how Φ i is partitioned between Φ i (A i ) and Φ i (D i ), we have We say a phase is successful if it leads the potential to decrease by at least a 3  10 fraction.So, a phase succeeds with probability at least 1 4 .Since the potential is an integer, and, as aforementioned, starts at Φ 0 = 4 3T∞−2 and ends at 0, then, there can be at most ln (4) ln 10   7   < 12T ∞ successful phases.If we think of each phase as a coin toss, where the probability that we get heads is at least 1  4 , then, the expected number of coins we have to toss to get heads 12T ∞ times is at most 48T ∞ .In the same way, the expected number of phases needed to obtain 12T ∞ successful ones is at most 48T ∞ .Consequently, the expected number of phases is O (T ∞ ).Moreover, as each phase contains O (P ) idle iterations, the expected number of idle iterations is O (P T ∞ ).Now, suppose the execution takes n = 48T ∞ + m phases.Each phase succeeds with probability greater or equal to p = 1 4 , meaning the expected number of successes is at least We now compute the probability that the number of X successes is less than 12T ∞ .We use Chernoff bound [6], = ε Thus, the probability that the execution takes 96T ∞ +16 ln 1 ε phases or more, is less than ε.With this we conclude that the number of idle iterations is at most O T ∞ + ln 1 ε P with probability at least 1 − ε.

Lemma 3 . 1 .Lemma 3 . 2 .
No invocation to push requires a MFence instruction.Proof.Since the push method operates only once over a single publicly accessible field (entries) of the SpDeque's state, no MFence instructions are required.No invocation to pop requires a MFence instruction.

Lemma 4 . 1 .
Consider any computation with work T 1 being executed by P processors, under Low Cost Work Stealing.The execution time is O T 1 P + I P , where I denotes the number of idle iterations executed by processors.

Lemma 4 . 3 .
Consider any computation being executed by the Low Cost Work Stealing algorithm, using P processors.The number of CAS and MFence instructions executed by processors during the computation's execution is at most O (I + P ), where I denotes the total number of idle iterations executed by processors.

Lemma A. 3 .
Consider any computation with work T 1 being executed by P processors, under Low Cost Work Stealing.The execution time is O T 1 P + I P , where I denotes the number of idle iterations executed by processors.
D i ) ,concluding the proof of this lemma.

Lemma A. 11 .
Consider any computation with work T 1 and critical-path length T ∞ being executed by Low Cost Work Stealing using P processors.The expected number of idle iterations is at most O (P T ∞ ).Moreover, with probability at least 1−ε the number of idle iterations is at most O T ∞ + ln 1 ε P .

− a 2 2npwith a = m 4 .− m 4 = 16 =
It follows, np − a = 12T ∞ + m 4 12T ∞ .Choosing m = 48T ∞ + 16 ln 1 ε , We haveP {X < 12T ∞ } < e e ln(ε) 1. there exists only one root and one sink in G; 2. the Finally, using Lemma 4.9, we can obtain bounds on both expected runtime of computations executed by the Low Cost Work Stealing algorithm, and the associated synchronization overheads.Theorem 4.10.Consider any computation with work T 1 and critical-path length T ∞ being executed by the Low Cost Work Stealing algorithm with P processors.The expected execution time is at most O T 1 P + T ∞ .Furthermore, with probability at least 1−ε the execution time is at most O T 1 P + T ∞ + ln 1 ε .Proof.This result follows directly from Lemmas 4.1 and 4.9.Theorem 4.11.Consider any computation with work T 1 and critical-path length T ∞ being executed by the Low Cost Work Stealing algorithm with P processors.The expected number of CAS and MFence instructions executed during the computation's execution caused by Low Cost Work Stealing is at most O (P T ∞ ).Moreover, with probability at least 1 − ε the number of CAS and MFence instructions executed is at most O P T ∞ + ln 1 Lemma 4.3 and Lemma 4.9 prove this theorem.Corollary 4.12.Consider the statement of Theorem 4.11.Furthermore, let C CAS and C M F ence denote, respectively, the maximum synchronization overheads incurred by the execution of a CAS and MFence instructions.Then, the expected synchronization overheads incurred by Low Cost Work Stealing are at most O ((C CAS + C M F ence ) P T ∞ ) Moreover, with probability at least 1 − ε the synchronization overheads incurred by Low Cost Work Stealing are at most ε .Proof.O (C CAS + C M F ence ) P T ∞ + ln 1 ε Proof.As already mentioned, and by considering the definition of Low Cost Work Stealing, depicted in Algorithm 1, the only synchronization mechanisms the scheduler uses are CAS and MFence instructions.This corollary is then a direct consequence of Theorem 4.11 that takes into account the maximum possible overhead incurred by the execution of a single CAS and MFence instructions.
Algorithm 1 The Low Cost Work Stealing algorithm.