Collapse of Superradiant Phase and Unstable Macroscopic Vacuum State in An-Optomechanical-Dual-Cavity with a Bose-Einstein Condensate

Based on spin-coherent-state variational method, we mainly study the multiple stable macroscopic quantum states and quantum phase transitions of a Bose-Einstein Condensate in an optomechanical dual-cavity by modulating the dual-cavity interaction and the nonlinear phonon-photon interaction. Especially, the collapse of superradiant phase can be tuned in the existing nonlinear photon-phonon interaction, but the critical quantum phase transition point or boundary hasn’t been influenced. As a result, the system occurs an additional phase transition from the superradiant phase to the inversely atomic populated state. Moreover, when dual-cavity coupling interaction increases to a certain value, a new quantum phase transition from the normal phase to the inversely atomic populated state will appear. Finally, the superradiant phase completely collapses and the normal phase also collapses into an unstable macroscopic vacuum state for a strong dual-cavity coupling interaction.


Introduction
At present, cavity optomechanics [1][2][3][4], which mainly study the interaction between the light field and the mechanical oscillator at low energy scale [5,6], has become an interdisciplinary field of crossing condensed matter physics, material science, optics, quantum optics and other fields. The development of cavity optomechanics provides a broad prospect for exploring the mechanical quantum properties of macroscopic objects [1,7]. A photon scattered from an object transfers the momentum to the scatterer and thereby radiation pressure force is applied on it [8]. In fact, the radiation pressure was predicted by Maxwell long long ago, but until a century ago it was observed experimentally [9]. Optical radiation pressure could induce various interesting quantum phenomena, such as quantum entanglement [10,11], ground-state cooling of the mechanical oscillator [12][13][14], electromagnetically-induced-transparency (EIT) [15,16], nonlinear quantum effect [17,18] and the phonon blocking [19,20]. The radiation pressure is a function of the displacement of the movable mirror, which is called the optical spring effect [21,22]. By using the mutual coupling between the cavity mode and the mechanical oscillator mode, the modulation and the control of the mechanical harmonic oscillator [23] can be firstly realized, so as to the ground state cooling of the mechanical harmonic oscillator [24]. Moreover, the mutual control and detection can also be realized, for example, the high fidelity metric quantum state transmission between the cavity mode and the mechanical oscillator mode [25]. More interestingly, the entanglement [26] and the confinement of the photons between the photons and the systems with different energy levels [27] are realized. We also found many implications, for example, the photon amplification technology [27], the fast and slow light effect [28], and the nonlinear Kerr effect [29]. The dynamical Casimir effect, strong quadrature squeezing and quantum amplification have been realized and discussed in a hybrid optomechanical cavity [30,31]. Long-time joint spectra and entanglement of two photoelectrons originating in interacting auto-ionization systems have been investigated [32].
Quantum phase transition (QPT) describes quantum many-body system's physical properties at absolute zero temperature. In this temperature situation, the system doesn't exist thermodynamic fluctuations, only quantum fluctuations [33,34]. When the internal order-parameter (such as the coupling strength between particles) is changed, the system will change continuously and dramatically from a disordered state to an ordered state at a critical point, so we say that the system has a phase transition. QPT has a wide range of applications, for example, quantum correlation [35,36], the energy spectrum's properties of ground state [33] and quantum information [37]. The realization of a Bose-Einstein Condensate in rarefied atomic gas [38,39] provides a new way to study quantum multibody physics. Furthermore, a Bose-Einstein Condensate has been trapped in optical cavity or optical lattice with the spin-orbit coupling in theoretical research [40]. Dicke model is the representative model to study the phenomenon of QPT.
Dicke model, which describes the collective interaction between N identical two-level atoms and a single-mode electromagnetic field, shows a second-order QPT from the normal phase (NP) to the superradiant phase (SP) [41,42]. The NP represents no collective excitation in the system, while the SP represents the system occurs the collective coherent excitation of the atomic and the photonic numbers. It plays an important role in the research of a BEC in optical cavities [43,44]. Ten year ago, this phase transition was firstly and successfully observed by introducing two optical Raman transitions into a four-level atomic ensemble in the experiment, which is trapped in a high-precision optical cavity in the experiment [45,46]. This experimental operation meets that the atom-field coupling strength is in the same order of magnitude as the atomic frequency, which is difficult to meet before. More importantly, this experimental setup can be seen as a powerful platform for exploring novel many body QPTs from atomic physics to quantum optics in a controllable manner. Dissipations are inevitable. Dissipation induced bistability in the two photon Dicke model has been presented by Franco Nori [47]. In a driven dissipative optomechanical cavity, a Green's function approach and the quantum Langevin equations are used to discuss the linear response, the nonlinear effects of atomic collisions, the dispersive interaction, the effects of quadratic coupling and force sensing [48][49][50][51][52]. Moreover, Ref. [53] presents the infuence of nanoparticles on freezing inside container equipped with fns. An interesting phase diagram is characterized by two quantum criticalities, which are a superradiant phase transition and a spectral collapse. The effects of both qubit and photon dissipation on phase transition and the instability have been detained discussed. The spectral collapse induces the instability. We know the physics of Dicke-typical systems changes drastically once dissipation is taken into account. The Lindblad formalism [54] is usally used to introduce different incoherent processes. In the presence of qubit decay and dephasing, the transition of the Dicke model could be modified, suppressed, or restored [55,56]. Assuming a Markovian environment and performing the Born approximation, the dissipation channels, that may be, individual qubit decay and dephasing, and photon loss, can be described by a Lindblad master equation [54]. Usually using a mean-field decoupling approximation, the steady-state expectation values of the observables, which signaling a symmetry breaking, identifying a first-order phase transition from the normal to the superradiant phase. The rich phase diagrams display stable, bistable, and unstable phases, which are depending on the dissipation rate [47]. Just different this idea, our related work has proposed such stable or unstable phase and similar optical bistability for an standard optomechanical system without including the decay by means of the spin-coherent-state variational method [41,57]. Ref. [47] presents that atomic decay and dephasing contribute to the stabilization of the SP. For verification and simulation such results, we firstly propose to discuss the ground state properties in an ideal opto-mechanical and cavity systems. Under this, we will consider the stability analysis of a Bose-Einstein Condensate in an optomechanical-dual-cavity in this paper. By tuning the dual-cavity coupling interaction, we hope that meaningful results, such as new phase transition and the stability can be obtained. This results will help to better understand many-body's ground state properties.
With the development of double-cavity optomechanical system [58][59][60], in the past several years, a novel hybrid cavity optomechanical system consisting of a mechanical resonator and two coupled single-mode cavities has been introduced [4,8,61]. Moreover, a Bose-Einstein Condensate is trapped to the feedback cavity 2 and the mechanical oscillator couples to Cavity 1. Such system in Ref. [8] can realize the optical bistability for much lower threshold power by tuning the frequency and the power of the driving. And a Bose-Einstein Condensate in the feedback cavity can largely affect the optical bistability. In all, such system allows more flexibility in controlling bistability compared with standard optomechanical cavity and may realize a contrallable optical switch by tuning the hybrid interaction. Such hybrid optomechanical system also can achieve the ground state cooling of mechanical resonator by an EIT-like cooling mechanism Ref. [61]. A Bose-Einstein Condensate makes the optical noise spectrum split into an EIT-like spectrum and greatly increase the asymmetry. Even in unresolved sideband regime, the ground state cooling of mechanical resonator can also be achieved by adjusting the atom-field coupling strength. This opens up the possibility for applying cavity quantum optomechanics beyond the resolved sideband regime. An imaginary-time functional path integral approach has be used to study the finite-temperature properties in such system [4]. Such high-finesse system induces the SP collapses and turns to an unstable non-zero photon number beyond the turning curve, which is similar to the optical bistability. These results can promote the development of finite temperature characteristics about hybrid cavity optomechanical system in the experiment.
Motivated by these works above, in this paper, we firstly present the energy functional of a Bose-Einstein Condensate in an optomechanical-dual-cavity [4,8,61] by means of the spin-coherent-state variational method [57], which is not only suitable for arbitrary number of atoms but also can take into account the inversely atomic populated state, which is necessary for the stimulated radiation and multiple stable states. The atomic population number and the mean photon number are plotted as a function of the atom-field coupling strength for the given dual-cavity coupling interaction J or the photon-phonon interaction . Furthermore, Hessian matrix [62] is used to judge the ground state stability of the system. We found that the famous Dicke QPT from the NP to the SP has been displayed in the ground state. The critical QPT point or boundary can be tunned by the dual-cavity coupling interaction J. The nonlinear photon-phonon coupling interaction has no effect on the critical QPT point or boundary, but induces the collapse of the SP. Meanwhile, the unstable macroscopic vacuum state (UMV) can be tuned in the suitable dual-cavity coupling interaction. These predicted properties can be experimentally detected by measuring the atom population, the mean photon number or the mean phonon number. In fact, in the resonance or detuning, in addition to the usual second-order Dicke QPT from the NP to the SP, the collapse of the SP can also be induced by adjusting nonlinear photon-phonon coupling interaction in the case of J < √ Δ 2 Δ 1 , while the UMV can be observed by adjusting J in the case of J > √ Δ 2 Δ 1 .

Hamiltonian
As shown in Fig. 1, we propose a Bose-Einstein Condensate in a realizable optomechanical dual-cavity, in which two linearly coupled single-mode optical cavities are coupled with a mechanical oscillator through the radiation pressure. The mechanical oscillator with the resonace frequency m and the effective mass m is coupled to the empty cavity 1 driven by a strong pump laser with the frequency p . In this case, the subsystem forms a standard optomechanical cavity. Cavity 2 with the frequency 2 contains a two-level ultracold atomic ensemble and is coupled with cavity 1 with the frequency 1 through the dual-cavity coupling strength J. The Hamiltonian of the system is where the first three terms respectively denote the free energies of the cavity mode and the mechanical oscillator mode with the creation operators a † i (b † ) and annihilation operators a i (b) . The forth term is the energy of the ultracold two-level atomic ensemble with the frequency 0 in cavity 2. S + , S − and S z are the collective spin operator. The fifth term describes the interaction between the ultracold two-level atomic ensemble and the optical cavity 2 with the atom-field coupling strength g. The sixth term represents the dual-cavity coupling interaction. The seventh term represents the photon-phonon interaction between the mechanical oscillator and cavity 1 with the coupling strength . The last term denotes the effect of the external driving pump laser term with the intensity = √ 2 1 P in, 1 ℏ p ( P in,1 is the input laser power and 1 is the damping rate of cavity 1). In a rotating coordinate frame, by time-dependent unitary transformation U = e i p t a † 1 a 1 +a † 2 a 2 +S z the time-dependent Hamiltonian (1) becomes a time-independent Hamiltonian Ignoring the high frequency oscillation terms a † 2 S + e 2i p t and a 2 S − e −2i p t , the time-independent Hamiltonian finally obtained

Variational Solution and Stability
In order to study the ground state properties of our system, we need to firstly get the system's energy functional by spin-coherent-state variational method. However, this method not only can't give the information of its transition, but also can't the result of what kind of state does it relax to under perturbation.
where the pseudospin Hamiltonian meets Noting that the trial wave function is the direct produce states about Bose coherent state of two mode optical cavity (the photons) and one mode mechanical oscillator (the phonon), satisfying variational parameters and the corresponding eigenvalues of Bose coherent states [57]. The detailed process can be found from the references [41,63]. Secondly the pseudospin Hamiltonian (5) can be diagonalized directly by the spincoherent states [64,65]. Starting from the macroscopic eigenstates | | ∓� ⃗ n ⟩ with the unit vector � ⃗ n = (sin cos , sin sin , cos ) , namely the spin-coherent states of south pole and north pole gauges, we can obtain where the maximum spin eigenstate | | ∓� ⃗ n ⟩ can be generated from the maximum Dicke states �s, ∓s⟩ with S z �s, ∓s⟩ = ∓s�s, ∓s⟩ by the unity transformation � � ∓� ⃗ n � = R( , )�s, ∓s⟩ with the unitary operator [64,65] where and are undetermined coefficients and correspond to the direction angle.
Diagonalized Hamiltonian (10) in the spin-coherent states | | ∓� ⃗ n ⟩ , we need to meet B( 2 , ; , ) = C( 2 , ; , ) = 0 and the eigen energy is After the algebraic calculation, the coefficient , the mean energy functional can be expressed as According to the extremum condition of phase parameter, we can simplify the coefficient A( 2 , )and the energy functional In the experiment, the order of magnitudes is much smaller than the effective frequencies of Δ 1 , Δ 2 , m and the coupling strength J, , g, so the term −2 1 in (15) has little and negligible effect on the energy level structure of the system. Finally the energy functional is rewritten as By variational principle, the equilibrium equations are found as The solutions of the macroscopic many particle quantum states can be obtained from the above equilibrium equation. From (19), (18), the variational parameters meet (20) into (17), the photon number solution of optical cavity 2 can be obtained from the equation From (21), we find that there're two types of the photon number solution. One is the zero photon number solution with 2 = 0 , while the other is the nonzero photon number solution 2 ≠ 0 . The boundary between 2 = 0 and 2 ≠ 0 is called the phase boundary and marked g ± c . Such boundary can be obtained from (21) with the following form Based on variational method, we easily realize that for the solution 2 = 0 , it may be a stable or unstable solution with the positvie or negtive second-order derivative of the ground state's energy functional E − . When the second-order derivative of ground state's energy functional E − is positive, i.e., 2 E − 2 = 0 ∕ 2 2 > 0 , the zero photon number solution 2 = 0 is stable and called the normal phase (NP), which is coincide with the theory of QPT in Ref. [41,42] and the related references. While when the second-order derivative of ground state's energy functional E − is negtive, i.e., 2 E − 2 = 0 ∕ 2 2 < 0 , the zero photon number solution 2 = 0 is unstable and called the unstable macroscopic vacuum (UMV), which is firstly put forward in Ref. [63]. Meanwhile, for the energy functional E + , whose energy is higher than the ground state E − , the stable zero photon number solution 2 = 0 is definited as the inverted state N + with the positive second-order derivative of the energy functional E + , i.e., 2 E + 2 = 0 ∕ 2 2 > 0 , while the unstable zero photon number solution for 2 E + 2 = 0 ∕ 2 2 < 0 has no physical meaning for the system and needs no consideration. Generally, the energy functional E + for the inverted state N + is higher energy than the ground state E − . While, in some case, there is no stable zero photon number solution N − for the energy E − and just stable zero photon number solution N + for the energy functional E + , at this time the energy functional E + is seen as the ground state, which can be seen the red line in Fig. 7(b).
Next, we consider and discuss the nonzero photon number solution in (21). Substituting the nonzero solution 2 into the second-order derivative of ground state's energy functional E − , we find that the second-order derivative of ground state's energy functional E − is positive, i.e., 2 E − 2 ≠ 0 ∕ 2 2 > 0 , so this nonzero photon number solution is stable and called the superradiant phase (SP), which is also coincide with the theory of QPT in Ref. [41,42] and there's a second-order Dicke QPT from the NP to the SP in phase boundary g − c satisfied (22). When the second-order derivative of ground state's energy functional E − is negtive, i.e., 2 E − 2 ≠ 0 ∕ 2 2 < 0 , the nonzero photon number can be regarded as unstable and marked S − us in the related line figures in the following part. For the energy functional E + , there's without stable nonzero photon number 2 and the later phase diagrams present the different and interesting phase regions for the stable state by the boundray g + c . The boundray g + c devides the UMV and the inverted state N + . Though the inverted state N + isn't corresponding with the solution from ground state, the N + becomes new ground state in some region in the following phase diagrams (blue region in Figs. 3, 5, 6, 8 and 9). However, the UMV appears because just such one macroscopic state exists in blank region of related phase diagrams. In all, g − c in (22) is the phase boundary between the NP and the SP, while g + c in (22) is the phase boundary between the inversely atomic populated state ( N + ) and the unstable macroscopic vacuum (UMV). The phase boundary is mainly influented by the dual-cavity coupling strength J, while the mechanical oscillator does not affect the phase boundary. In fact, these rich states and phase boundary mainly because of the spin-coherent-state variational method, which can distinguish the energy functional E + , which is ignored and not discussed. Most works are just by variational method to discuss the ground state energy functional E − and present the QPT from the NP and SP. In this paper, we capure abundant and interesting phase transition and quantum properties.
In order to systematacially judge the stability of different phase region, now we use the Hessian matrix [62] to deal with several parameters's case ( 1 , 2 , ) of the many-body system, which is more convenient and efficient than variational method for many parameters' system. The 3 × 3 Hessian matrix has the following form where the matrix elements are calculated by formula M ij = 2 E∕ x i x j ( x = 1 , 2 , and i, j = 1, 2, 3).
By all the eigenvalues of matrix M, the ground state stability of the system can be judged. All eigenvalues of Hessian matrix M can be calculated by the formula det If M is positive definite, that is to say, all eigenvalues of M are positive, the energy functional E ∓ has local minimums accompanied by a stable phase region. If M is indefinite, that is to say, some eigenvalues are positive and some eigenvalues are negative, the energy functional E ∓ has some saddle points accompanied by a dynamically unstable region. If M is negative definite, that is to say, all eigenvalues of M are negative, then the energy functional E ∓ has local maximums with extremely unstable region. we do not consider this negative definite. Moreover, the phase boundary between the stable phase and the dynamically unstable phase can be obtained by the conditions min ( ) = 0.

Mean Photon Number, Atomic Population Number and Mean Energy
The mean photon number can be generated from the complete trial wave function. Based on the stable solution ∓ 2s from the equilibrium (21), we can obtain the mean photon number of cavity 2 where ∓ 2s 2 = ∓ 2s 2 ∕N . By the value of the order parameter about the average photon number n ∓ p2 , Dicke quantum phase transition can be characterized with n ∓ p2 > 0 in SP and n ∓ p2 = 0 in NP. The atomic population number is calculated and finally obtained in the full trial state where the atomic population number Δn ∓ a = ∓ 1 2 in the NP. We can obtain the mean energy of a single atom ∓ according to the energy functional theory and (16) where ∓ = ∓ 1 2 Δ 0 in the NP. The mean phonon number can also be given from (20) and the related states or phases correspond to the mean photon number, atomic population number and mean energy.

Quantum Langevin Equations and the Stability
with the thermal phonon number of the mechanical resonator n th = exp T is the environmental temperature and k B is the Boltzmann constant. The Brownian noise operator associated with the damping of the mechanical oscillator can be given All the operator can be rewritten as the stable values of the operator and the corresponding fluctuation parts, i.e., a 1 = ⟨a 1 ⟩ + a 1 , so the steady-state values can be obtained The bistability according to (29) can be discussed and optical bistability occurs in open optomechanical-dual-cavity with a Bose-Einstein Condensate, which is presented in Ref. [8]. While we would like observe whether the similar optical bistable behavior will occur in an closed optomechanical-dual-cavity with a Bose-Einstein condensate without quantum noises and dissipations. In fact, in some parameters there is an similar optical bistability for beyond the turning point g − t , the unstable nonzero photon state S − us (blue dot line) turns back to an upper branch seen in Figs. 4 and 6. So we have a dual state for one value of g. This is similar to the optical bistability, however, the state S − us is unstable in our considered system. The conditions are that the dual-cavity coupling interaction must be less than 1 and the mechanical oscillator (the photon-phonon interaction) need to exsist. The detailed discussions are in the parts of Figs. 4 and 7.
Following the usual linearization approach [67,68] for the steady part ⟨a 1 ⟩ is much bigger than 1 in optomechanical systems, the corresponding effective linearized Hamiltonian of the fluctuation operators (hereafter dropping the notation " " for all the fluctuation operators for the sake of simplicity) is written as where the new Δ 1 and are respectively equal to Δ 1 − � ⟨b⟩ * + ⟨b⟩ � and ⟨a 1 ⟩.

Multiple Quantum Phase Transition
According to the extremum conditions and stability criteria in Part III, in the resonance ( Δ 1 ∕Δ 0 = Δ 2 ∕Δ 0 = 1 ), phases diagrams of J − g plane and the corresponding curve graph are respectively plotted from Figs. 2 to 5. Figure 2 displays the curves for atomic population number Δn ∓ a as a function of the atom-field coupling strength g for different dual-cavity coupling interactions J∕Δ 0 = 0, 0.5, 1.5 and the absence of the photon-phonon interaction = 0 in the resonance ( Δ 1 ∕Δ 0 = Δ 2 ∕Δ 0 = 1 ). When cavity 1 vanishes with J = 0 , our system returns to Dicke Hamiltonian and appears a famous second QPT from the NP ( N − ) to the SP ( S − ) in the critical point g − c = √ Δ 2 Δ 0 . The inverted state N + (red dotted line) for the mean energy + exists in the whole region. Though it's stable for positive definite Hessian matrix M, the mean energy + is higher than the ground-state energy − . With the increase of dual-cavity  Fig. 2(b). Before g − c , N − coexists with N + (corresponding to the yellow region in Fig. 3), while S − coexists with N + after g − c (corresponding to the pink region in Fig. 3). When J > 1 , for example, J∕Δ 0 = 1.5 in Fig. 2(c), the critical point g + c = 1.11 . Before g + c , there just exists zero-photon number solution, which is unstable for negative definite Hessian matrix M, called the unstable macroscopic vacuum (UMV) (corresponding to the blank region in Fig. 3). After g + c , just a stable N + exists and goes to the ground state (corresponding to the green region in Fig. 3). The corresponding and detailed phase diagram is depicted in Fig. 3. Yellow region is the bistable NP bi N − , N + before g − c , while the coexistence phase of SP co S − , N + after g − c in pink region. Noticing that the white line corresponds to the value J∕Δ 0 = 1 , where the system has no physical solution according to (22). It can be seen that large J induces the disappearances of quantum phase and QPT and arises the new stable N + as the ground state and interesting UMV. Figure 4 depicts the curves for atomic population number Δn ∓ a as a function of the atom-field coupling strength g for different dual-cavity coupling interactions J∕Δ 0 = 0, 0.3, 0.8, 1.3 and the photon-phonon interaction ∕Δ 0 = 1 in the resonance Δ 1 ∕Δ 0 = Δ 2 ∕Δ 0 = 1 . Fig. 4(a) for J = 0 is the same as Fig. 2(a). Compared with Figs. 2(b), 4(b) appears a new critical point g − t , which is called the turning point and similar to the turning point in Ref. [4,57]. Before g − t , a new solution for unstable nonzero photon number appears, labeled as S − us (blue dot line), in which the corresponding Hessian matrix M is indefinite. S − us coexists with N + or S − , similar to optical bistability, but different , which is one new result. Though the state N + exists in all region, but may be stable or not and ground state or not, distinguishing by g + c and g − t . With the increase of dual-cavity coupling strength, for example, J∕Δ 0 = 0.8 , the SP of the S − collapes all the time and finally completely collapses. At this moment, g − t coincides with g − c and another QPT from N − to N + at g − c occurs. This new phase transition just is the atomic population transfer (or spin flip) between two atom levels . When J continues to increase, for example, J∕Δ 0 = 1.5 , just an stable N + exists after g > g + c . Before g + c , just an unstable zero photon number exists and corresponds to UMV (blank region) in Fig. 5.
The corresponding and detailed phase diagram is depicted in Fig. 5. The new lightspot is that the photon-phonon interaction induces the collapse of the S − in g − t , but has no effect on the bistable NP bi N − , N + , the critical point g − c , the UMV, NP N + and g + c . Before g − c , the region is the bistable NP bi N − , N + (yellow region), while the coexistence phase of SP co S − , N + between g − c and g − t (pink region). Between g − t and g + c , just NP N + (blue region) exists, which just exists in the case of J > 1 without the photon-phonon interaction in Fig. 3. Noticing that the white line corresponds to the value J∕Δ 0 = 1 , where the system has no physical solution according to (22). It can be seen that large J induces the disappearances of quantum phase and QPT and arises the new stable N + as the ground state and interesting UMV.
In Fig. 6, phase diagram of g − space is plotted for different dual-cavity coupling interaction J in resonance Δ 1 ∕Δ 0 = Δ 2 ∕Δ 0 = 1 . Seen from Fig. 6(a), the NP of N − and the critical point g − c between NP bi N − , N + and SP co S − , N + aren't affected by the mechanical oscillator. However, the SP of S − is bounded by g − t and collapses beyond g − t on account of the mechanical oscillator's resonant damping [57,62]. Increasing photon-phonon interaction and dual-cavity coupling interaction J can accelerate the collapsing, especially completely disappearance of the SP for J∕Δ 0 = 1.01 . However, the appearance of the collapsing and the g − t is due to the mechanical oscillator, not dual-cavity coupling interaction J, which also can been seen from Figs. 2 and 3. The g + c separates the two regions of the UMV and NP N + in Fig. 6(c,d).
Corresponding to Figs. 6(a), 7 shows the curves of the mean photon number n ∓ p2 , atomic population number Δn ∓ a and mean energy ∓ with the atom-field coupling strength g for given dual-cavity coupling interaction J∕Δ 0 = 0.3 and the photon-phonon interaction = 0.3(a), 1.0(b) in the resonance Δ 1 ∕Δ 0 = Δ 2 ∕Δ 0 = 1 . Seen from Fig. 7(a), Dicke QPT from N − to S − occurs at the critical point g − c for small photon-phonon coupling strength and the stable N + exists in all the region with higher mean energy. In addition, the unstable state S − us (blue dot line) also exists and coexists with N − or S − , and the corresponding energy in Fig. 7(3) is higher than the ground state (black line). Compared with Fig. 7(a), (b) exhibits an additional QPT at g − t from S − to N + for large and the state S − us disappears at g − t . Figures 8 and 9 are respectively in blue detuning ( Δ 1 ∕Δ 0 = Δ 2 ∕Δ 0 = 2 ) and red detuning ( Δ 1 ∕Δ 0 = Δ 2 ∕Δ 0 = 0.5 ) plotted phase diagrams of g − J for given ∕Δ 0 = 0 (a), 1.0(b). Compared Fig. 8(a)

Cooling with Quantum Noise Spectrum and the Steady-State Phonon Number
According to the effective Hamiltonian (30) and using Fermi's golden rule as given in Ref. [68], the rate equations of the mechanical oscillator can be written out and finally the steady-state final mean phonon number of the mechanical oscillator has the following form We notice that n f → n c when m = 0 from (31), so n c and c are respectively called the quantum limit of cooling and the cooling rate. To cool the mechanical oscillator to its ground state, we usually require that the cooling rate c is much greater than the heat decay rate of the mechanical oscillator m i.e., c ≫ m . In all, the positive-and negative-frequency parts (32)   of quantum noise spectrum S FF (± m ) mainly determine the steady-state final mean phonon number of the mechanical oscillator n f seen from (32). Seening from (32), (33), we need to strengthen the positive-frequency part S FF (+ m ) and suppress the negative-frequency part S FF (− m ) to satisfy simultaneously small cooling limit n c and large cooling rate c .
Under the weak-coupling regime, quantum noise spectrum S FF ( ) is completely determined by the optical part in Hamiltonian (30). Using Fourier transform and inverse transform to rewrite the correlation functions of the noise operators about the frequency and according to the optical force F = a † 1 + a 1 , after a tedious calculations as given in Ref. [69], quantum noise spectrum S FF ( m ) can be written as with Based on (31) and (34), Figs. 10 and 11 respectively depict quantum noise spectrum and the final steady-state mean phonon number.
In Fig. 10, quantum noise spectrum S( ) versus the frequency is depicted with three different dual-cavity coupling interactions J. The existence of J makes the single Lorentzian peak split into two relatively narrower peaks and makes a dip emerge between them. Physical explaination about the origin of the dip is that it's similar to the two-photon resonance in the EIT phenomenon of a three-level atomic system in Ref. [69]. The minimal point of the spectrum S( ) locates at = Δ 2 , corresponding to the two-photon resonance condition in EIT or EIT-like phenomenon. When = 0.3 m , the positions of the two peaks of quantum noise spectrum S( ) depend strongly on J. In order to maximize the transition rate of the cooling process, quantum noise spectrum S( = + m ) should be as large as possible. So the center of the right-hand peak should be around = + m .
. In order to consider the final cooling of the mechanical oscillator, we take a set of experimentally feasible parameters as follows: m = 1.55 × 20MHz , Q m = m ∕ m = 6.2 × 10 4 , = 0.3 m , | | = 6000 m (corresponding to the driving power P ∼ mW ), and the initial phonon number n m = 403 (environment temperature T = 300mK ). In Fig. 11, the steadystate phonon number n f is given as a function of the dual-cavity coupling interactions , and the corresponding cooling limit in equation. (32), n c for fixed 0 = 10 m is plotted. In principle, the cooling limit n c becomes closer and closer to zero as dual-cavity coupling interactions increases. However, in a realistic system, the final steady-state phonon number, n f , just takes the cooling limit n c when the mechanical oscillator thermal effect is much larger than the effect induced by the optical field, that is, m n m ≪ c n c . This will be not always valid, especially when n c → 0 for large J, along with J increases, their deviation is getting large. The main result in Fig. 11 is that when dual-cavity coupling interactions J is smaller, along with J increases, n f and n c reduce; when J is larger, along with J increases, n f increase gradually until the hot phonons number. So in order to good cooling the mechanical oscillator, the dual-cavity coupling interactions can't be too large, which is consistent with the region of quantum phase transition.

Conclusion
Multiple stable macroscopic quantum states and new QPT are presented in an-optomechanical-dual-cavity with a Bose-Einstein Condensate by means of the spin-coherent-state variational method. Without the dual-cavity coupling interaction J and the nonlinear phononphoton coupling interaction of the mechanical oscillator , the system is just a Dicke model with a second-order QPT from the NP ( N − ) to the SP ( S − ). By tuning the J in the absence of the , quantum phase and QPT can disappear, meanwhile, a new stable N + as the ground state and the UMV appears. By tuning the , we conclude that the mechanical oscillator has no effect in the vacuum cavity, while greatly effect in the SP because it is coupled to the cavity through the radiation pressure. The mechanical oscillator induces the SP(S − ) to collapse and finally completely disappear. This behavior is completely different from Dicke QPT. The collapse of the SP leads to another QPTs from the SP or the NP to the inversely atomic populated state ( N + ) at g − t or g − c . In addition, the N + also collapses to the UMV for a dual-cavity strong coupling interaction. In the detuning, quantum phases and QPTs are the same as the case in the resonance, but phase boundaries are changed. The spincoherent-state variational method can be used to study many body system with arbitrary atomic number and consider the system's normal and inversely pseudospin state, which can be applied in laser physics. This is different from the Holstein-Primakof transformation method in the large particle limit. The existence of the dual-cavity coupling interactions makes the single Lorentzian peak split into two relatively narrower peaks and makes a dip emerge between them. The origin of the dip is that it's similar to the two-photon resonance in the EIT phenomenon of a three-level atomic system. The dual-cavity coupling interactions can be good for cooling the mechanical oscillator.