-
Heavy neutral leptons N are well motivated to explain the origin of tiny neutrino masses. Due to the singlet nature of heavy neutral leptons under the standard model gauge group, we can write a Majorana mass term
m_N \overline{N^c}N , which results in light Majorana neutrino masses asm_\nu\sim m_D^2/m_N via the type-I seesaw mechanism [1, 2]. To generate the sub-eV neutrino masses, the heavy neutral lepton masses should be at very high scalem_N\gtrsim10^{14} GeV ifm_D is at the electroweak scale, which is far beyond current experimental reach. Alternatively, an electroweak scalem_N is usually assumed for phenomenological studies [3], which is then determined by the mixing parameterV_{\ell N} .To confirm the Majorana nature of neutrinos, lepton number violation signatures are expected. The most sensitive experiment is the neutrinoless double beta decay, which could probe the inverted mass ordering scenario in next-generation experiments [4]. Meanwhile, the detection of the lepton number violation signature at colliders could unravel the explicit mechanism of tiny neutrino masses [5]. For heavy neutral leptons in the type-I seesaw, the distinct signature at the hadron collider is
pp\to W^{\pm*}\to \ell^\pm N\to\ell^\pm\ell^\pm jj [6]. However, the sensitive region of this signature heavily depends on a relatively large mixing parameterV_{\ell N} [7, 8], which is usually much higher than the natural seesaw predictionV_{\ell N}\sim\sqrt{m_\nu/m_N} .Besides the canonical seesaw, there are many theories with extended gauge groups, such as the
U(1)_{B-L} model [9, 10] and left-right symmetric model [11, 12]. In these models, the heavy neutral leptons are charged under the extended gauge group, which opens new production mechanisms for heavy neutral leptons. Pair production of heavy neutral leptons via the decay of theZ' boson in the framework of the left-right symmetric model has been searched for recently by the CMS collaboration [13]. The region withinm_N\lesssim1.4 TeV andm_{Z'}\lesssim4.4 TeV has been excluded by the dimuon channel. Meanwhile, the ATLAS collaboration searches for the right-handedW' boson decaying to heavy neural lepton N and leptons\ell , which can exclude the region withm_N\lesssim3.8 TeV andm_{W'}\lesssim6.4 TeV [14].The construction of a multi-TeV muon collider was proposed recently [15, 16]. Since then, searches for heavy neutral leptons at the multi-TeV muon collider have drawn increasing interest [17−21]. One promising signature is
\mu^+\mu^-\to N\nu , which could probe the mixing parameter|V_{\mu N}|^2\gtrsim10^{-6} at the 10 TeV muon collider [22−24]. The Majorana nature of heavy neutral leptons can be confirmed with double peaks of the rapidity distribution of the reconstructed N [25]. Another interesting signature is the lepton number violation signal via vector boson scattering processW^\pm Z/\gamma\to \ell^\pm N [26], via associated production process\mu^+\mu^-\to N W^\pm \ell^\mp [27], or at the same-sign muon collider via\mu^+\mu^+\to W^+W^+ process [28]. These studied signatures also require that the mixing parameterV_{\ell N} is not too small. Meanwhile, under the tight constraints from current experimental searches [13, 14], the first stage of a 3 TeV muon collider [29] is not promising to probe prompt heavy neutral leptons in theU(1)_{B-L} and left-right symmetric models.The multi-TeV muon collider is a perfect machine to test the muon-philic forces. One attractive option is the gauged
U(1)_{L_\mu-L_\tau} model [30−34]. Compared toU(1)_{B-L} or left-right symmetry,U(1)_{L_\mu-L_\tau} symmetry is less constrained due to the lack of direct couplings to electrons and quarks. For instance, there is only a loose constraint from neutrino trident production whenm_{Z'}\gtrsim100 GeV [35]. Therefore, this model has been extensively studied to explain the anomaly of the muon magnetic moment [36, 37], B meson anomaly [38, 39], dark matter [40, 41], and neutrino masses [42, 43]. It is shown that a 3 TeV muon collider is powerful enough to probe a relatively large parameter space withm_{Z'}\lesssim10 TeV [44, 45]. In this paper, we consider the gaugedU(1)_{L_\mu-L_\tau} model with three heavy neutral leptonsN_e,N_\mu,N_\tau . We then study the pair production of heavy neutral lepton N viaZ' boson at a 3 TeV muon collider. To test the Majorana nature of neutrinos, we focus on the lepton number violation process\mu^+\mu^-\to Z^{\prime *}\to NN\to \mu^\pm\mu^\pm W^\mp W^\mp and\mu^+\mu^-\to Z^{\prime} \gamma\to NN \gamma \to \mu^\pm\mu^\pm W ^\mp W^\mp \gamma with the hadronic decay of W.The remainder of this paper is organized as follows. In Section II, we review the gauged
U(1)_{L_\mu-L_\tau} model with three heavy neutral leptons and discuss relevant experimental constraints. The decay properties of gauge bosonZ' and heavy neutral lepton N are also considered in this section. In Section III, we study the pair production of heavy neutral lepton at the 3 TeV muon collider. Analysis of the lepton number violation signatures is performed in Section IV. Finally, Section V summarizes our conclusions. -
In this paper, we consider the anomaly-free gauged
U(1)_{L_\mu-L_\tau} extension of the type-I seesaw mechanism. Three heavy neutral leptonsN_e,N_\mu,N_\tau withU(1)_{L_\mu-L_\tau} charge(0,1,-1) are introduced to generate tiny neutrino masses. Predicting type C^R of the two-zero minor [46, 47], the minimal model with only one scalar singlet\Phi_1 , which carriesU(1)_{L_\mu-L_\tau} charge+1 , is now tightly constrained by the neutrino oscillation data and the sum of light neutrino masses [48, 49]. Thus, the second scalar singlet\Phi_2 withU(1)_{L_\mu-L_\tau} charge+2 is also employed to reduce the above conflict [50]. The Yukawa interactions and mass terms relevant to neutrino masses are given by [51]\begin{aligned}[b] {\cal{L}}\supset & - y_e \bar{L}_e \tilde{H} N_e - y_\mu \bar{L}_\mu \tilde{H} N_\mu - y_\tau \bar{L}_\tau \tilde{H} N_\tau \\ & -\frac{1}{2} M_{ee} \overline{N^c_e}N_e-M_{\mu\tau}{N^c_\mu}N_\tau -y_{e\mu} \Phi_1^\dagger \overline{N^c_e}N_\mu \\ & -y_{e\tau} \Phi_1 \overline{N^c_e}N_\tau-\frac{1}{2}y_{\mu\mu} \Phi_2^\dagger \overline{N^c_\mu}N_\mu -\frac{1}{2}y_{\tau\tau} \Phi_2 \overline{N^c_\tau}N_\tau+ \text{h.c.}, \end{aligned}
(1) where
L_e,L_\mu,L_\tau are the lepton doublets, H is the standard model Higgs doublet, and\tilde{H}=i\tau_2 H^* . After the spontaneous symmetry breaking, we can denote the vacuum expectation values of scalars as\langle H \rangle=v_0,\langle \Phi_1 \rangle=v_1,\langle \Phi_2 \rangle=v_2 . Then, the Dirac neutrino mass matrix and heavy neutral lepton mass matrix are given by\begin{aligned}[b] & M_D=\left( \begin{array}{c c c} y_e v_0 & 0 & 0\\ 0 & y_\mu v_0&0 \\ 0 & 0 & y_\tau v_0 \end{array} \right), \\ & M_N = \left( \begin{array}{c c c} M_{ee} & y_{e\mu}v_1 & y_{e\tau}v_1\\ y_{e\mu}v_1 & y_{\mu\mu} v_2& M_{\mu\tau} \\ y_{e\tau}v_1 & M_{\mu\tau} & y_{\tau\tau} v_2 \end{array}\right). \end{aligned}
(2) Light neutrino masses are generated via the type-I seesaw formula
\begin{aligned}[b] M_\nu\simeq - M_D M_N^{-1} M_D^T. \end{aligned}
(3) Without any specific structure of heavy neutral lepton mass matrix
M_N , the obtained light neutrino mass matrixM_\nu is a general symmetric matrix; thus, it can easily fit the neutrino oscillation data [52]. Conversely, we can use Eq. (3) to express the mass matrixM_N as [53]\begin{aligned}[b] M_N=-M_D^T M_\nu^{-1} M_D, \end{aligned}
(4) where
M_\nu=U^*\text{diag}(m_{\nu_1},m_{\nu_2},m_{\nu_3}) U^\dagger , and U is the light neutrino mixing matrix. In this way, the mass matrixM_N is determined by the light neutrino oscillation data and the Yukawa couplingy_e,y_\mu,y_\tau . The mass matrixM_N can be diagnalized by a unitarity matrix Ω as\begin{aligned}[b] \Omega^T M_N \Omega = \text{diag}(m_{N_1},m_{N_2},m_{N_3}). \end{aligned}
(5) Considering the mixing with light neutrinos, the heavy neutral leptons interact with the
W^\pm,Z gauge boson and Higgs boson as\begin{aligned}[b] {\cal{L}}\supset & -\frac{g}{\sqrt{2}} \bar{N}_k V_{\ell k}^{*} \gamma^\mu P_L \ell W_\mu -\frac{g}{2\cos\theta_W} \bar{N}_k V_{\ell k}^{*} \gamma^\mu P_L \nu_\ell Z_\mu \\ & - \frac{g m_N}{2 m_W} \bar{N}_k V_{\ell k}^{*} P_L \nu_\ell h + \text{h.c.}, \end{aligned}
(6) where the mixing matrix
V\simeq M_D M_N^{-1} . For electroweak-scale heavy neutral leptons, we assume a seesaw-induced mixing parameterV_{\ell N}\sim \sqrt{m_\nu/m_N}\sim10^{-6} , which is allowed by current experimental limits [3].The interactions of the new boson
Z' with fermions are\begin{aligned}[b] {\cal{L}}\supset & g'(\bar{\mu}\gamma^\mu\mu-\bar{\tau}\gamma^\mu\tau+\bar{\nu}_\mu\gamma^\mu P_L\nu_\mu-\bar{\nu}_\tau\gamma^\mu P_L\nu_\tau\\ & +\bar{N}_\mu\gamma^\mu P_R N_\mu-\bar{N}_\tau\gamma^\mu P_R N_\tau)Z'_\mu. \end{aligned}
(7) In terms of mass eigenstates, the new neutral current of neutral leptons can be rewritten as
\begin{aligned}[b] {\cal{L}}\supset & g'[\bar{\nu}_i (U_{\mu i}^{*}U_{\mu j}-U_{\tau i}^{*}U_{\tau j})\gamma^\mu P_L\nu_j \\ & +\bar{N}_i (\Omega_{\mu i}^{*}\Omega_{\mu j}-\Omega_{\tau i}^{*}\Omega_{\tau j})\gamma^\mu P_R N_j]Z'_\mu, \end{aligned}
(8) From Eqs. (6) and (8), it is obvious that the collider phenomenology of heavy neutral leptons depends on the mixing patterns, which makes the results model-dependent. In the following collider simulation, we consider a muon-philic heavy neutral lepton N, which couples exclusively to muons [54]. We also take the mixing parameter
\Omega_{\mu N}^{*}\Omega_{\mu N}-\Omega_{\tau N}^{*}\Omega_{\tau N}=1 in the following calculation, so the general model-dependent results can be obtained by a simple rescaling. -
In this section, we briefly summarize the constraints on the gauged
U(1)_{L_\mu-L_\tau} model. One important motivation of this symmetry is to explain the anomaly of the muon magnetic moment\Delta a_\mu=(251\pm59)\times10^{-11} [36]. The new gauge bosonZ' contributes to\Delta a_\mu at one loop level, which is evaluated as [37]\begin{aligned}[b] \Delta a_\mu=\frac{g^{\prime 2}}{4\pi^2}\int_0^1{\rm{d}}x\frac{m_\mu^2 x(1-x)^2}{m_\mu^2(1-x)^2+m_{Z'}^2 x}. \end{aligned}
(9) The viable parameter space to interpret
\Delta a_\mu is shown in Fig. 1. For very lightm_{Z'} , we have\Delta a_\mu\simeq g^{\prime 2}/8\pi^2 , which requiresg'\sim4\times10^{-4} to explain\Delta a_\mu . In the heavyZ' limit,\Delta a_\mu\simeq g^{\prime 2} m_\mu^2/12\pi^2 m_{Z'}^2 ; thus, the experimental discrepancy requiresg'/m_{Z'}\sim 5\times10^{-3}\; \text{GeV}^{-1} .For the gauge boson
Z' , one tight constraint is the dimuon production from the inelastic neutrino-nucleus scattering\nu_\mu N_u \to \nu_\mu N_u \mu^+\mu^- , whereN_u denotes the nucleus [60]. The CCFR experiment measured the neutrino trident production cross section, which is consistent with the standard model prediction as\sigma/\sigma_\text{SM}=0.82\pm0.28 [35]. This result has excluded the\Delta a_\mu favored region withm_{Z'}\gtrsim0.3 GeV. Recently, the NA64μ experiment has excludedg'\gtrsim6\times10^{-4} withm_{Z'}\lesssim0.1 GeV by the process of\mu N_u\to \mu N_u Z' \to \mu N_u \bar{\nu} \nu [56], which makesm_{Z'}\sim0.01 GeV the\Delta a_\mu -favored region. A more stringent constraint comes from thee^+e^-\to \mu^+\mu^- Z' \to 4\mu searches at BABAR in the mass region of0.212\; { \rm GeV}<m_{Z'}<10 GeV [57]. ForZ' mass in the range of [5, 81] GeV, the LHC experiment has excludedg'\gtrsim0.003 to 0.2 (depending onm_{Z'} ) by the processpp\to Z'\mu^+\mu^-\to4\mu with 139 fb^{-1} data [58]. Within the same mass region, a complement search by the processpp\to Z'\mu^\pm\nu\to 3\mu \nu with 140 fb^{-1} data was also performed [59], which has better sensitivity in the lightZ' region. In this paper, we consider the combined limit of the3\mu and4\mu channels at the LHC [59]. Meanwhile, forZ' lighter than approximately 6 MeV, it is disfavored by\Delta N_\text{eff} constraints from big-bang nucleosynthesis (BBN) [55], because decays ofZ' could heat the neutrino population and delay the process of neutrino decoupling. In the following study, we considerm_{Z'}\gtrsim 100 GeV to avoid these tight constraints.For the heavy neutral lepton N, the promising signature at the LHC is
pp\to W^{\pm(*)}\to \ell^\pm N , which depends on the mixing parameterV_{\ell N} . Whenm_N<m_W , N is produced from on-shell W boson decay; then, LHC could exclude|V_{\ell N}|^2\gtrsim2.9\times10^{-6} via the fully-leptonic decay channelN\to \ell \ell \nu [61]. Meanwhile, for light N with properV_{\ell N} , it becomes long-lived and leads to displaced vertex signature. Currently, the LHC has excluded|V_{\mu N}|^2 > 4\times10^{-7} form_N=10 GeV [62]. Abovem_W , production of N is mediated by off-shell W boson, so the LHC limit becomes much weaker, e.g.,|V_{\mu N}|^2=1.1\times10^{-3} form_N=100 GeV [61]. In this paper, we also considerm_N>100 GeV with seesaw-induced mixing|V_\ell N|^2\lesssim10^{-12} to satisfy current LHC limits. -
Before studying the explicit signatures, we first review the decay properties of new gauge boson
Z' and heavy neutral lepton N. The partial decay widths ofZ' are calculated as\Gamma(Z'\to \ell^+\ell^-) = \frac{g'^2}{12\pi}m_{Z'},
(10) \Gamma(Z'\to \nu_\ell \nu_\ell) = \frac{g'^2}{24\pi}m_{Z'},
(11) \Gamma(Z'\to N_\ell N_\ell ) = \frac{g'^2}{24\pi}m_{Z'}\left(1-4\frac{m_{N_\ell }^2}{m_{Z'}^2}\right),
(12) where
\ell=\mu,\tau , and we have assumed vanishing masses for\ell,\nu_\ell in the above calculations. For simplicity, we assume that the tree-level kinetic mixing between bosonZ' and photon γ is absent. However, mixing betweenZ' and γ in the form of-\dfrac{\epsilon}{2} Z'_{\mu\nu} F^{\mu\nu} appears at one-loop level, where the kinetic mixing factor\epsilon is calculated as [55]\begin{aligned}[b] \epsilon=-\frac{e g'}{12\pi^2}\log\left(\frac{m_\tau^2}{m_\mu^2}\right) \simeq - \frac{g'}{70}. \end{aligned}
(13) In this way,
Z' also couples to electron e via mixing with γ. TheZ'\to e^+e^- decay width is suppressed by the kinetic mixing factor\epsilon\simeq -g'/70 , so it is not taken into account in this study.After the production of heavy neutral lepton N, it decays via mixing with light neutrinos. When the heavy neutral lepton is heavier than the standard model gauge bosons, the two-body decays are the dominant modes. The partial decay widths are given by
\Gamma(N\to \ell^\pm W^\mp) = \frac{|V_{\ell N}|^2}{16\pi } \frac{(m_N^2-m_W^2)^2(m_N^2+2 m_W^2)}{m_N^3 v_0^2},
(14) \Gamma(N\to \nu Z) = \frac{|V_{\ell N}|^2}{32\pi } \frac{(m_N^2-m_Z^2)^2(m_N^2+2 m_Z^2)}{m_N^3 v_0^2},
(15) \Gamma(N\to \nu h) = \frac{|V_{\ell N}|^2}{32\pi } \frac{(m_N^2-m_h^2)^2}{m_N v_0^2 }.
(16) In Fig. 2, we show the branching ratio (BR) of gauge boson
Z' and heavy neutral lepton N. The dilepton modeZ'\to \mu^+\mu^-,\tau^+\tau^- is always the dominant decay channel ofZ' . Neglecting the final state's phase space effect, we have BR(Z'\to \nu_\ell \nu_\ell)= BR(Z'\to N_\ell N_\ell )=1/4 for two degenerate heavy neutral leptons. In the heavym_N limit, we have BR(N \to \ell^\pm W^\mp) : BR(N \to \nu Z) : BR(N \to \nu h)= 2:1:1 . To reconstruct the heavy neutral lepton mass, we focus on the decay modeN\to \ell^\pm W^\mp withW\to q \bar{q}' in the collider simulation. -
Through mixing with light neutrinos, the pair production of heavy neutral leptons at the muon collider can occur through s-channel Z exchange and t-channel W exchange. However, the production cross section is proportional to the fourth power of the mixing parameter
|V_{\ell N}|^4 . Provided natural seesaw predicted valueV_{\ell N}\lesssim 10^{-6} , the predicted cross section is negligible tiny.The heavy neutral lepton N is charged under
U(1)_{L_\mu-L_\tau} symmetry, which induces the pair production of heavy neutral lepton viaZ' at the muon collider. With fixed collision energy, the direct pair production process\mu^+\mu^-\to NN via the s-channel off-shellZ' is more promising whenm_{Z'}>\sqrt{s} . Meanwhile, whenm_{Z'}<\sqrt{s} , the new gauge bosonZ' can be produced on-shell with an associated photon from the initial legs as\mu^+\mu^-\to Z' \gamma \to NN \gamma , which also requiresm_N<m_{Z'}/2 to make the decayZ'\to NN kinematically allowed. -
We first consider the pair production of heavy neutral leptons without associated photon. The production cross section of
\mu^+\mu^-\to Z^{\prime *}\to NN with a center of mass energy\sqrt{s} is calculated as\begin{aligned}[b] & \sigma(\mu^+\mu^-\to Z^{\prime *}\to NN)\\ = & \frac{g^{\prime 4}}{24\pi}\frac{s}{(s-m_{Z'}^2)^2+m_{Z'}^2\Gamma_{Z'}^2}\left(1-4\,\frac{m_N^2}{s}\right)^{3/2}, \end{aligned}
(17) where
\Gamma_{Z'} is the total decay width ofZ' .The theoretical cross section of
\mu^+\mu^-\to Z^{\prime *}\to NN at a 3 TeV muon collider for specific scenarios is shown in Fig. 3, where we directly use Eq. (17) without considering the initial state radiation effect. It is obvious that whenm_{Z'}\simeq \sqrt{s} , the on-shell production ofZ' can greatly enhance the cross section. The maximum value is approximately12\pi \text{BR}(Z'\to \mu^+\mu^-) \text{BR}(Z'\to NN)/m_{Z'}^2\approx 3\pi/(8m_{Z'}^2) [45], which could be over 50 pb. For a lightZ' , the cross section reduces toFigure 3. (color online) Cross section of
\mu^+\mu^-\to Z^{\prime *}\to NN at a 3 TeV muon collider. We have fixedm_N=100 GeV in the left panel andm_{Z'}=2000 GeV in the right panel.\begin{aligned}[b] \sigma(\mu^+\mu^-\to Z^{\prime *}\to NN)\simeq \frac{g^{\prime 4}}{24\pi} \frac{1}{s} \left(1-4\,\frac{m_N^2}{s}\right)^{3/2}, \end{aligned}
(18) which is independent of
Z' mass. Typically, forg'=0.3 andm_N=1000 GeV, we have\sigma(\mu^+\mu^-\to Z^{\prime *}\to NN)\simeq4.7 fb. Meanwhile, a heavyZ' leads to\begin{aligned}[b] \sigma(\mu^+\mu^-\to Z^{\prime *}\to NN)\simeq \frac{g^{\prime 4}}{24\pi} \frac{s}{m_{Z'}^4} \left(1-4\,\frac{m_N^2}{s}\right)^{3/2}. \end{aligned}
(19) In this scenario, the cross section is suppressed by the large value of
m_{Z'} . While single production could probem_N\lesssim \sqrt{s} , the pair production of heavy neutral lepton is kinematically allowed whenm_N<\sqrt{s}/2 . Thus, the 3 TeV muon collider could only probem_N\lesssim1500 GeV. For heavierm_N , we require a more energetic muon collider. -
The photon from initial state radiation carries part of the collision energy, which results in the gauge boson
Z' produced on-shell whenm_{Z'}\lesssim \sqrt{s} . In this case, there is a resonance peak in the invariant mass spectrum of the dilepton [44]. Here, we study the pair production of heavy neutral leptons with associated photon process as\begin{aligned}[b] \mu^+\mu^-\to Z^{\prime (*)}\gamma\to NN\gamma. \end{aligned}
(20) Here, the scenario with off-shell
Z' contribution is also considered whenm_{Z'}\leqslant 2 m_N orm_{Z'}\geqslant \sqrt{s} .In Fig. 4, we show the cross section of
\mu^+\mu^-\to NN\gamma at a 3 TeV muon collider, where the numerical results are calculated by MadGraph. There is also a sharp peak aroundm_{Z'}\simeq \sqrt{s} . To avoid the soft photon singularity, the detected photon is required to satisfy the following pre-selection cuts:Figure 4. (color online) Cross section of
\mu^+\mu^-\to NN\gamma at a 3 TeV muon collider. We have fixedm_N=100 GeV in the left panel andm_{Z'}=2000 GeV in the right panel.\begin{aligned}[b] P_T(\gamma)>20\; \text{GeV},\; |\eta(\gamma)|<2.5. \end{aligned}
(21) When
m_{Z'}>2 m_N , the on-shell production ofZ' followed by the cascade decayZ'\to NN can notably enhance the cross section of\mu^+\mu^-\to NN\gamma . Using the narrow-width approximation, the cross section can be expressed as [63]\begin{aligned}[b] \sigma(\mu^+\mu^-\to NN\gamma)\simeq \sigma(\mu^+\mu^-\to Z'\gamma) \times \text{BR}(Z'\to NN). \end{aligned}
(22) Because the branching ratio of
Z'\to NN is independent of the new gauge couplingg' , the cross section of\mu^+\mu^-\to Z'\gamma\to NN\gamma is proportional toe^2g^{\prime 2} , while the cross section of the off-shell process\mu^+\mu^-\to Z ^{\prime (*)}\gamma \to NN\gamma is proportional toe^2g^{\prime 4} . Thus, we observe that the enhancement effect becomes larger when the gauge couplingg' is smaller. Forg'=0.1,\; m_N=100 GeV, andm_{Z'}=1000 GeV, we have\sigma(\mu^+\mu^-\to Z'\gamma\to NN\gamma)\sim 1 fb, while\sigma(\mu^+\mu^-\to Z^{\prime *}\to NN) is less than 0.1 fb; therefore, the former process is expected to be more promising at the muon collider. -
Pair production of heavy neutral leptons followed by various cascade decay modes could lead to many interesting signatures, such as monolepton, dilepton, and trilepton signals [64, 65]. In this paper, we focus on the most distinct lepton number violation, same-sign dimuon signal, which also has a much cleaner background compared with the lepton number conserving one.
The simulation procedure is as follows. The FeynRules package [66] is used to implement the model. The parton level events are simulated with MadGraph5_aMC@NLO [67]. We then use PYTHIA 8 [68] for parton shower. The detector effects are included by employing Delphes 3 [69] with the detector card of the muon collider. The W boson from the heavy neutral lepton decay could be highly boosted, so the two jets from cascade W decay merge into one fat-jet J. We use the Valencia algorithm [70] with
R=1.2 to reconstruct the fat-jets. -
The full production process of the same-sign dimuon signature without associated photon is
\begin{aligned}[b] \mu^+\mu^-\to Z^{\prime *}\to N N\to \mu^\pm W^\mp +\mu^\pm W^\mp, \end{aligned}
(23) with the hadronic decays of W. The standard model backgrounds come from processes such as
\begin{aligned}[b] \mu^+\mu^-\to \mu^+\mu^- W^+W^- , \mu^+\mu^-W^+ W^-\gamma/Z, W^\pm W^\pm W^\mp \mu^\mp \nu. \end{aligned}
(24) The contributions of
\mu^+\mu^- W^+W^- and\mu^+\mu^-W^+ W^-\gamma/Z are from lepton charge misidentification, which are suppressed by the misidentification rate 0.1% [17]. TheW^\pm W^\pm W^\mp \mu^\mp \nu is dominant by the vector boson fusion process, where the two same-sign W bosons decay leptonically. There is also one possible background process\mu^+\mu^-\to W^+W^-jj with the light jets mistagged. However, by requiring the light jet masses close to the W mass, this background can easily be suppressed [17]. Meanwhile, thet\bar{t}W^\pm \mu^\mp\nu process also contributes to the same-sign dimuon signature, which can be further reduced by cut on the opposite-sign lepton [26]. Thus, we do not include contributions of the\mu^+\mu^-\to W^+W^- jj andt\bar{t}W^\pm \mu^\mp\nu processes in this paper.We first apply the following pre-selection cuts on the transverse momentum and pseudorapidity of the muon and fat-jets:
\begin{aligned}[b] P_T (\mu^\pm)>50\; \text{GeV}, |\eta (\mu^\pm)|<2.5,P_T (J)>50\; \text{GeV}, |\eta (J)|<2.5. \end{aligned}
(25) In Fig. 5, we show the distributions of some variables for the
\mu^\pm\mu^\pm JJ signal and backgrounds after applying the pre-selection cuts. We have setm_{N}=1000 GeV,m_{Z'}= 2500 GeV, andg'=0.6 as the benchmark point of the signal. The cut flow for the\mu^\pm\mu^\pm JJ signature and backgrounds are summarized in Table 1.Figure 5. (color online) Normalized distribution of transverse momentum of muon
P_T(\mu^\pm) (up-left panel), transverse momentum of fat-jetP_T(J) (up-right panel), fat-jet massm_J (down-left panel), and invariant mass of muon and fat-jetm_{\mu J} (down-right panel) for the\mu^\pm\mu^\pm JJ signature and corresponding backgrounds.σ(fb) \mu^\pm\mu^\pm W^\mp W^\mp \mu^+\mu^- W^+W^- \mu^+\mu^-W^+ W^-\gamma/Z W^\pm W^\pm W^\mp \mu^\mp \nu Pre-selection 68.19 21.57 2.13 3.11 N_{\mu^\pm}=2 41.38 2.6\times10^{-2} 2.5\times10^{-3} 1.8\times10^{-1} N_J=2 24.87 1.1\times10^{-2} 1.3\times10^{-3} 7.1\times10^{-2} 50 GeV \leqslant m_J\leqslant 100 GeV14.41 3.2\times10^{-3} 2.1\times10^{-4} 2.0\times10^{-2} 0.8 m_N<m_{\mu J}<1.2 m_N 13.62 3.4\times10^{-4} 2.7\times10^{-5} 8.1\times10^{-4} Significance 476.6 Total Background 1.2\times10^{-3} Table 1. Cut flow table for the
\mu^\pm\mu^\pm JJ signal at the 3 TeV muon collider and corresponding backgrounds. The significance is calculated with Eq. (29) by assuming an integrated luminosity of 1000 fb^{-1} .The lepton number violation signature is satisfied for events with two same-sign muons. In the single production of the heavy neutral lepton process [26, 27], only one boosted W is expected from N decay. To distinguish from the single production and also suppress the background from vector boson fusion processes, we require exactly two fat-jets in the final states
\begin{aligned}[b] N_{\mu^\pm}=2, \; N_J=2, \end{aligned}
(26) although tagging one fat-jet has greater efficiency for the signal. To be identified as W bosons, the fat-jet masses are also required in the following range:
\begin{aligned}[b] 50 \; \text{GeV} \leqslant m_J \leqslant 100 \; \text{GeV}. \end{aligned}
(27) Because there are two heavy neutral leptons in the signal, we reconstruct their masses through the two-muon and two-fat-jet system by minimizing
\chi^2= (m_{\mu_1 J_1} - m_N)^2+(m_{\mu_2 J_2} -m_N)^2 [17]. We require that both the invariant mass of muon and fat-jetm_{\mu J} from reconstructed N satisfy\begin{aligned}[b] 0.8 m_N<m_{\mu J}<1.2 m_N. \end{aligned}
(28) According to the distributions in Fig. 5, we may tighten the cuts on
P_T(\mu^\pm) andP_T(J) to suppress the background. However, we can see from Table 1 that the cut on the invariant massm_{\mu J} is relatively efficient to reduce the background. After applying all these cuts, the total cross section of the background is approximately1.2\times10^{-3} fb. Thus, with an integrated luminosity of 1000 fb^{-1} , there are only 1.2 background events, while the signal events are over10^4 . In this manner, the significance will be 476.6 for the benchmark point with 1000 fb^{-1} data, where the significance is calculated as [71]\begin{aligned}[b] {\cal{S}}=\sqrt{2\left[(N_S+N_B)\ln\left(1+\frac{N_S}{N_B}\right)-N_S\right]}. \end{aligned}
(29) Here,
N_S andN_B are the event numbers of signal and background, respectively. To reach the5\sigma discovery limit, we require only 0.11 fb^{-1} data.Based on the above analysis, we then explore the sensitivity of the
\mu^\pm \mu^\pm JJ signature at the 3 TeV muon collider. The results are shown in Fig. 6. There are three free parametersg',m_{Z'} , andm_N related to the collider phenomenology. With fixed mass relationm_{N} = m_{Z'}/3 , we could probem_{Z'} approximately in the range of [300, 4500] GeV. Around the resonance regionm_{Z'}\sim3000 GeV, the gauge couplingg' could be down to approximately 0.025 with 1000 fb^{-1} data. We also find that form_N\lesssim300 GeV, the cut efficiency decreases quickly asm_N becomes smaller. One main reason for this is that the heavy neutral leptons are also highly boosted at the 3 TeV muon collider for such lightm_N , so the muons and W-jets from boosted N decays are mostly non-isolated [72]. One may use the substructure-based variables as lepton sub-jet fraction (LSF) and lepton mass drop (LMD) [73] to probe the lightm_N , which is beyond the scope of this study.Figure 6. (color online) Sensitivity of the
\mu^\pm \mu^\pm JJ signature at the 3 TeV muon collider. In the upper-left panel, we fix the mass relation withm_N=m_{Z'}/3 . In the upper-right panel, we setm_N=1000 GeV. In the lower panel, we fix the gauge couplingg'=0.6 . The dark-green region corresponds to sensitivity with 1 fb^{-1} data, while the light-green region corresponds to sensitivity with 1000 fb^{-1} data. The orange region is excluded by neutrino trident production at CCFR [35]. The magenta region is not allowed kinematically. The gray line is the projected sensitivity at the HL-LHC [74]. The red line is the combined sensitivity of the 3 TeV muon collider via processes\mu^+\mu^-\to \ell^+\ell^-(\ell=\mu\; \text{or}\; \tau), \ell\bar{\ell}\gamma(\ell=\mu\; \text{or}\; \tau\; \text{or}\; \nu_{\mu/\tau}) [44].For pair production of heavy neutral leptons without associated photon signature, the gauge boson
Z' does not need to be heavier than2 m_N . In the upper-right panel of Fig. 6, we have fixedm_N=1000 GeV for illustration. The production cross section of\mu^+\mu^-\to NN is approximately constant for lightZ' , sog'\sim0.13 is nearly independent ofm_{Z'} whenZ' is lighter than 1 TeV. This channel is most sensitive for the largem_{Z'} region. For instance, we may probem_{Z'}\sim10 TeV withg'\gtrsim0.4 andm_N\sim1000 GeV at the 3 TeV muon collider. Near the resonance region withm_{Z'}\sim \sqrt{s} , we can even have a promising signature wheng'\gtrsim0.14 with only 1 fb^{-1} data. Compared with the direct dilepton channels\mu^+\mu^-\to \ell^+\ell^-(\ell=\mu\; \text{or}\; \tau), \; \ell\bar{\ell}\gamma(\ell=\mu\; \text{or}\; \tau\; \text{or}\; \nu_{\mu/\tau})
the sensitivity region of lepton number violation signal from heavy neutral lepton pair at the muon collider is smaller due to lower tagging efficiency of N.
We then fix the gauge coupling
g'=0.6 and explore the sensitivity region on them_N-m_{Z'} plane. With 1 fb^{-1} data, a large parameter space near the resonance region with 2 TeV\lesssim m_{Z'}\lesssim4 TeV can be probed. The most sensitive heavy neutral lepton mass is around 500 GeV. This is because a lighterm_N leads to smaller acceptance efficiency, while a largerm_N has a smaller production cross section. With an integrated luminosity of 1000 fb^{-1} , the region withm_{Z'}\lesssim 13 TeV and 150 GeV\lesssim m_N<1500 GeV is within the reach of the 3 TeV muon collider. -
The full production process of the same-sign dimuon signature with associated photon is
\begin{aligned}[b] \mu^+\mu^-\to Z^{\prime (*)} \gamma \to N N \gamma\to \mu^\pm W^\mp +\mu^\pm W^\mp+\gamma, \end{aligned}
(30) with the hadronic decays of W. The standard model backgrounds come from processes such as
\begin{aligned}[b] \mu^+\mu^-\to \mu^+\mu^- W^+W^-\gamma , W^\pm W^\pm W^\mp \gamma \mu^\mp \nu . \end{aligned}
(31) During the simulation, the pre-selection cuts on associated photon in Eq. (21) are also applied. Based on the cut-flow in Table 1, we expect that the cross section of the process
\mu^+\mu^-W^+ W^-\gamma\gamma/Z with one additional\gamma/Z gauge boson is approximately an order of magnitude smaller than that of the process\mu^+\mu^- W^+W^-\gamma . Therefore,\sigma(\mu^+\mu^-W^+ W^-\gamma\gamma/Z) is approximately10^{-5} fb after all cuts, which will not be considered in the analysis of the\mu^\pm\mu^\pm JJ\gamma signal.The cross sections of the background processes with associated photons are approximately one order of magnitude smaller than those without associated photons. If we apply exactly the same selection cuts as the previous
\mu^\pm\mu^\pm JJ signal, the total cross section of background is expected to be of the order of10^{-4} fb, which will have approximately 0.1 background events, even with a total of 1000 fb^{-1} data. Therefore, we can loosen the selection cut to keep more signal events. In the analysis of the\mu^\pm\mu^\pm JJ\gamma signal, we consider that at least one fat-jet is detected in the final states\begin{aligned}[b] N_J\geqslant 1, \end{aligned}
(32) while we further require one detected photon in this signal.
In Fig. 7, we show the distributions of some variables for the
\mu^\pm\mu^\pm JJ\gamma signal and backgrounds after applying the pre-selection cuts. The benchmark point for the\mu^\pm\mu^\pm JJ\gamma signal is the same as for the previous\mu^\pm\mu^\pm JJ signal. With relatively similar distributions ofP_T(\mu^\pm) ,P_T(J) ,m_J , andm_{\mu J} , we then apply the same selection cuts for these variables as the\mu^\pm\mu^\pm JJ signal. The cut flows for the\mu^\pm\mu^\pm JJ\gamma signature and backgrounds are summarized in Table 2.Figure 7. (color online) Normalized distribution of transverse momentum of muon
P_T(\mu^\pm) (upper-left panel), transverse momentum of fat-jetP_T(J) (upper-right panel), fat-jet massm_J (middle-left panel), and invariant mass of muon and fat-jetm_{\mu J} (middle-right panel). Energy of photonE_\gamma (lower-left panel) and invariant mass of reconstructed heavy neutral lepton pairm_{NN} (lower-right panel) for the\mu^\pm\mu^\pm JJ \gamma signature and corresponding backgrounds.σ/fb \mu^\pm\mu^\pm W^\mp W^\mp \gamma \mu^+\mu^- W^+W^- \gamma W^\pm W^\pm W^\mp\gamma \mu^\mp \nu Pre-selection 24.91 1.15 0.14 N_{\gamma}=1,N_{\mu^\pm}=2 13.28 1.1\times10^{-3} 7.4\times10^{-3} N_J\geq1 13.25 1.0\times10^{-3} 7.2\times10^{-3} 50 GeV \leqslant m_J\leqslant 100 GeV9.61 5.7\times10^{-4} 4.5\times10^{-3} 0.8 m_N<m_{\mu J}<1.2 m_N 8.86 2.6\times10^{-4} 1.2\times10^{-3} Significance 369.0 Total Background 1.5\times10^{-3} Table 2. Cut flow table for the
\mu^\pm\mu^\pm JJ\gamma signal at the 3 TeV muon collider and corresponding backgrounds. The significance is calculated by Eq. (29), assuming an integrated luminosity of 1000 fb^{-1} .Although with one additional photon, the cross section of the
\mu^\pm\mu^\pm JJ\gamma signal is of the same order as the\mu^\pm\mu^\pm JJ signal. After all selection cuts, the cross section of the\mu^\pm\mu^\pm JJ\gamma process is 8.86 fb. With a loose cut on the fat-jet number, the total cross section of the background is approximately1.5\times10^{-3} fb after all cuts. With an integrated luminosity of 1000 fb^{-1} , the significance of the\mu^\pm\mu^\pm JJ\gamma signal could be approximately 369 for the benchmark point. Meanwhile, 0.19 fb^{-1} data is sufficient to discover the benchmark point at the5\sigma level.Besides the invariant mass of dilepton
m_{\ell^+\ell^-}(\ell=\mu,\tau) , there are two more pathways to confirm the on-shell production of gauge bosonZ' in the\mu^\pm \mu^\pm JJ\gamma signature. For the two-body process\mu^+\mu^-\to Z' \gamma , the photon energy is connected to the gauge boson mass as [44]\begin{aligned}[b] E_\gamma=\frac{s-m_{Z'}^2}{2\sqrt{s}}. \end{aligned}
(33) For the benchmark point with
m_{Z'}=2500 GeV at the 3 TeV muon collider, the peak value ofE_\gamma is approximately 458 GeV. The sharp bump above the background is clearly shown in the lower-left panel of Fig. 7. Meanwhile, we can also measurem_{Z'} through the invariant mass of two reconstructed heavy neutral leptonsm_{NN} , which also has a resonance nearm_{Z'} . From the distribution ofm_{NN} in the lower-right panel of Fig. 7, the background processes\mu^+\mu^-W^+W^-\gamma andW^\pm W^\pm W^\mp \gamma \mu^\mp \nu have peaks around 3 TeV and 1.5 TeV, respectively. Besides, with less than two background events after all cuts, the backgrounds have little impact on the distribution ofm_{NN} for the signal.The sensitivity of the
\mu^\pm \mu^\pm JJ\gamma signature at the 3 TeV muon collider is explored with the cuts in Table 2, and Fig. 8 depicts the results. The upper-left panel shows the sensitivity region with fixed mass relationm_N=m_{Z'}/3 , so the decay channelZ'\to NN is always allowed. In this way, we might probem_{Z'} in the range of [300, 4300] GeV. Compared with the\mu^\pm \mu^\pm JJ signature, we find that this signal is more promising approximately in the region of [400, 3000] GeV. For example, withm_{Z'}=2000 GeV, the\mu^\pm \mu^\pm JJ\gamma signature could probeg'\gtrsim0.015 , while the\mu^\pm \mu^\pm JJ signature could only reachg'\sim0.1 . Nearm_{Z'}\sim \sqrt{s}=3 TeV, we can even revealg'\gtrsim0.005 . Whenm_{Z'} is larger than the collision energy\sqrt{s} , the associated photon with the off-shell contribution ofZ' makes the\mu^\pm \mu^\pm JJ\gamma signature less detectable than the\mu^\pm \mu^\pm JJ signal.Figure 8. (color online) Same as Figure 6 but for the sensitivity of the
\mu^+\mu^-JJ\gamma signal.The upper-right panel of Fig. 8 shows the sensitivity region with fixed
m_N=1000 GeV. Whenm_{Z'}\leqslant 2 m_N , the decay channelZ'\to NN is not allowed, so production ofNN\gamma is through the off-shellZ' . With nearly a constant cross section, the\mu^\pm \mu^\pm JJ\gamma signature could testg'\gtrsim0.23 form_{Z'}\leqslant 2000 GeV, which is clearly less promising than the\mu^\pm \mu^\pm JJ signature. However, once the decay modeZ'\to NN is open, the cross section ofNN\gamma is enhanced, so the\mu^\pm \mu^\pm JJ\gamma signal becomes quite promising. According to our simulation, we find that this signature could probeg'\gtrsim0.11 with only 1 fb^{-1} data when 2000 GeV\lesssim m_{Z'}\lesssim\sqrt{s} .The sensitivity region in
m_N-m_{Z'} with fixedg'=0.6 is shown in the lower panel of Fig. 8. With 1 fb^{-1} data, a large portion of the parameter space with2m_N\lesssim m_{Z'}\lesssim \sqrt{s} could be tested. With a similar reason as the\mu^\pm \mu^\pm JJ signature, the most sensitivem_N of the\mu^\pm \mu^\pm JJ\gamma signature is also approximately 500 GeV. Outside the above resonanceZ' region, the significance of the\mu^\pm \mu^\pm JJ\gamma signature decreases due to the associated photon. Although less promising than the\mu^\pm \mu^\pm JJ signature, the\mu^\pm \mu^\pm JJ\gamma signal could probe the region withm_{Z'}\lesssim8.3 TeV and 120 GeV\lesssim m_N\lesssim1470 GeV at the 3 TeV muon collider with an integrated luminosity of 1000 fb^{-1} . -
Heavy neutral leptons N are introduced in the type-I seesaw mechanism to generate tiny neutrino masses. While electroweak scale N is extensively studied at colliders, a relatively large mixing parameter
V_{\ell N} is usually required. In this paper, we considered the gaugedU(1)_{L_\mu-L_\tau} extension of the type-I seesaw mechanism, where three heavy neutral leptonsN_e,N_\nu,N_\tau withU(1)_{L_\mu-L_\tau} charge(0,1,-1) were introduced. Charged under the new gaugedU(1)_{L_\mu-L_\tau} symmetry, there are new production channels of the heavy neutral lepton at colliders. We then investigated the lepton number violation signatures of heavy neutral lepton N in the gaugedU(1)_{L_\mu-L_\tau} at the 3 TeV muon collider.Mediated by the new gauge boson
Z' , the heavy neutral lepton N can be pair produced at the muon collider via the processes\mu^+\mu^-\to Z^{\prime *}\to NN and\mu^+\mu^-\to Z^{\prime (*)}\gamma\to NN \gamma . Cross sections of these processes are determined by the new gauge couplingg' , gauge boson massm_{Z'} , and heavy neutral lepton massm_N , which are independent of the mixing parameterV_{\ell N} . While both processes have a maximum cross section atm_{Z'}\simeq \sqrt{s} , the former is more promising whenm_{Z'}>\sqrt{s} . For lighter new gauge bosons satisfying2m_N<m_{Z'}<\sqrt{s} ,Z' is produced on-shell with an associated photon, and the decay modeZ'\to NN is also allowed. In this way, the cross section of\mu^+\mu^-\to NN\gamma can be enhanced.Cascade decays of heavy neutral leptons can raise various interesting signatures. Provided the Majorana nature of heavy neutral leptons in the seesaw model, we focus on the lepton number violation signatures in this study. For illustration, we further assume that the heavy neutral lepton N preferentially couples to muons via mixing with light neutrinos. Thus, the dominant decay mode of heavy neutral leptons is
N\to \mu^\pm W^\mp . The hadronic decays of W bosons are considered to reconstructm_N , which are treated as one fat-jet J. The explicit signatures are\mu^+\mu^-\to NN\to \mu^\pm\mu^\pm JJ and\mu^+\mu^-\to NN\gamma\to \mu^\pm\mu^\pm JJ\gamma .With relatively clean backgrounds, a cut-based analysis was then performed, which indicates that a large part of the viable parameter space is within the reach of the 3 TeV muon collider. For example,
\mu^\pm\mu^\pm JJ could probem_{Z'} in the range of [300, 4500] GeV with fixed mass relationm_{N}=m_{Z'}/3 and an integrated luminosity of 1000 fb^{-1} . Around the resonance regionm_{Z'}\sim\sqrt{s} , the gauge couplingg' could be down to approximately 0.025. We can probeg'\gtrsim0.13 withm_N=1000 GeV and lightZ' . Otherwise, this signal could probem_{Z'}\lesssim 13 TeV withg'=0.6 . Meanwhile, if the gauge boson mass satisfies2 m_N<m_{Z'}<\sqrt{s} , the\mu^\pm\mu^\pm JJ\gamma signature would be more promising than the\mu^\pm\mu^\pm JJ signature. It is also notable that around the resonance regionm_{Z'}\sim\sqrt{s} , both signatures could probe a sizable parameter space even with only 1 fb^{-1} data forg'=0.6 .
