Search for heavy vector-like B quark via pair production in fully hadronic channels at CLIC

Extra quarks are expected to address fundamental standard model (SM) issues, such as electroweak symmetry breaking and the mass generation of fundamental particles in new physics models [1−11]. However, a fourth generation of SM-like quarks suffer serious electroweak precision and Higgs search constrains [12−17]. Unlike conventional SM quarks and those of the sequential fourth generation, the left- and right-handed chiral components of the vector-like quarks (VLQs) transform under similar SM gauge group representation. Hence, VLQs masses do not arise from interactions with the Higgs field and are not excluded by Higgs-related measurements at the LHC.

Vector-like quarks (VLQs) are itroduced in new physics scenarios to address certain SM issues [18, 19]. For instance, VLQs are anomaly-free and can naturally stabilize the Higgs boson mass through their loop effects. They enabling realistic models with spontaneous CP violation [20] and provide an alternative solution to the challenging CP problem without requiring axions [21−24]. Additionally, VLQs may contribute to the unification of the coupling constants [25−27] and provide the simplest solution to the CKM unitarity problem [28−32]. Depending on their charge assignments under the SM electroweak gauge group, they can be electroweak singlets [T,B], doublets [(X,T), (T,B) or (B,Y)] or triplets [[X,T,B] or [T,B,Y]]. The vector-like top partner T (VLQ-T) and bottom quark B (VLQ-B) are key components in new physics models, addressing theoretical challenges while generating distinctive collider signatures. The VLQ-T naturally cancels dangerous quadratic divergences in the Higgs boson mass corrections and provides a mechanism to interpret the large mass of top quark [33, 34], thereby establishing it as a key element in numerous scenarios. The VLQ-B is generally predicted in grand unification theory [25−27] and certain composite models [18]. As regards single or pair-produced VLQs, ATLAS and CMS have conducted extensive studies (For recent reviews, see Refs. [35, 36]). Although no evidence has been found, these studies have set lower mass bounds above the TeV [35−38].

Current experimental studies have considered VLQ decays into SM particles. For example, the ATLAS collaboration conducted a serach for pair-produced VLQs, wherein at least one decays into a leptonically decaying Z boson and a third-generation quark using the full Run 2 dataset corresponding to 139 fb^–1 [38]. Considering only three possible decay modes of $ B\rightarrow tW, bh, bZ $, the lower mass limits of VLQ-B (with charge –1/3) are 1.20 TeV and 1.32 TeV for the weak-isospin singlet and doublet models, respectively [38].

VLQs can decay into new channels in numerous new physics scenarios that predict additional scalars. Recently, exotic decays of VLQs in different set-ups with rich collider signatures have been investigated [29, 39−56]. These new channels reduce the branching ratios (BRs) into SM final states, thereby alleviating current mass bounds [57]. Compared to the LHC, the clean collision environment and advanced detector techniques at future lepton colliders enable VLQ detection or provide complementary constraints on them. The presence of VLQs modifies Higgs coupling vertices and affects Higgs production and decay through loop processes. Although the CEPC lacks sufficient collision energy to directly produce TeV-scale VLQs, its high-precision measurements of Higgs boson couplings can constrain the VLQ parameter space or provide of their existence via deviations at the precision frontier [58−61]. Instead of probing VLQs indirectly, the compact linear collider (CLIC) could produce the VLQs directly. The CLIC is a proposed high-luminosity, TeV-scale linear electron-positron collider which holds promise for new physics detecting [62, 63]. Recently, studies on VLQs at the CLIC have garnered attention, including studies on pair [64, 65] and single production [66−74].

In this study, we investigate TeV-scale VLQ-B quark via pair production at the CLIC with the center-of-mass of 3 TeV, focusing on the$ B\rightarrow bZ $ and $ B\rightarrow bh $ decay channels. Using the singlet VLQ-B as a benchmark, we focus on the fully hadronic decays of the Z and h bosons, which offer large branching ratios to compensate for the relatively low production rate. The large mass of VLQ-B results in highly boosted Z and h bosons, which tend to appear as fat-jets. The clean collision environment and low background rates for large-mass fat-jets at future lepton colliders make this channel highly promising.

The paper is organized as follows: Section II summarizes the interactions and decay modes of the singlet vector-like B quark. In Section III, we present a detailed analysis of the detection probability for vector-like B produced via the process $ e^+e^- \rightarrow B\bar{B} $, followed by $ B(\bar{B})\rightarrow bZ $ or bh, with subsequent hadron decays of the Z and h bosons at CLIC. Finally, conclusions and discussions are provided in Section IV.

The VLQ-B appear in numerous new physics scenarios. Restricting to their couplings with third generation SM quarks, the effective Lagrangian for the singlet VLQ-B is expressed as follows [33]:

where g represents the $ S U(2)_L $ gauge coupling constant and $ \theta_{\rm W} $ the Weinberg angle. The VLQ-B mass $ m_B $ and relative coupling strength $ \kappa_B $ denote free parameters.

For a heavy weak-isospin singlet VLQ-B, it is assumed to decay into a third-generation quark and either a $ W/Z $ or Higgs boson. The relationship of the BRs of the three standard decay modes is expressed as follows:

This relationship provides an excellent approximation based on the Goldstone boson equivalence theorem [75−79].

VLQ investigations at the LHC assume only three standard decay modes with the corresponding BR relationship or focus on a specific decay channel with a 100% BR. However, the VLQs can naturally decay into light particles in new physics scenarios, such as $ B\rightarrow bS $, where S denotes an additional scalar or pseudoscalar particle [55, 56]. With the introduction of new exotic decay modes $ B\rightarrow X $, the BR relationships can be expressed as follows:

where $ \beta_{\rm new} $ represents the BR of the new exotic decay channels. In such scenario, the BRs of the standard modes may be reduced, thereby weakening the LHC constraints on VLQ-B. In the subsequent section, we present the $ B\rightarrow bZ(bh) $ decay modes under the assumption that exotic decays exist.

At the 3 TeV CLIC, VLQ-B pairs can be produced via $ \gamma $ or Z mediation. For a VLQ-B with mass 1 (1.4) TeV, the cross section $ e^+ e^- \rightarrow B \bar{B} $ can reach 5.423 (3.075) fb at the CLIC with $ \sqrt{s}=3 $ TeV. Unlike the QCD-induced production at the LHC, the production rate of VLQ-B is modest. However, the low background events, clean collision environment, and rapid development of detector technologies make CLIC a promising tool for detecting VLQ-B signatures.

Using the $ B\rightarrow bh $ and $ B\rightarrow bZ $ decay modes, coupled with the fully hadronic decays of the Z and h bosons, we assess the detection of VLQ-B. In these channels, the large hadronic BR can compensate for the low production rate. The large mass of VLQ-B at the TeV scale will induce highly boosted bosons, while the hadronical decay products of highly boosted bosons Z or h are more likely to form a fat-jet than two isolated jets. To effectively capture these decay products, a large jet radius R is employed. The Feynman diagram for the processes is shown in Fig. 1. At the LHC, jet substructure techniques - by analyzing internal jet features - effectively distinguish signal jets (originating from boosted massive particles) from background jets (typically QCD jets) [80]. Over the past two decades, numerous substructure methods have been developed. Early prong-finder algorithms [80], including BDRS algorithm [81], JH TopTagger [82], and N-subjettiness [83], identify multiple hard cores in a jet to differentiate the signals from common QCD jets. Studies have shown that investigating VLQ-B in fully hadronic channels using a large R and jet substructure technique can redue combinatorics problems and efficiently suppress backgrounds at the LHC [84, 85].

Figure 1.  Feynman diagrams for VLQ-B pair production at the CLIC with subsequent decay into bh.

In this study, the clean environment and rare boosted fat-jet backgrounds enable VLQ-B detection in the fully hadronic channels. Our analysis utilizes large-R jets to capture high-$ p_T $ mass jets without employing jet substructure algorithms, as the low production rate of the signal and internal jet structure resemble those of the background. In this section, we conduct Monte Carlo simulation and phenomenological analyses of the boosted h and Z channels, respectively. The parameter $ \kappa_B=0.1 $ and $ {\rm Br}(B\rightarrow bh)= {\rm Br} (B\rightarrow bZ)=0.25 $ used during calculation.

In our Monte Carlo simulation, parton-level events for the signals and SM backgrounds are generated using MadGraph5_aMC_v3.3.2 [86], followed by fragmentation and showering using Pythia8 [87]. Subsequently, all event samples are fed into Delphes3 [88] program with CLIC Delphes card [89] to simulate detector effects. In our analysis, jets are clustered using the FASTJET package [90]. The Valencia linear collider (VLC) algorithm is a sequential jet reconstruction method designed for future lepton colliders. Unlike conventional hadron collider algorithms that utilize the [transverse momentum, rapidity] basis, the VLC algorithm adopts the [energy, polar angle] basis as inter-particle distance [91, 92]. This distinctive approach provides enhanced robustness against background at future lepton colliders. In this study, the VLC algorithm is employed with a large jet radius parameter to capture fat-jets. Finally, a cut-based analysis is performed using MadAnalysis 5 [93].

In this subsection,we analyze the signal and background events at the 3-TeV CLIC using the $ e^+e^-\rightarrow B\bar{B} \rightarrow bh\bar{b}h\rightarrow J_hJ_hb\bar{b} $ process, where $ J_h $ denotes the boosted fat-jet from higgs decay. The highly boosted higgs jets produced from the decay of TeV-scale VLQ-B can be captured using a large jet radius R=1.0.

The dominant SM backgrounds primarily arise from the following processes:

● $ e^+ e^- \rightarrow W^+ W^- Z $ with $ Z/W^{\pm} \rightarrow j j $.

● $ e^+ e^- \rightarrow W^+ W^- h $ with $ W^{\pm} \rightarrow j j $ and $ h \rightarrow b \bar{b} $.

● $ e^+ e^- \rightarrow t \bar{t} $ with $ t(\bar{t}) \rightarrow W^{\pm} b $ and $ W^{\pm} \rightarrow j j $.

● $ e^+ e^- \rightarrow t \bar{t} h $ with $ t(\bar{t}) \rightarrow W ^{\pm}b $, $ W^{\pm} \rightarrow j j $ and $ h \rightarrow b \bar{b} $.

In the following analyses, b-tagging is omitted, as it is ineffective in discriminating against backgrounds because numerous background processes contain b-jets.

To simulate detector acceptance and ensure effective triggering, we choose the basic cuts for the signals and SM backgrounds as follows:

where $ P_T(j) $ and $ \eta(j) $ represent the transverse momentum and pseudo-rapidity of jets, respectively. In signal events, the heavy mass of VLQ-B is transferred to the large transverse momenta of its decay products, b and h. After selecting events with more than 4 jets in the final state, we resort the final jets by mass and present the normalized kinematic distributions for the signal and backgrounds as shown in Fig. 2. For the signal, the hadronic decay products of the boosted higgs bosons are captured by the large jet radius R=1.0. As shown in the figure, a large fraction of the signal events exhibit two massive fat-jets with masses near the Higgs mass $ m_h $ and two isolated low-mass jets. While some backgrounds events contain boosted fat-jets owing to the high collision energy at the CLIC, their mass distributions differ considerably. Notably, it is rare for backgrounds to feature two massive jets with masses around the Higgs mass. The first massive jet of $ t\bar{t} $ and $ t\bar{t}h $ backgrounds are always a top-jet, and only a tiny fraction of $ t\bar{t} $ and $ t\bar{t}h $ events can mimic two Higgs-jets. In addition, we show the normalized kinematic distributions for transverse momentum and rapidity in Fig. 3. The transverse momentum of the massive jets of the signal peaks around $ m_B/2 $ owing to kinematics, whereas the background exhibits a broad distribution. Furthermore, the WWZ background exhibits distict features, with numerous events containing jets of high rapidity and low $ P_T $.

Figure 2.  (color online) The normalized distributions for the mass of jets for the signal in boosted higgs channel and backgrounds.

To extract the signal from the backgrounds, a set of optimized cuts are applied.

● Cut-1: At least four isolated jets are required, and the scalar sum of all transverse momenta of jets must exceed 600 GeV, i.e, $ (N(j) \geq 4) $ and $ H_T > $600.

● Cut-2: The first two massive jets have masses in the Higgs-mass window ( $ 100<M_{j_{1,2}}<150 $). Additionally, there must be at least two light jets with masses $ M_{j_{3,4}}<70 $.

● Cut-3: The combined reconstructed mass of each massive jet with a light jet must exceed 300.

The cross sections for signals ($ m_B = 1000, 1200,1400 $ GeV) and backgrounds are presented in Table 1. Herein, all the SM backgrounds are effeciently suppressed after applying the cuts, while a relatively high signal effeciency is maintained. The statistical significance $ S/\sqrt{S+B} $ is presented in the table. For $ m_B=1200 $ GeV and an integrated luminosity of 5 ab^–1, a significance of 6.447 is achieved. Better significance is expected for lighter VLQ-B masses or higher integrated luminosities.

Cuts	Signals				Backgrounds
	1000 GeV	1200 GeV	1400 GeV		WWZ	WWh	$ t\bar{t} $	$ t\bar{t}h $
Basic	0.1314	0.118	0.07206		7.532	0.4031	7.792	0.1372
Cut-1	0.09796	0.0754	0.0397		0.5632	0.0395	1.076	0.0401
Cut-2	0.02197	0.0169	0.0088		0.000422	0.00037	0.036	0.00208
Cut-3	0.02005	0.0155	0.0082		0.000166	0.0002684	0.01196	0.00098
$ S/\sqrt{S+B} $	7.747	6.447	3.944

Table 1.  Cut flow of the cross sections (in fb) for the signal benchmarks at three VLQ-B masses and background processes in the boosted Higgs channel. The statistical significance (SS) is calculated assuming an integrated luminosity of 5 ab⁻¹.

In this subsection, we consider the second scenario, where the heavy VLQ-B decays via $ B \rightarrow Z b $, followed by the hadronic decay $ Z \rightarrow j j $. The Feynaman diagram topology for this channel is similar to that in Section IIIA. the final state comprises $ J_ZJ_Zb\bar{b} $, where $ J_Z $ represents the fat jet derived from highly boosted Z boson. The dominant SM backgrounds for this signal are similar to those in the boosted h channel, namely: $ e^+ e^- \rightarrow W^+ W^- Z \rightarrow 6j $, $ e^+ e^- \rightarrow W^+ W^- h \rightarrow j j j j b \bar{b} $, $ e^+ e^- \rightarrow t \bar{t} \rightarrow j jb j j\bar{b} $, and $ e^+ e^- \rightarrow t \bar{t} h\rightarrow j j b j j \bar{b}b \bar{b} $.

Figure 3.  (color online) The normalized distributions for transverse momentum and pseudo-rapidity of jets for the signal in boosted higgs channel and backgrounds.

In this channel, we cluster the final hadrons using a large jet radius R=0.8, applying similar basic cuts to those in Section III.A. The jets are then reordered by mass. The mass distributions for the mass ordering jets $ j_{i} $ (i = 1 ~ 4) for the signal and backgrounds are presented shown inFig. 4. The transverse momentum and pseudo-rapidity disttributions in the boosted Z channel are similar to those in boosted Higgs channel; hence, they are not shown here to avoid redundancy. The dominant background is $ t\bar{t} $ owing to its large production rate and decay topology. Although the mass peaks of the two leading massive jets for the signal and $ WWZ $ background overlap, the majority of $ WWZ $ background events can be suppressed by applying a strong $ H_T $ cut. To further reduce the backgrounds, we adopt a set of improved cuts similar to those used in Section 111A:

Figure 4.  (color online) The normalized distributions for the mass of jets in boosted Z channel and backgrounds.

● Cut-1: At least four isolated jets are required, and the scalar sum of all transverse momenta of jets must 600 GeV, i.e, $ (N(j) \geq 4) $ and $ H_T > $600.

● Cut-2: The masses of the two leading jets must lie in the Z-mass window, $ 80<M_{j_{1,2}}<100 $. Additionally, there must be at least two light jets with masses $ M_{j_{3,4}}<70 $.

● Cut-3: The combined reconstructed mass of each massive jet with a light jet must exceed 300.

● Cut-4: The combined reconstructed mass of the two light jets must exceed 100.

The cross sections for the signals with three benchmark masses ($ m_B=1000, \; 1200,\; 1400 $ GeV) and the corresponding backgrounds are presented Table 2. Herein, the optimized cuts effectively suppress the backgrounds. For $ m_B=1200 $ GeV, a significance of 6.198 for a luminosity of 5 ab^–1 is achieved. The significance decreases for heavier VLQ-B masses.

Cuts	Signals				Backgrounds
	1000 GeV	1200 GeV	1400 GeV		WWZ	WWh	$ t\bar{t} $	$ t\bar{t}h $
Basic	0.09796	0.08189	0.05245		7.532	0.4031	7.792	0.1372
Cut-1	0.09139	0.07474	0.04559		2.3128	0.1425	3.355	0.1057
Cut-2	0.01962	0.01753	0.01161		0.2564	0.007156	0.0742	0.00209
Cut-3	0.0172	0.01533	0.01033		0.1005	0.00424	0.0111	0.0005328
Cut-4	0.0169	0.01499	0.01018		0.002448	0.00148	0.01001	0.000464
$ S/\sqrt{S+B} $	6.7514	6.198	4.58

Table 2.  Cut flow of the cross sections (in fb) for the signal benchmarks at three VLQ-B masses and background processessthe in boosted Z channel. The statistical significance (SS) is calculated assumin an integrated luminosity of 5 ab⁻¹.

With the introduction of exotic decay modes, the BRs of standard channels may decrease, thereby relaxing current limits from ATLAS and CMS. In this study, we investigate TeV-scale weak-singlet VLQ-B quarks via pair production at the 3 TeV CLIC, focusing on two decay channels: $ B\rightarrow bZ $ and $ B\rightarrow bh $. The heavy VLQ-B induces highly boosted Z and h bosons, which are reconstructed as fat-jets in the hadronic final state. A full simulation is performed for the boosted Higgs and Z channels. After selecting a large jet radius and applying numerous cuts, the clean collision environment and rare fat-jets backgrounds at the CLIC enable effective signal extraction in certain parameter spaces. For $ m_B=1.2 $ TeV (1.4 TeV), a signal significance of 6.447 (3.944) can be achieved in the boosted Higgs channel with an integrated luminosity of 5 ab^–1. In the boosted Z channel, a significance of 6.198 (4.58) can be achieved under a similar luminosity.

We further evaluate the CLIC sensitivity to Br($ B\rightarrow bh $) and $ {\rm Br}(B\rightarrow bZ) $. Fig. 5 shows the 95% plot CL exclusion limits and 5$ \sigma $ sensitivity for boosted Higgs channel and $ Z $ channels at the 3 TeV CLIC with an integrated luminosity of 5 ab^–1. In the boosted Higgs channel, the VLQ-B can be excluded for Br$ (B\rightarrow bh ) \in [0.107,$ $ 0.25] $ and $ m_B \in [1000, 1480] $ GeV with an integrated luminosity of 5 ab^–1. In the boosted Z channel, the VLQ-B can be excluded for Br($ B\rightarrow bZ $) $ \in [0.119,0.25] $ and $ m_B \in [1000, 1490] $ GeV with an integrated luminosity of 5 ab^–1. As shown in the figure, fully hadronic channels and leptonic channels [64] at the CLIC exhibit similar performances in VLQ-B. For comparison, we show the observed 95% CL exclusions limits from the 13 TeV LHC, projections from the future HL-LHC with an integrated luminosity of 3000 fb^–1, and previous leptonic-channel exclusions [64]. The future 3TeV CLIC with an integrated luminosity of 5 ab^–1 offers improved sensitivity than the current LHC in the mass range of $ 1120<m_B<1480 $, surpassing that of the future HL-LHC in some mass regions. Reagrding discovery prospects, a VLQ-B with a mass between 1 TeV and 1.4 TeV and sizable Br($ B\rightarrow bZ)>0.185 $ could be discovered at future 3 TeV CLIC.

Figure 5.  (color online) Exclusion limits (at 95% CL) and discovery prospects (at 5$ \sigma $) sensitivity for boosted Higgs and Z channels at the 3 TeV CLIC with an integrated luminosity of 5 ab⁻¹

The analysis presented in this study is applicable to a $ \sqrt{s}=3\; $ TeV moun collider. At 10 TeV, despite the large phase space, the pair production cross section is considerably suppressed owing to the s-channel effect. For $ m_B=1.5$ TeV (2.5 TeV), the cross section reaches only 0.5345(0.5231) fb. Overall, the low production rate makes detecting the TeV-scale VLQs at future 10 TeV muon collider unlikely.