Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (2024)

J.Lasker\orcidlink0000-0003-2999-4873  A.Carnero Rosell\orcidlink0000-0003-3044-5150  A.D.Myers  A.J.Ross\orcidlink0000-0002-7522-9083  D.Bianchi\orcidlink0000-0001-9712-0006  M.M.SHanif\orcidlink0009-0006-2583-5006  R.Kehoe  A.deMattia  L.Napolitano\orcidlink0000-0002-5166-8671  W.J.Percival\orcidlink0000-0002-0644-5727  R.Staten  J.Aguilar  S.Ahlen\orcidlink0000-0001-6098-7247  L.Bigwood  D.Brooks  T.Claybaugh  S.Cole\orcidlink0000-0002-5954-7903  A.de la Macorra\orcidlink0000-0002-1769-1640  Z.Ding\orcidlink0000-0002-3369-3718  P.Doel  K.Fanning\orcidlink0000-0003-2371-3356  J.E.Forero-Romero\orcidlink0000-0002-2890-3725  E.Gaztañaga  S.Gontcho A Gontcho\orcidlink0000-0003-3142-233X  G.Gutierrez  K.Honscheid  C.Howlett\orcidlink0000-0002-1081-9410  S.Juneau  A.Kremin\orcidlink0000-0001-6356-7424  M.Landriau\orcidlink0000-0003-1838-8528  L.LeGuillou\orcidlink0000-0001-7178-8868  M.E.Levi\orcidlink0000-0003-1887-1018  M.Manera\orcidlink0000-0003-4962-8934  A.Meisner\orcidlink0000-0002-1125-7384  R.Miquel  J.Moustakas\orcidlink0000-0002-2733-4559  E.Mueller  J.Nie\orcidlink0000-0001-6590-8122  G.Niz\orcidlink0000-0002-1544-8946  M.Oh  N.Palanque-Delabrouille\orcidlink0000-0003-3188-784X  C.Poppett  F.Prada\orcidlink0000-0001-7145-8674  M.Rezaie\orcidlink0000-0001-5589-7116  G.Rossi  E.Sanchez\orcidlink0000-0002-9646-8198  D.Schlegel  M.Schubnell  H.Seo\orcidlink0000-0002-6588-3508  D.Sprayberry  G.Tarlé\orcidlink0000-0003-1704-0781  M.Vargas-Magaña\orcidlink0000-0003-3841-1836  B.A.Weaver  MichaelJ.Wilson  Y.Zheng

(Last updated YYYY MM DD; in original form YYYY MM DD)

Abstract

A critical requirement of spectroscopic large scale structure analyses is correcting for selection of which galaxies to observe from an isotropic target list. This selection is often limited by the hardware used to perform the survey which will impose angular constraints of simultaneously observable targets, requiring multiple passes to observe all of them. In SDSS this manifested solely as the collision of physical fibers and plugs placed in plates. In DESI, there is the additional constraint of the robotic positioner which controls each fiber being limited to a finite patrol radius. A number of approximate methods have previously been proposed to correct the galaxy clustering statistics for these effects, but these generally fail on small scales. To accurately correct the clustering we need to upweight pairs of galaxies based on the inverse probability that those pairs would be observed (Bianchi & Percival 2017). This paper details an implementation of that method to correct the Dark Energy Spectroscopic Instrument (DESI) survey for incompleteness. To calculate the required probabilities, we need a set of alternate realizations of DESI where we vary the relative priority of otherwise identical targets. These realizations take the form of alternate Merged Target Ledgers (AMTL), the files that link DESI observations and targets. We present the method used to generate these alternate realizations and how they are tracked forward in time using the real observational record and hardware status, propagating the survey as though the alternate orderings had been adopted. We detail the first applications of this method to the DESI One-Percent Survey (SV3) and the DESI year 1 data. We include evaluations of the pipeline outputs, estimation of survey completeness from this and other methods, and validation of the method using mock galaxy catalogs.

1 INTRODUCTION

Large scale spectroscopic sky surveys have been a primary way of studying the universe over the past twenty five years [1, 2, 3]. They obtain redshifts of many millions of galaxies in order to measure their position in three dimensional space and extract information regarding the clustering of galaxies and the dark matter fields which they trace throughout cosmic time. These measurements not only provide insight into the composition of the universe and the physics which drive the evolution of its expansion, but also the formation of the galaxies themselves.

The Dark Energy Spectroscopic Instrument (DESI) survey, like the extended Baryon Oscillation Spectroscopic Survey (eBOSS) and the Sloan Digital Sky Survey (SDSS) before it, utilizes a multi object spectrograph (MOS) which enables multiplexing of spectroscopic observations. However, DESI is able to multiplex at a scale similar-to\sim5x larger than that of eBOSS, capturing greater-than-or-equivalent-to\gtrsim4000 spectra of astronomical objects per pointing. This increase in efficiency is largely due to the fact that SDSS and eBOSS hand-placed the fibers which fed their spectrographs into plates, while the fibers which feed the DESI spectrographs are controlled robotically. These positioners place all 5000 fibers on targets within the focal plane [4]. DESI also utilizes a dynamic exposure time calculator [5, 6] which takes information on observing conditions including seeing, transparency, and sky brightness to estimate the true signal-to-noise accumulated during an exposure. The capability to optimize exposure times and to construct new "plates" as needed, combined with a fivefold increase in available fibers and a 2.5xsimilar-toabsent2.5𝑥\sim 2.5x∼ 2.5 italic_x increase in the mirror area, allowed DESI to obtain similar-to\sim10x as many extragalactic redshifts in its first year as eBOSS did in its entire 5 year survey.

While robotically controlled fibers have provided many gains to the DESI survey, they have not eliminated the problem of fiber collisions. A fiber collision occurs when two targets are close enough on the sky that two fibers and their plugs (in SDSS and eBOSS) or their positioners (in DESI) would have collided were a survey to attempt to simultaneously observe both [7]. DESI also has a limitation similar to that of fiber collisions where only one fiber may be placed within a single fiber patrol radius, with a small amount of overlap towards the edges of the patrol radii of neighboring fibers. This enforces uniform areal observation density within a single field of view which can only begin to reflect the non-uniform density of targets after multiple passes. Since they have similar effects on the ability to observe close pairs, we will refer to both effects collectively as ’fiber collisions.‘

Additionally, robotic fiber positioners can suffer from time-variable issues which were not present in the hand-plugged fiber systems used by SDSS, such as losses to positioner power and software-based failures which interrupt the positioning process. Fiber-collision issues combine with more traditional problems such as sky subtraction failures, atmospheric variability, and redshift determination errors, to create incompleteness effects which vary spatially and temporally across the DESI survey.

Fiber collisions affect the smallest scale clustering since they are limited to angular scales on the sky of approximately <1.48absent1.48<1.48< 1.48arcminutes which projects to a distance of 0.99 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT at an intermediate redshift of z = 1.0. Real time hardware events like power failures to the devices which control groups of fiber positioners (CAN busses) can also cause missing observations at larger sets of separations. These smaller scales are critical to analyses of the galaxy-halo connection. These analyses, whether they use Halo Occupation Distribution (HOD) methods [8, 9, 10, 11, 12, 13] or SubHalo Abundance Matching (SHAM) methods [14, 15, 16], rely on clustering measurements at small scales to determine the placement and quantity of satellite galaxies.

Prior to the usage of pairwise inverse probability weighting, (e)BOSS and SDSS used have used several different methods to correct for incompleteness. The BOSS DR9 BAO analysis [17] used the method developed by Guo, Zehavi, and Zheng in 2012 [18] to correct for fiber collisions. These weights are determined by dividing the galaxy sample into those which are not subject to any fiber collision effects and those which are subject to those effects. The relative fraction of galaxies within the collided sample which are observed is calculated. Then the pair counts which include galaxies from the collided sample are upweighted based on that fraction. This method does not account for the fact that unobserved pairs are not statistically equivalent to those which are observed. The Reid et al 2014 [19] analysis of the growth of structure using small scale clustering utilizes a combination of angular upweighting on small scales, and nearest neighbor upweighting, which accounts for unobserved targets by adding one to a "close pair" weight for an observed target for each unobserved target which is closer to it than any other observed member of its target class, at large scales, and a truncation of the measured multipoles at small separation perpendicular to the line of sight. Other standard SDSS and eBOSS analyses [20, 21, 22, 23, 24] have used nearest neighbor upweighting. This method assumes that all fiber collisions occur between galaxies within clusters or are otherwise at similar redshifts.

To address this issue of measuring small scale clustering, Bianchi and Percival 2017 (hereafter BP17; [25]) developed a new method of correction for incompleteness by upweighting each pair of galaxies rather than individual galaxies or the total number of paircounts. They devise a method of Pairwise Inverse Probability (PIP) weights which maintains consistent numbers of both weighted pair counts and weighted individual galaxy counts on all scales as long as there is some area of the survey footprint which is observed multiple times.

Bianchi and Percival note that, despite the nominal possibility of computing such pairwise weights analytically, for modern surveys like DESI which will observe 𝒪(107)𝒪superscript107\mathcal{O}(10^{7})caligraphic_O ( 10 start_POSTSUPERSCRIPT 7 end_POSTSUPERSCRIPT ) galaxies, such calculations will be practically impossible. They do note that these weights can be effectively estimated by randomly repeating the galaxy selection process a sufficient number of times and estimating the probability of each pair based on the frequency at which each pair is selected. The remainder of this paper will show a new method which allows us to estimate those probabilities, show its effect on the clustering measurements, and validate the weights using mocks.

Mohammad et al 2018 [26] showed the first application of the BP17 PIP weighting to data using the VIPERS survey. Additionally, they compute angular upweighting as described in Percival and Bianchi 2017 (hereafter PB17; [27]). This application to a slit-based survey is not strictly the same as the DESI application, but showed the validity of the BP17 PIP + PB17 angular upweighting method when applied to mocks and data. Mohammad et al 2020 [28] showed the first application of the BP17 PIP weighting + PB17 angular upweighting method to MOS data using the eBOSS DR16 catalogs. They used a version of the eBOSS tiling software which was modified to enable easier parallelization of fiber assignment across 1860 realizations of the eBOSS survey in which they varied only the initial random number generation used by the software. They store the results of the realizations as bitweights as recommended in BP17 and then use them to compute pairwise inverse probability weights. The authors then use EZ Mocks to validate that their application of PIP weighting successfully recovers one-halo clustering down to a scale of 0.1 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT .

In this paper we present a new method of calculating PIP weights to correct the DESI SV3 and Y1 clustering measurements for incompleteness. This method utilizing Alternate MTLs (AMTLs) is a nearly-maximally realistic monte carlo estimation of the likelihood that any target or collection of targets would be observed over an infinite repetition of the DESI experiment. This method incorporates more realism than prior treatements in eBOSS by utilizing real observation results, hardware states, and sequential changes in priority as overlapping tiles are observed. This method will enable unbiased measurement of clustering of galaxies down to scales 0.1absent0.1\leq 0.1≤ 0.1 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT .

In section 2 we describe the DESI survey strategy and how this affects the observation of different classes of targets, in Section 3 we briefly discuss the mock galaxy catalogs used to validate this method, in Section 4 we describe both the regular and alternate MTLs and the procedure for how we create and propagate the alternate MTLs, in Section 5 we describe the DESI clustering code, Pycorr and how it implements these weights, in Section 6 we show results from the data and mocks to validate the method and show the differences with other methods of incompleteness determination, and finally in Section 7 we discuss the results and their applicability to other analyses within DESI.

2 Survey

The Dark Energy Spectroscopic Instrument (DESI: [29, 30, 31]) is the Stage IV successor to large scale sky surveys such as the Sloan Digital Sky Survey (SDSS: [1]) and the (extended) Baryon Oscillation Spectroscopic Survey ((e)BOSS: [3]). The DESI experiment is currently in its third year of a projected five year program and has already recorded redshifts for over 30 million galaxies out to redshifts over z3.5similar-to𝑧3.5z\sim 3.5italic_z ∼ 3.5. The primary goal of this survey is to further constrain cosmological parameters such as the dark energy equation of state parameter (w𝑤witalic_w) and the Hubble Constant (H0subscript𝐻0H_{0}italic_H start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT). The survey plans to do this by performing the most precise measurement of the Baryon Acoustic Oscillation (BAO) feature and of Redshift Space Distortions (RSD) in its clustering measurements. However, the science goals of DESI are not limited to dark energy. Already-published analyses from DESI include explorations of the Galaxy Halo Connection, measurements of the local stellar population in the Milky Way, studies of Quasistellar Object (QSO) astrophysics, and many tests of the survey and pipeline performances. This paper is part of a set of analyses surrounding the first Data Release of DESI (DR1, [32]) which includes two point clustering measurements and validation [33], measurements of the BAO feature in Galaxies, QSOs, and the Lyman-α𝛼\alphaitalic_α forest [34, 35], full shape measurements of Galaxy and QSO clustering [36], as well as cosmological constraints measured from the above features including tests of primordial non-gaussianity [37, 38, 39].

DESI selects the galaxies and stars that it observes from a parent population of Milky Way targets (MWS; [40]), Bright, low redshift, Galaxies (BGS; [41]), Luminous Red Galaxies (LRGs; [42]), Emission Line Galaxies (ELGs; [43]), and Quasistellar Objects (Quasars/QSOs; [44]). Targets are obtained via prior imaging of the DESI footprint[45]. These targets are divided into two concurrent programs with BGS and MWS targets being observed during bright times or in other sub-optimal conditions while LRG, ELG, and QSO targets are observed only in sufficiently good dark time [5].

During the One-Percent Survey (also referred to as SV3), these targets were observed within 20 rosettes [46]. SV3 was designed to optimize the observation of complete samples of tracers over an area approximately equal to one percent of the final survey area. This survey also provided a final test for the survey operations procedures and software prior to the main survey. These rosettes were selected to be evenly distributed between the northern and southern galactic caps as well as to overlap with various fields of interest from CFHTLS, COSMOS, DES, GAMA, Euclid, ELAIS, GOODS, HSC, KiDS, VVDS, XBootes, and XDEEP as well as overlapping with the Coma cluster and the ecliptic pole. For each rosette, 12/10 (dark/bright) tiles were designed with tile centers spaced evenly around the rosette center at a distance of 0.12superscript0.120.12^{\circ}0.12 start_POSTSUPERSCRIPT ∘ end_POSTSUPERSCRIPT as shown in Figure1. Observation of these rosettes followed a sequence where, for each rosette, one tile was designed and fiberassigned in the afternoon before observing or earlier, observed at night, and then the Merged Target Ledgers (MTLs111MTLs will be further explained in §4.3 and more details can be found in §6 of [5].) containing the targets which were observed in that tile222MTLs are divided into HEALPixels (https://healpix.jpl.nasa.gov/) with nside = 32 (pixel size 1.8similar-toabsentsuperscript1.8\sim 1.8^{\circ}∼ 1.8 start_POSTSUPERSCRIPT ∘ end_POSTSUPERSCRIPT) for ease of reading and storage were updated. Then the next overlapping tile on the same rosette was designed, incorporating the information from the previous night’s observations, and observed in the same manner until all 12/10 tiles were observed. A flowchart of this process is shown in Figure2.

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (1)
Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (2)

This strategy enabled high completeness of 96.2% for MWS targets, 98.9% for BGS targets, 98.6% for LRG targets, 95.2% for ELG targets, and 99.4% for QSO targets within a range of rosette radii of 0.2 and 1.45 degrees[46]. Despite the nearly 100% completeness in that range, there is still an inner and outer region in which we can study the incompleteness measurements and weights from the AMTL method. Additionally, some regions within the central complete area are still subject to fiber collision effects due to high target number density and nightly focal plane positioning issues.

The observed SV3 spectra were processed by the DESI spectroscopic pipeline [47] in a hom*ogeneous processing run denoted as ‘Fuji’, with the resulting redshift and large-scale structure (LSS) catalogs used in this paper released in the early data release (EDR [48]). The results of the AMTL pipeline we describe in this paper were included in the EDR LSS catalogs.

The main survey maintained the strategy of not observing overlapping areas of sky before updating the MTLs, but is no longer confined to rosettes. The tiles for the main survey were chosen based on the Hardin, Sloane, and Smith 2001 icosahedral tiling333http://neilsloane.com/icosahedral.codes/[49] and are often fiber assigned "on-the-fly" as survey observations are occurring. This allows for greater flexibility in where one can point the telescope during a night of observing due to changing conditions including wind and clouds.The main survey is ongoing, but the DESI data release 1 (DR1 [32]) will include data from just over the first year of its operations, May 14, 2021 through June 14, 2022. Analyses of this dataset are ongoing and the resulting LSS catalogs have completeness 62% in BGS, 69% in LRG, 35% in ELG, and 88% in QSOs [33]. While we do not focus this analysis on the DR1 data, we do discuss some of the Y1 pipeline outputs and validation that the pipeline is working as expected. The relative incompleteness of the DR1 sample makes the AMTL pipeline a vital piece in the analysis, especially for the creation of realistic mocks [50, 51] and analyses of small-scale clustering.

3 Mocks

In order to validate the AMTL method, we use a set of 25 mock realizations processed in a similar manner to the data, reproducing the SV3 processing. They are built from the dedicated AbacusSummit444https://abacussummit.readthedocs.io/en/latest/ N-body simulations [52, 53] and include different ensembles for LRG, ELG and QSO. These are then combined into a single file that mimics the structure of the observation data and that are suitable to be processed through fiber assignment and LSS pipelines.

Mocks are made from Abacus realizations with fiducial cosmology (Planck 2015 ΛΛ\Lambdaroman_ΛCDM cosmology [54]). For each tracer type (ELG, LRG and QSO) we build cubic mocks of 2 Gpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT size each, at different primary redshifts following a HOD modelling [55, 56] tuned to the clustering amplitude of SV3 data. In SV3, we chose the following Abacus snapshots: LRGs at z=0.8, ELGs at z=1.1 and QSOs at z=1.4.

Once the cubic box mocks are made, we transform them to CutSky mocks, translating from Cartesian coordinates to sky coordinates and including the effect of redshift-space distortions, resulting in galaxy catalogs already in redshift space. These catalogs are resampled to match the same n(z) distribution and footprint as SV3 data and prepared such that we add the columns needed to go through fiber assignment. In addition, we also make random catalogs following the same footprint and redshift distribution as the mock catalogs.

These mock catalogs are then processed through the AMTL pipeline (describe in §4) and LSS pipeline (described in §4 in [48]), creating clustering catalogs analogous to those of the data. Not all the observational effects are modelled, and therefore, we adapt the AMTL and LSS pipelines to accommodate these differences.

The only significant difference in the AMTL pipeline is that we do not pull any information from the redshift-fitting process [47] and the effect of redshift failures is ignored through the entire AMTL loop (fixing ZWARN = 0 in all steps). In summary, we assume that all targets assigned have a good redshift. In future releases we expect to model this effect more robustly, although we expect negligible effect on the clustering signal, since redshift failures will depend on the individual physical properties of the targets and is assumed to be independent of the underlying clustering.

Finally, the output products of the mock AMTL method are run through the LSS pipeline to create clustering ready catalogs with the same completeness properties as the data. Together with the mock data, we process the associated randoms. Here we have modified the pipeline to ignore any effect coming from low data quality (assuming a 100% success rate) and ignoring any angular mask applied to the data. These implied subtle changes in the pipeline at the full catalog stage (see §4.2.1 in [48]).

4 Alternate Merged Target Ledgers

4.1 Pairwise Inverse Probability Weighting

Before describing the Alternate MTLs, we will first describe how they are used in the context of the Pairwise Inverse Probability (PIP) weighting of BP17.

PIP weighting, as the name suggests, involves weighting pairs of galaxies based on the inverse of the probability that each individual pair would be observed. This probability can theoretically be calculated analytically, but the large number of galaxies observed in DESI (𝒪(107)𝒪superscript107\mathcal{O}(10^{7})caligraphic_O ( 10 start_POSTSUPERSCRIPT 7 end_POSTSUPERSCRIPT )) prohibits this from being practical. Instead of this, we can run Monte Carlo realizations of DESI, varying only those selection properties which are truly random.

When deciding how many realizations to generate for the calculation of PIP weights, we consider that in order for the PIP weights to formally unbiased, there must be no zero probability pairs. While we cannot guarantee zero probability pairs in a finite number of realizations, we show in Table4 that in SV3 there no zero probability QSOs and BGS_BRIGHT targets with 𝒪(103)%𝒪percentsuperscript103\mathcal{O}(10^{-3})\%caligraphic_O ( 10 start_POSTSUPERSCRIPT - 3 end_POSTSUPERSCRIPT ) % zero probability LRGs and ELGs while in Y1, the fraction of zero probability objects is less than 1%.

To mitigate the effect of zero probability pairs, we also employ the angular upweighting of PB17. The combination of PIP weighting with angular upweighting was shown in Bianchi et al 2018 [57] as well as in subsequent analyses of data and simulations [26, 58, 28] to provide practically good results in the presence of zero probability pairs despite a lack of perfect theoretical rigor.

4.2 The Merged Target Ledgers

DESI stores potential targets in a set of files referred to as the merged target lists or ledgers (MTLs; see §6 of[5]). These MTLs contain information generated and updated by the DESI targeting software [59] and used by the DESI assignment code [60], to place fibers on targets. This information includes a unique TARGETID field to identify objects, sky coordinates expressed as Right Ascension ( RA) and Declination ( DEC), a PRIORITY (determined by a target’s class), a randomly determined SUBPRIORITY (in the range 0 to 1), and bitmasks indicating the different target classes and programs to which an object belongs. A summary of all the parameters in the MTLs is given in table 1 and a more detailed description is given in §6 of [5].

Parameter NameParameter Description
TARGETIDUnique identifier for each DESI target (see [59]).
RARight Ascension in decimal degrees.
DECDeclination in decimal degrees.
REF_EPOCHReference epoch for stellar targets used to adjust coordinates for proper motion.
PARALLAXComponent of stellar coordinate change due to parallax.
PMRA/PMDECComponent of stellar coordinate change due to true (proper) motion of stars.
SV3_DESI_TARGET /SV3_BGS_TARGET / SV3_MWS_TARGET /SV3_SCND_TARGETBitmasks described completely in desitarget.targetmask (see [59]) which record which target class(es) a target falls into.
PRIORITYThe priority which fiberassign uses to determine which targets to assign fibers. This is determined by target class and observation status.
PRIORITY_INITThe priority for a target that has yet to be observed.
SUBPRIORITYA small additional priority which fiberassign uses to reproducibly resolve collisions. This is a random number in the range 0 to 1.
OBSCONDITIONSBitmask which represents the observing conditions (bright, dark or backup) in which an object can be targeted (see [5]).
NUMOBS_INITThe number of observations required for a target that has yet to be observed.
NUMOBS_MOREThe number of successful observations until the target is considered "done".
NUMOBSThe current number of observations of the target.
ZThe redshift of the target.
ZWARNBitmask which records Redrock warning flags for the redshift determination.
ZTILEIDThe ID(s) of the tiles on which a target obtained a good observation.
TARGET_STATEA human-readable representation of the current state of the target factoring in its various target classes and current status.
TIMESTAMPThe timestamp at which the MTL entry was made.
VERSIONThe version of the desitarget code which was used to make the MTL update.

When an observation is performed, each target for which good data was obtained has an entry added to the end of ledger that records the results of the observation. A highly simplified example of the initial ledger is presented in table 2 and the result of updating the states in the simplified initial ledger is presented in table 3. As shown in tables 2 and 3, the updates made to a target after a successful observation include an increment in NUMOBS, a drop in PRIORITY, an associated change in TARGET_STATE, and updates to the Z and ZWARN fields based on the output from the DESI spectroscopic pipeline [47] and redshift-fitting software.

TARGETIDPRIORITYNUMOBSZZWARNTARGET_STATE
11032000-1.0-1LRG\|UNOBS
21031000-1.0-1ELG__\__HIP\|UNOBS
31034000-1.0-1QSO\|UNOBS
TARGETIDPRIORITYNUMOBSZZWARNTARGET_STATE
11032000-1.0-1LRG\|UNOBS
21031000-1.0-1ELG__\__HIP\|UNOBS
31034000-1.0-1QSO\|UNOBS
1210.6840LRG\|DONE
210310011.454ELG__\__HIP\|MORE_ZWARN
310335011.90QSO\|MORE_MIDZQSO

The updating of MTLs in-between repeat observations of the same patches of sky combined with the resulting drop in priority for a successfully observed targets highlights the utility of the AMTL method. Crucially, a single change in which a target was observed on the first pass through a region can cause cascading effects due to fiber placement on subsequent tiles.

4.3 Generating the Alternate Merged Target Ledgers

Having described the regular MTLs, we will now describe how we generate the alternate MTLs.

The primary difference between the regular and alternate MTLs is the randomization of the subpriority field. The other major difference is, for both SV3 and main survey, a reproduction of the BGS promotion of 20% of BGS_FAINT targets to the higher priority class of BGS_BRIGHT, and for the main survey only, a reproduction of the ELG promotion of 10% of all ELG targets to the higher priority ELG_HIP class.

To begin the alternate MTL generation process, the survey MTLs are loaded and then truncated so that only the initial entries are maintained. For the initial entries, the SUBPRIORITY field is replaced with new random numbers from the basic numpy uniform random number generator. The random number generation is seeded based on the healpixel number of the initial MTL, the realization number, and a user provided random seed. The SV3 MTLs were randomized with a version of the code which did not add in the healpixel number to the random seed. This randomization is still valid, but we include this information, as well as a tagged version of the code as it was for the SV3 MTLs555https://github.com/desihub/LSS/tree/v1.0.0-EDA to allow for reproduction. The promotion of the BGS targets is a multiple step process where first, the initially promoted 20% are demoted back to the standard BGS_FAINT priority and then a desitarget convenience function (random_fraction_of_trues) is used to select 20% of the total BGS_FAINT targets to promote to BGS_FAINT_HIP. The promotion of ELG targets is similar with two additional considerations. First, targets are being drawn from two starting classes: ELG_LOP and ELG_VLO and must be returned to their resepective classes. Second, not all demoted objects will see a drop in priority since it is not forbidden to have an object be targeted as both an ELG and an LRG (same priority as ELG_HIP) or as both an ELG and QSO (higher priority than ELG_HIP).

Tracer type% of Zero Probability% of Zero Probability
Objects in SV3Objects in Y1
BGS_BRIGHT00.21
LRG3E-30.22
ELG3E-30.17
QSO00.28

This procedure is done in parallel for up to 128 alternate MTLs per node on the Perlmutter supercomputer at NERSC.666www.nersc.gov/perlmutter For the SV3 alternate MTLs, we have chosen to only have the 128 realizations of alternate MTLs for the data and 256 for the mocks. This choice was made from a consideration of the number of zero probability objects that result from a given number of realizations as shown in figure 3 for SV3 and 4 for Y1. Each panel of both figures shows the number of zero probability objects from each target type (from left to right: BGS_BRIGHT, LRG, ELGs with QSO targets removed (ELG_notqso), and QSO) as the number of realizations increases from 8 to 128. The multiple lines in each figure are random reorderings of the 128 realizations to confirm that there is no dependence of the convergence on the (random) ordering of realizations. The exact percentages of zero probability objects for SV3 data and Y1 data for the chosen number of 128 realizations is given in Table4. A zero probability object is a target that is never observed over a given number of realizations. BP17 states that their method of weighting is not completely valid where there are zero probability pairs. While we have essentially eliminated zero probability objects in SV3, the trend in Figure4 shows that it would be prohibitive to run enough realizations to eliminate all zero probability objects in Y1. However, we will show that the perecentages of zero probability objects shown in table 4 from 128 realizations will sufficiently mitigate the problem.

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (3)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (4)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (5)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (6)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (7)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (8)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (9)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (10)

4.4 The Observation Loop

The core of the utility of the AMTL method as compared to other implementations of PIP weighting is the ability to update the alternate MTLs using real observation information. In order to do this, we create an observation loop that follows the same ordering as the survey data processing and observation.

Following the creation of the initial randomized ledgers, the SV3 AMTL loop would use the desitarget.mtl function tiles_to_be_processed to check the list of observed SV3 tiles from the data against the list of already processed tiles for each realization. Then the set of tiles corresponding to the earliest remaining observation date were selected. All tiles which were observed on that date were put through an observation loop designed to replicate the SV3 observing process.

The observation loop begins by running fiberassign [60] using the alternate ledgers as the primary target ledgers. Other than the primary target ledgers, this procedure uses the same configuration as the corresponding fiber assignment run in the real survey including the list of sky fibers, calibration standards, secondary targets, and the hardware configuration at the exact time at which fiberassign was run for the main survey. This produces a set of files which map the alternate targets to DESI fibers for the alternate observations.

In order to accurately track the completeness effects due to improperly positioned fibers, fiber positioning system failures, and other focal plane-level effects, we map the observation result from each fiber in the original survey to the same fiber in the alternate realization. For example, if fiber 1 were pointed at target 1 in tile 1 of the real survey and fiber 1 were pointed at target 2 in tile 1 of an alternate realization, target 2 would would receive the observation result of target 1 in that alternate realization. Inclusion of the observational results from the survey means any real-time positioning failures are automatically incorporated. The survey pipeline marks observations as bad if they are come from fibers with flags indicating bad hardware status.

This can, and does, lead to the assignment of observations of one target class to observations of another target class. However, since the primary consideration is whether or not a target has been placed on a fiber which received a good observation rather than whether it received a successful redshift, this approach is still mostly valid. For the case of QSOs where the value of the redshift determines the number of future observations this is not exactly correct. For this purpose we are developing a method to assign results from a "twin fiber" which is nearby the true fiber and matches in as many properties as possible, primarily the target class. For the full five year sample, we also plan to substitute QSO redshifts from original observations with the final survey redshift results. This will be possible since QSOs are the highest priority target class and will be nearly fully complete by the end of the survey. In this analysis, we will note that the fraction of observations whose primary target information changes is 6.7 % (13.0%) of all SV3 (Y1) dark time observations and 13.0 % (31.8%) of SV3 (Y1) dark time observations which result in a change in target.

The main survey data added an additional complication to the observation loop, the addition of reprocessed tiles. A reprocessing occurs when the results of the daily pipeline are modified due to (e.g.) a run of the offline pipeline discovering that a "good“ redshift was not actually a good fit. Since reprocessings can change the state of already observed targets and they occur independently of the observations themselves, reprocessings necessitated a separation of the alternate fiber assignment step and the alternate MTL update step where we check for reprocessings of tiles. This also resulted in the storage of the maps between the real and alternate fibers from the regular observation loops so that the reprocessed redshift catalogs/ledgers can be modified in the same way as the original ones.

4.5 Outputs of the pipeline

The AMTL pipeline produces two primary outputs. First is the collection of N (for SV3 and Y1, N = 128 ) realizations of MTLs which includes information on every (alternate) observation of every target. The columns for these MTLs are the same as in the real survey MTLs, which you can find in table 1.

The second output is a bitweight file with three columns, it can be attached to the LSS catalogs (and is indeed already available for the EDR catalogs.777https://data.desi.lbl.gov/doc/ [48])First is the unique TARGETID which is necessary to match the information to other catalogs. Second is two columns with information on how frequently each target was observed over the 128 realizations. One of those two columns is the probability that the individual target would be observed over those 128 realizations, PROB_OBS. PROB_OBS is calculated as the number of realizations in which a target was observed divided by the total number of realizations. The other remaining column is the bitweights, a shorthand encoding of whether each target was observed in each realization as first outlined in BP17. Bitweights are stored as an ordered list of 64 bit integers where each bit is set to 1 if a target was observed (NUMOBS \geq 1) in a given realization and set to 0 if it was not observed in that realization.888In order to manipulate these bitweights, we provide the functions within desihub’s LSS.bitweights module

4.6 Validation of the pipeline

Validation of the pipeline has been a multiple step process. Prior to running the full set of 128 realizations, we have run a single realization with the part of the pipeline which randomizes the subpriorities turned off and confirmed that this version of the pipeline is able to reproduce both SV3 and Y1 exactly. We have also validated that the pipeline produces a statistically equivalent set of promotions to ELG_HIP and BGS_BRIGHT out of the initial randomized alternate ledgers.

In §6 we show clustering measurments validating the PIP weights generated from the pipeline on the mocks described in §3.

5 Clustering Measurements

We will measure clustering based on pair-counts as a function of the redshift-space transverse, ssubscript𝑠perpendicular-tos_{\perp}italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT, and radial, s||s_{||}italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT separation, measured in co-moving h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPTMpc using the fiducial cosmology. The 2-point correlation function probes the excess probability to find in a galaxy sample two galaxies at a given separation. We use the so-called Landy-Szalay estimator[61]

ξ^(s,s||)=DD(s,s||)DR(s,s||)RD(s,s||)+RR(s,s||)RR(s,s||)\widehat{\xi}(s_{\perp},s_{||})=\frac{DD(s_{\perp},s_{||})-DR(s_{\perp},s_{||}%)-RD(s_{\perp},s_{||})+RR(s_{\perp},s_{||})}{RR(s_{\perp},s_{||})}over^ start_ARG italic_ξ end_ARG ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) = divide start_ARG italic_D italic_D ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) - italic_D italic_R ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) - italic_R italic_D ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) + italic_R italic_R ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) end_ARG start_ARG italic_R italic_R ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) end_ARG(5.1)

where XY(s,s||)XY(s_{\perp},s_{||})italic_X italic_Y ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) correspond to the weighted number of pairs of objects X𝑋Xitalic_X and Y𝑌Yitalic_Y in a s,s||s_{\perp},s_{||}italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT bin divided by the total weighted number of pairs. Specifically, DD(s,s||)DD(s_{\perp},s_{||})italic_D italic_D ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) is the weighted number of data (galaxy and quasar) pairs and RR(s,s||)RR(s_{\perp},s_{||})italic_R italic_R ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) is the weighted number of pairs of objects in the random catalog which samples the selection function. For the autocorrelation (single-tracer) measurements performed in our analysis, by symmetry DR(s,s||)=RD(s,s||)DR(s_{\perp},s_{||})=RD(s_{\perp},s_{||})italic_D italic_R ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ) = italic_R italic_D ( italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT ). For standard estimates pair weights are the product of the total individual weights of the two objects in the pair w1subscript𝑤1w_{1}italic_w start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT and w2subscript𝑤2w_{2}italic_w start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT, themselves obtained as the product of systematic correction weights and FKP weights. We use 48 logarithmically spaced ssubscript𝑠perpendicular-tos_{\perp}italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT bins between 0.01 and 100 Mpc/hMpc\mathrm{Mpc}/hroman_Mpc / italic_h and 40 linearly spaced s||s_{||}italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT-bins between 0 and 40Mpc/h40Mpc40\;\mathrm{Mpc}/h40 roman_Mpc / italic_h.

To optimize computing time we use the technique of Keihänen et al 2019[62], which consists in summing DR𝐷𝑅DRitalic_D italic_R, RD𝑅𝐷RDitalic_R italic_D and RR𝑅𝑅RRitalic_R italic_R pair counts obtained using several random catalogs of a size approximately matching that of the data catalog. In practice, we use the maximum number available for the SV3 LSS catalogs, which is 18 and corresponds to 28, 67, 220, and 54×\times× the data number for ELGs, LRGs, QSOs, and BGS_BRIGHT respectively.

In this work, we will show the results of the projected correlation function wp(rp)subscript𝑤𝑝subscript𝑟𝑝w_{p}(r_{p})italic_w start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ( italic_r start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ), with rpssubscript𝑟𝑝subscript𝑠perpendicular-tor_{p}\equiv s_{\perp}italic_r start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ≡ italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT and

wp(rp)=2040𝑑s||ξ(rp,s||)w_{p}(r_{p})=2\int_{0}^{40}ds_{||}\xi(r_{p},s_{||})italic_w start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ( italic_r start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ) = 2 ∫ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 40 end_POSTSUPERSCRIPT italic_d italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT italic_ξ ( italic_r start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT , italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT )(5.2)

the projected correlation function wpsubscript𝑤𝑝w_{p}italic_w start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT integrates the full two point correlation function ξ𝜉\xiitalic_ξ over a range in the line-of-sight distance spanning 40 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT . This distance is chosen to sufficiently remove all rsd effects while not introducing significant noise.

Correlation function measurements are performed with pycorr999https://github.com/cosmodesi/pycorr which wraps a version of the Corrfunc package[63, 64] modified to also run on GPU and support alternative lines-of-sight (first-point, end-point) and weight definitions (PIP and angular upweights) and the θ𝜃\thetaitalic_θ-cut, along with jackknife error estimation.

5.1 Weights

The set of weights applied in the DESI SV3 TPCFs include the "default“ (IIP) weights, FKP weights, angular upweights, and bitwise (PIP) weights. The PIP and IIP weights are never applied simultaneously since they are derived from the same set of AMTLs and correct for the same incompleteness.

The ”default“ IIP weights are calculated from the bitweights output from the AMTL pipeline with the ’efficient estimator‘ of Bianchi and Verde 2020 [65].

wi,IIP=Nreal+1(popcntwi(b))+1subscript𝑤𝑖IIPsubscript𝑁real1popcntsubscriptsuperscript𝑤𝑏𝑖1w_{i,\rm IIP}=\frac{N_{\rm real}+1}{({\rm popcnt}\>w^{(b)}_{i})+1}italic_w start_POSTSUBSCRIPT italic_i , roman_IIP end_POSTSUBSCRIPT = divide start_ARG italic_N start_POSTSUBSCRIPT roman_real end_POSTSUBSCRIPT + 1 end_ARG start_ARG ( roman_popcnt italic_w start_POSTSUPERSCRIPT ( italic_b ) end_POSTSUPERSCRIPT start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) + 1 end_ARG(5.3)

FKP weights were derived in Feldman, Kaiser, and Peaco*ck 1994[66] as a variance-minimizing weight for a given scale of a power spectrum. These FKP weights are given by

wFKP(z)=11+n(z)P0subscript𝑤𝐹𝐾𝑃𝑧11𝑛𝑧subscript𝑃0w_{FKP}(z)=\frac{1}{1+n(z)P_{0}}italic_w start_POSTSUBSCRIPT italic_F italic_K italic_P end_POSTSUBSCRIPT ( italic_z ) = divide start_ARG 1 end_ARG start_ARG 1 + italic_n ( italic_z ) italic_P start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT end_ARG(5.4)

where n(z)𝑛𝑧n(z)italic_n ( italic_z ) is the number density of galaxies at redshift z𝑧zitalic_z, and P0subscript𝑃0P_{0}italic_P start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT is the clustering power at the scale in the power spectrum at which we desire to minimize the variance. In DESI SV3, we chose the following P0subscript𝑃0P_{0}italic_P start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPTvalues for each tracer:

LRG: 10000ELG: 4000QSO: 6000BGS: 7000

Angular upweighting, as first outlined in PB17 [27] and applied in Mohammad et al 2018 [26] and Mohammad et al 2020[28], is derived from the angular clustering of the parent catalog of all potential targets of a tracer type and the angular clustering of the targets which have been observed. These pairwise weights are generated separately for the Data-Data pairs and the Data-Random pairs with the bitweights factoring into both sets of counts as shown below.

wANGDD=DDpar(θ)DDPIPfib(θ)subscriptsuperscript𝑤DDANG𝐷superscript𝐷par𝜃𝐷subscriptsuperscript𝐷fibPIP𝜃w^{\rm DD}_{\rm ANG}=\frac{DD^{\rm par}(\theta)}{DD^{\rm fib}_{\rm PIP}(\theta)}italic_w start_POSTSUPERSCRIPT roman_DD end_POSTSUPERSCRIPT start_POSTSUBSCRIPT roman_ANG end_POSTSUBSCRIPT = divide start_ARG italic_D italic_D start_POSTSUPERSCRIPT roman_par end_POSTSUPERSCRIPT ( italic_θ ) end_ARG start_ARG italic_D italic_D start_POSTSUPERSCRIPT roman_fib end_POSTSUPERSCRIPT start_POSTSUBSCRIPT roman_PIP end_POSTSUBSCRIPT ( italic_θ ) end_ARG(5.5)
wANGDR=DRpar(θ)DRIIPfib(θ)subscriptsuperscript𝑤DRANG𝐷superscript𝑅par𝜃𝐷subscriptsuperscript𝑅fibIIP𝜃w^{\rm DR}_{\rm ANG}=\frac{DR^{\rm par}(\theta)}{DR^{\rm fib}_{\rm IIP}(\theta)}italic_w start_POSTSUPERSCRIPT roman_DR end_POSTSUPERSCRIPT start_POSTSUBSCRIPT roman_ANG end_POSTSUBSCRIPT = divide start_ARG italic_D italic_R start_POSTSUPERSCRIPT roman_par end_POSTSUPERSCRIPT ( italic_θ ) end_ARG start_ARG italic_D italic_R start_POSTSUPERSCRIPT roman_fib end_POSTSUPERSCRIPT start_POSTSUBSCRIPT roman_IIP end_POSTSUBSCRIPT ( italic_θ ) end_ARG(5.6)

In equations 5.5 and 5.6, "par“ and "fib“ refer to the parent sample and fiber-assigned samples respectively. The fiber-assigned Data-Data pairs are weighted by the inverse of the probability that the pair would be observed (PIP). Similarly, the fiber-assigned Data-Random pairs are weighted by the inverse of the probability that the data galaxy would be observed (IIP).

Finally, the bitwise PIP weights are calculated using the efficient estimator from Bianchi and Verde 2020 [65]:

wij=Nreal+1popcntwi(b)andwj(b)+1subscript𝑤𝑖𝑗subscript𝑁real1popcntsubscriptsuperscript𝑤𝑏𝑖andsubscriptsuperscript𝑤𝑏𝑗1w_{ij}=\frac{N_{\rm real}+1}{{\rm popcnt}\>{w^{(b)}_{i}{\rm and}\>w^{(b)}_{j}}%+1}italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT = divide start_ARG italic_N start_POSTSUBSCRIPT roman_real end_POSTSUBSCRIPT + 1 end_ARG start_ARG roman_popcnt italic_w start_POSTSUPERSCRIPT ( italic_b ) end_POSTSUPERSCRIPT start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT roman_and italic_w start_POSTSUPERSCRIPT ( italic_b ) end_POSTSUPERSCRIPT start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT + 1 end_ARG(5.7)

where wijsubscript𝑤𝑖𝑗w_{ij}italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT is the weighting applied to the pair of galaxies i𝑖iitalic_i and j𝑗jitalic_j, Nrealsubscript𝑁realN_{\rm real}italic_N start_POSTSUBSCRIPT roman_real end_POSTSUBSCRIPT is the number of AMTL realizations, and popcnt wi(b)andwj(b)subscriptsuperscript𝑤𝑏𝑖𝑎𝑛𝑑subscriptsuperscript𝑤𝑏𝑗w^{(b)}_{i}andw^{(b)}_{j}italic_w start_POSTSUPERSCRIPT ( italic_b ) end_POSTSUPERSCRIPT start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT italic_a italic_n italic_d italic_w start_POSTSUPERSCRIPT ( italic_b ) end_POSTSUPERSCRIPT start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT is the count of the bitwise "and“ operation between the bitweights of galaxies i and j. In the case of PIP weighting, the count of DD pairs is also normalized as in Equation 22 of Bianchi and Verde 2020 [65].

These weights are provided to pycorr by multiplicatively combining all non-bitwise weights and then adding them to a (list of) bitweight(s).

Figure5 shows the projected clustering results using the above methods for all tracer types which have a ’clustering‘ catalog produced from DESI SV3 Data.

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (11)

Figure6 shows the clustering amplitude of four tracers (BGS_ANY, LRG, ELG, and QSO) as a function of their separation in bins of distance parallel and perpendicular to the line of sight ( s||s_{||}italic_s start_POSTSUBSCRIPT | | end_POSTSUBSCRIPT and ssubscript𝑠perpendicular-tos_{\perp}italic_s start_POSTSUBSCRIPT ⟂ end_POSTSUBSCRIPT respectively).

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (12)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (13)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (14)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (15)

6 Results

In this section we display results from the first two runs of the AMTL pipeline, one run covering SV3, and the other run covering Y1. Statistics which we will show include a comparison of the number of targets in each class observed in the data and averaged over the AMTL runs, the estimation of completion from the AMTL pipeline, the difference in the completeness weights between the AMTL pipeline and methods which bootstrap completeness estimation from only the data, and finally the effects of weighting using PIP weights from this pipeline and other weighting methods on clustering statistics.

6.1 Alternate MTL statistics data

6.1.1 Comparison of number of observed targets by class

Since the AMTL pipeline functions by assigning real observation results from the DESI survey to alternate targets, we should end up with similar number of total observed targets between the average AMTL realization and in the data for the major target classes.

In table 5 we show the number of observed targets in the classes BGS_BRIGHT, LRG, ELG, and QSO in the SV3 Data and the Y1 Data as well as the sum of the PROB_OBS values for each of those target classes in the 128 realizations of the AMTLs.

Tracer type# ObservedSum of PROB_OBS%# ObservedSum of PROB_OBS%
in SV3in SV3differencein Y1in Y1difference
BGS_BRIGHT1477211489560.8423405642745061.0
LRG1270201285061.2247743925030301.0
ELG3543983584321.1398582940212080.9
QSO57060573410.5203649920512940.72

Table 5 reveals a small, but consistent surplus in the number of observations in the AMTLs vs the data. This difference is also present and of similar magnitude in the mocks. Since the mock "observation" sample is produced in the same way as a random AMTL realization (see table6), we believe this is reasonable behavior.

6.1.2 SV3 completeness estimation from AMTL

Figure7 shows the completeness of one of the SV3 rosettes (rosette 16; rosettes are described in §2 and [46]) as measured by the average PROB_OBS for targets of (left to right) LRGs, ELGs with QSO targets removed (ELG_notqso), and QSOs in RA/Dec bins within the rosette. The binning is varied in width based on the number of targets of each type with ELGs having the most targets and QSOs having the fewest. The tiling pattern of the rosette leads to the high, nearly 100% completeness in the intermediate parts of the rosette between 0.2 and 1.4 degrees from the center. Regions near the center of the rosette are subject to hardware failures more frequently since they are always covered by the same part of the focal plane and regions towards the exterior of the rosette are only tiled a few times.

In order to study the potential for the AMTL-derived bitweights to impact clustering relative to weighting derived from other measures of incompleteness, we will compare the AMTL-derived observation probabilities with those other measures of incompleteness, focusing our comparisons on the most central and furthest exterior regions where completeness is not 100%.

Figure8 shows the number density of targets within rosette 16 for each of the same groups of targets in the same order as Figure7. This is shown in units of number of targets per fiber patrol radius to better express the likelihood that a given bin in RA/Dec will experience fiber collisions. Focusing specifically on the upper left portion of the "ELGnotqso_full", we can see a correlation between regions of high number density and additional areas of low completeness where we can display the impact of the AMTL incompleteness weights.

Figures9 and 10 are the equivalents of figures 7 and 8 for rosette 1. We show this rosette as a counterpoint to the rosette 16 since this rosette has much less pronounced differences in areal target number density and represents a more typical rosette.

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (16)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (17)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (18)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (19)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (20)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (21)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (22)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (23)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (24)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (25)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (26)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (27)

Figures 11 and 12 show the mean value of the difference between PROB_OBS, the average probability that a target is observed as determined via the AMTL pipeline presented in this paper and FRACZ_TILELOCID101010The way SV3 catalogs were constructed [48], this is the most appropriate comparison. In the main survey LSS catalog processing, we have defined an additional column FRAC_TLOBS_TILES, which accounts for additional incompleteness and should be multiplied by FRACZ_TILELOCID; see [67]. , the fraction of similar objects within a DESI tile which were successfully observed and received a redshift. These differences show that a utility like the AMTL pipeline is necessary. A simple monte-carlo simulation of each target independently using what is, at first glance, a reasonable proxy for the probability that it is observed, can result in differences in probability for groups of objects on low completeness pixels of up to, occasionally exceeding, 20%.

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (28)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (29)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (30)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (31)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (32)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (33)

6.2 Alternate MTL statistics SV3 mocks

Tracer type# ’Observed‘Sum of PROB_OBS%
in SV3 mocksin SV3 mocksdifference
LRG1409521419950.73
ELG3867093889640.58
QSO60129603850.42

We run the AMTL pipeline over 25 mocks generated from the AbacusSummit simulations as described in §3, reproducing the same algorithm we apply on the observed data using 256 realizations instead of 128. The statistics averaged over the 25 mock realizations, as shown in Table 6, show that the number of targets marked as ’observed‘ in the mocks is 0.5%similar-toabsentpercent0.5\sim 0.5\%∼ 0.5 % smaller than the sum of the PROB_OBS in the mocks. Due to a larger number of available targets in the mocks, the mocks have significantly more observed targets than the data (shown in Table 5) at a 10%similar-toabsentpercent10\sim 10\%∼ 10 % level.

We show the same plots as for the data AMTLs but with a stacked set of all 25 mocks to show that the mock AMTL process successfully replicates the data PROB_OBS distribution as well as the difference between PROB_OBS and FRACZ_TILELOCID within the limits of the similarity between the mock and data target density. These plots are the 2D histograms of PROB_OBS (Figures 13 and 15), target number density (Figures 14 and 16), and the difference between PROB_OBS and FRACZ_TILELOCID (Figures 17 and 18).

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (34)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (35)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (36)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (37)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (38)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (39)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (40)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (41)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (42)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (43)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (44)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (45)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (46)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (47)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (48)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (49)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (50)

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (51)

6.3 SV3 2PCF results

Figures 19, 20, and 21 show the mean projected TPCF over 25 SV3 Abacus mocks with errorbar coming from the standard deviation over the 25 mocks in four cases. The first case, which we treat as truth is the projected TPCF parent catalog of all possible mock targets and is shown in black. The remaining cases are different weights on what was chosen to be the "observed“ set of targets. The "default“ weight (orange) includes the IIP completeness weight, a combination of the default weights and the angular upweighting of PB17 (green; [27] and §5.1 for more details), and the combination of the angular weights with the bitwise PIP weights discussed here (blue).

The inset on each plot shows the fractional difference between each weighting scheme with the "truth“ with the blue (PIP weighted) line having error bars equal to the ratio of the "truth“ errorbar and its value in each bin.

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (52)
Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (53)
Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (54)

These plots show that the bitweights produced by the AMTL method are able to recover the parent clustering well within 1σ1𝜎1-\sigma1 - italic_σ for all tracers down to 0.05 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT . Furthermore, the PIP weighting improves the recovery of the parent catalog clustering in all but a few separation bins.

The case of ELGs is most informative, since ELGs are the least complete target class. Clustering measured using only the default weighting is off by 20% uniformly down to a scale of nearly 1Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT . Additionally, the addition of angular upweights causes a spurious shift which pushes the 20% discrepancy down to a scale over 10 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT . The PIP weights when compbined with the above schemes shows agreement within a few percent between scales of 0.05 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT and 25 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT with agreement within 1σ1𝜎1-\sigma1 - italic_σ at all scales shown.

7 Discussion

Comparing the statistics of the data AMTL with the data shows a small, but persistent 1%similar-toabsentpercent1\sim 1\%∼ 1 % excess in the number of observed targets. The mocks also show a similar excess. This is likely due to the AMTL pipeline not fully accurately sampling observation results from the data for the alternate targets. As mentioned in section §4.4, we take observation results in the AMTL from the exact same fiber in the observations without regard to the target type which is assigned in the alternate realizations. In a future release of the AMTL pipeline we will have a fiber twin method implemented which will select observation results from a nearby fiber with nearly identical observational properties including target type.While the number of total observations shows some disagreements, the shape of the PROB_OBS distribution is recovered very well. For Rosette 16, the data PROB_OBS distributions in Figure7 and mock PROB_OBS distributions in Figure13 show nearly identical structure with the mocks showing much less fluctuation due to higher statistics and the data showing a small decrease in PROB_OBS towards the lower right of the rosette not present in the mocks due to a high target number density in that region. Rosette 1 also shows good agreement between Figures9 and 15 with the significantly higher PROB_OBS in the top left of the data rosette as compared to the mock coming from the very low data target number density.

The difference between PROB_OBS and the more generic FRACZ_TILELOCID in figures11, 12, 17, and 18 show similar patterns. There is very little difference between the two completeness metrics in the centers of both rosettes since completeness is at or near 100% at all locations. Towards the edges, we see a significant amount of fluctuation between the PROB_OBS and FRACZ_TILELOCID suggesting that completeness is somewhat different when measured through more approximate methods than an AMTL method.

Figures19, 20, and 21 show the validation of the AMTL method by using the bitweights obtained by using the method on mocks to recover the clustering of the parent mocks. Since the true redshift of all targets in the mock are known, this enables us to validate against a known "truth“ projected two point correlation function.

The validation results show that the bitwise weights calculated from the AMTL method are a strict improvement over the default completeness weights at small scales. For all three dark time tracers, the (blue) PIP+ANG upweighted mock projected two point correlation functions are an improvement of the default completeness weights at nearly all small ( 1absent1\leq 1≤ 1Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ) scales. Furthermore, down to scales of 0.02, 0.01, and 0.1 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT (for LRGs, ELGs, and QSOs respectively), the PIP+ANG upweighted clustering from the mocks is consistent with the parent mock clustering within the scatter among the 25 mocks. These results, especially the ELG clustering, show that the PIP weighting and angular upweighting are both necessary to correct the clustering measurements for incompleteness. For a sample like SV3 which is highly complete, IIP weights function similarly to nearest neighbor weights, indicating that those weights would be insufficient for incompleteness correction at small scales.

The larger scale clustering (s1𝑠1s\geq 1italic_s ≥ 1 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT and especially s10𝑠10s\geq 10italic_s ≥ 10Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ), while statistically consistent with the parent clustering within the mock variance, is not as well recovered. However, there are several reasons why this should be the case. First, the SV3 rosette design strongly limits the number of pairs which we can observe at separations nearing 100 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT . The size of an individual rosette at a redshift z=1.0 is only 70similar-toabsent70\sim 70∼ 70Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT . While DESI did place some rosettes adjacent to each other enabling the observation of some pairs at those separations, the vast majority of rosettes were isolated.

In conclusion, we have demonstrated the generation of 128 realizations of DESI SV3 and Y1 of the main survey using a nearly-maximally-realistic method utilizing alternative merged target ledgers. We have validated the AMTL method on 256 realizations of DESI SV3-like mocks showing the recovery of clustering down to scales of 0.01 Mpc h1superscript1h^{-1}italic_h start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT for the least complete tracer, Emission Line Galaxies.

Acknowledgements

JL and RK acknowledge support of the U.S. Department of Energy (DOE) in funding grant DE-SC0010129 for the work in this paper. Computational resources for JL were provided by SMU’s Center for Research Computing. This work made use of Astropy:111111http://www.astropy.org a community-developed core Python package and an ecosystem of tools and resources for astronomy [68, 69, 70]ADM was supported by the U.S.Department of Energy, Office of Science, Office of High Energy Physics, under Award Number DE-SC0019022.

This material is based upon work supported by the U.S. Department of Energy (DOE), Office of Science, Office of High-Energy Physics, under Contract No. DE–AC02–05CH11231, and by the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility under the same contract. Additional support for DESI was provided by the U.S. National Science Foundation (NSF), Division of Astronomical Sciences under Contract No. AST-0950945 to the NSF’s National Optical-Infrared Astronomy Research Laboratory; the Science and Technology Facilities Council of the United Kingdom; the Gordon and Betty Moore Foundation; the Heising-Simons Foundation; the French Alternative Energies and Atomic Energy Commission (CEA); the National Council of Humanities, Science and Technology of Mexico (CONAHCYT); the Ministry of Science and Innovation of Spain (MICINN), and by the DESI Member Institutions: https://www.desi.lbl.gov/collaborating-institutions. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the U. S. National Science Foundation, the U. S. Department of Energy, or any of the listed funding agencies.

The authors are honored to be permitted to conduct scientific research on Iolkam Du’ag (Kitt Peak), a mountain with particular significance to the Tohono O’odham Nation.

8 Data Availability

The SV3 data used in this analysis is currently availabile to the public at https://data.desi.lbl.gov/doc/releases/edr/ .The Y1 data used in this analysis will be made public along the Data Release 1 (details in https://data.desi.lbl.gov/doc/releases/)

References

  • York etal. [2000]DonaldG. York, J.Adelman, Jr. Anderson, JohnE., and SDSSCollaboration.The Sloan Digital Sky Survey: Technical Summary.AJ, 120(3):1579–1587, September 2000.doi: 10.1086/301513.
  • Dawson etal. [2013]KyleS. Dawson, DavidJ. Schlegel, ChristopherP. Ahn, etal.The Baryon Oscillation Spectroscopic Survey of SDSS-III.AJ, 145(1):10, January 2013.doi: 10.1088/0004-6256/145/1/10.
  • Dawson etal. [2016]KyleS. Dawson, Jean-Paul Kneib, WillJ. Percival, etal.The SDSS-IV Extended Baryon Oscillation Spectroscopic Survey:Overview and Early Data.AJ, 151(2):44, February 2016.doi: 10.3847/0004-6256/151/2/44.
  • DESI Collaboration etal. [2022]DESI Collaboration etal.Overview of the Instrumentation for the Dark Energy SpectroscopicInstrument.AJ, 164(5):207, November 2022.doi: 10.3847/1538-3881/ac882b.
  • Schlafly etal. [2023]EdwardF. Schlafly, David Kirkby, DavidJ. Schlegel, etal.Survey Operations for the Dark Energy Spectroscopic Instrument.AJ, 166(6):259, December 2023.doi: 10.3847/1538-3881/ad0832.
  • Kirkby et al. [2024]Kirkby et al., in preparation, 2024.
  • Blanton etal. [2003]MichaelR. Blanton, Huan Lin, RobertH. Lupton, F.Miller Maley, NealYoung, Idit Zehavi, and Jon Loveday.An Efficient Targeting Strategy for Multiobject SpectrographSurveys: the Sloan Digital Sky Survey “Tiling” Algorithm.AJ, 125(4):2276–2286, April 2003.doi: 10.1086/344761.
  • Gao etal. [2023]Hongyu Gao, Y.P. Jing, Shanquan Gui, etal.The DESI One-Percent Survey: Constructing Galaxy-Halo Connectionsfor ELGs and LRGs Using Auto and Cross Correlations.ApJ, 954(2):207, September 2023.doi: 10.3847/1538-4357/ace90a.
  • Pearl etal. [2023]AlanN. Pearl, AndrewR. Zentner, JeffreyA. Newman, etal.The DESI One-Percent Survey: Evidence for Assembly Bias fromLow-Redshift Counts-in-Cylinders Measurements.arXiv e-prints, art. arXiv:2309.08675, September 2023.doi: 10.48550/arXiv.2309.08675.
  • Rocher etal. [2023]Antoine Rocher, Vanina Ruhlmann-Kleider, Etienne Burtin, etal.The DESI One-Percent survey: exploring the Halo OccupationDistribution of Emission Line Galaxies with ABACUSSUMMIT simulations.J. Cosmology Astropart. Phys, 2023(10):016, October 2023.doi: 10.1088/1475-7516/2023/10/016.
  • Smith etal. [2023]A.Smith, C.Grove, S.Cole, etal.Generating mock galaxy catalogues for flux-limited samples like theDESI Bright Galaxy Survey.arXiv e-prints, art. arXiv:2312.08792, December 2023.doi: 10.48550/arXiv.2312.08792.
  • Yuan etal. [2023a]Sihan Yuan, Hanyu Zhang, AshleyJ. Ross, etal.The DESI One-Percent Survey: Exploring the Halo OccupationDistribution of Luminous Red Galaxies and Quasi-Stellar Objects withAbacusSummit.arXiv e-prints, art. arXiv:2306.06314, June2023a.doi: 10.48550/arXiv.2306.06314.
  • Yuan etal. [2023b]Sihan Yuan, RisaH. Wechsler, Yunchong Wang, etal.Unraveling emission line galaxy conformity atz~1 with DESI early data.arXiv e-prints, art. arXiv:2310.09329, October2023b.doi: 10.48550/arXiv.2310.09329.
  • Yu etal. [2024]Jiaxi Yu, Cheng Zhao, Violeta Gonzalez-Perez, etal.The DESI One-Percent Survey: exploring a generalized SHAM formultiple tracers with the UNIT simulation.MNRAS, 527(3):6950–6969, January 2024.doi: 10.1093/mnras/stad3559.
  • Gao etal. [2024]Hongyu Gao, Y.P. Jing, Kun Xu, etal.The DESI One-Percent Survey: A Concise Model for the GalacticConformity of Emission-line Galaxies.ApJ, 961(1):74, January 2024.doi: 10.3847/1538-4357/ad09d6.
  • Prada etal. [2023]F.Prada, J.Ereza, A.Smith, etal.The DESI One-Percent Survey: Modelling the clustering and halooccupation of all four DESI tracers with Uchuu.arXiv e-prints, art. arXiv:2306.06315, June 2023.doi: 10.48550/arXiv.2306.06315.
  • Anderson etal. [2012]Lauren Anderson, Eric Aubourg, Stephen Bailey, etal.The clustering of galaxies in the SDSS-III Baryon OscillationSpectroscopic Survey: baryon acoustic oscillations in the Data Release 9spectroscopic galaxy sample.MNRAS, 427(4):3435–3467, December 2012.doi: 10.1111/j.1365-2966.2012.22066.x.
  • Guo etal. [2012]Hong Guo, Idit Zehavi, and Zheng Zheng.A New Method to Correct for Fiber Collisions in Galaxy Two-pointStatistics.ApJ, 756(2):127, September 2012.doi: 10.1088/0004-637X/756/2/127.
  • Reid etal. [2014]BethA. Reid, Hee-Jong Seo, Alexie Leauthaud, JeremyL. Tinker, andMartin White.A 2.5 per cent measurement of the growth rate from small-scaleredshift space clustering of SDSS-III CMASS galaxies.MNRAS, 444(1):476–502, October 2014.doi: 10.1093/mnras/stu1391.
  • Anderson etal. [2014]Lauren Anderson, Éric Aubourg, Stephen Bailey, etal.The clustering of galaxies in the SDSS-III Baryon OscillationSpectroscopic Survey: baryon acoustic oscillations in the Data Releases 10and 11 Galaxy samples.MNRAS, 441(1):24–62, June 2014.doi: 10.1093/mnras/stu523.
  • Reid etal. [2016]Beth Reid, Shirley Ho, Nikhil Padmanabhan, etal.SDSS-III Baryon Oscillation Spectroscopic Survey Data Release 12:galaxy target selection and large-scale structure catalogues.MNRAS, 455(2):1553–1573, January 2016.doi: 10.1093/mnras/stv2382.
  • Ross etal. [2020]AshleyJ. Ross, Julian Bautista, Rita Tojeiro, etal.The Completed SDSS-IV extended Baryon Oscillation SpectroscopicSurvey: Large-scale structure catalogues for cosmological analysis.MNRAS, 498(2):2354–2371, October 2020.doi: 10.1093/mnras/staa2416.
  • Lyke etal. [2020]BradW. Lyke, AlexandraN. Higley, J.N. McLane, etal.The Sloan Digital Sky Survey Quasar Catalog: Sixteenth DataRelease.ApJS, 250(1):8, September 2020.doi: 10.3847/1538-4365/aba623.
  • Raichoor etal. [2021]Anand Raichoor, Arnaud de Mattia, AshleyJ. Ross, etal.The completed SDSS-IV extended Baryon Oscillation SpectroscopicSurvey: large-scale structure catalogues and measurement of the isotropic BAObetween redshift 0.6 and 1.1 for the Emission Line Galaxy Sample.MNRAS, 500(3):3254–3274, January 2021.doi: 10.1093/mnras/staa3336.
  • Bianchi and Percival [2017]Davide Bianchi and WillJ. Percival.Unbiased clustering estimation in the presence of missingobservations.MNRAS, 472(1):1106–1118, November 2017.doi: 10.1093/mnras/stx2053.
  • Mohammad etal. [2018]F.G. Mohammad, D.Bianchi, W.J. Percival, etal.The VIMOS Public Extragalactic Redshift Survey (VIPERS). Unbiasedclustering estimate with VIPERS slit assignment.A&A, 619:A17, November 2018.doi: 10.1051/0004-6361/201833853.
  • Percival and Bianchi [2017]WillJ. Percival and Davide Bianchi.Using angular pair upweighting to improve 3D clusteringmeasurements.MNRAS, 472(1):L40–L44, November 2017.doi: 10.1093/mnrasl/slx135.
  • Mohammad etal. [2020]FaizanG. Mohammad, WillJ. Percival, Hee-Jong Seo, etal.The completed SDSS-IV extended baryon oscillation spectroscopicsurvey: pairwise-inverse probability and angular correction for fibrecollisions in clustering measurements.MNRAS, 498(1):128–143, October 2020.doi: 10.1093/mnras/staa2344.
  • Levi etal. [2013]Michael Levi, Chris Bebek, Timothy Beers, Robert Blum, Robert Cahn,Daniel Eisenstein, Brenna Flaugher, Klaus Honscheid, Richard Kron,Ofer Lahav, Patrick McDonald, Natalie Roe, David Schlegel, andrepresenting the DESI collaboration.The DESI Experiment, a whitepaper for Snowmass 2013.arXiv e-prints, art. arXiv:1308.0847, August 2013.
  • DESI Collaborationetal. [2016a]DESI Collaboration etal.The DESI Experiment Part I: Science,Targeting, and Survey Design.arXiv e-prints, art. arXiv:1611.00036, October2016a.
  • DESI Collaborationetal. [2016b]DESI Collaboration etal.The DESI Experiment Part II: Instrument Design.arXiv e-prints, art. arXiv:1611.00037, October2016b.
  • DESI Collaboration [2025]DESI Collaboration.DESI 2024 I: Data Release 1 of the Dark Energy SpectroscopicInstrument.in preparation, 2025.
  • DESI Collaboration [2024a]DESI Collaboration.DESI 2024 II: Sample definitions, characteristics and two-pointclustering statistics.in preparation, 2024a.
  • DESI Collaboration etal. [2024a]DESI Collaboration etal.DESI 2024 III: Baryon Acoustic Oscillations from Galaxies andQuasars.arXiv e-prints, page arXiv:2404.03000, 2024a.doi: 10.48550/arXiv.2404.03000.
  • DESI Collaboration etal. [2024b]DESI Collaboration etal.DESI 2024 IV: Baryon Acoustic Oscillations from the Lyman AlphaForest.arXiv e-prints, page arXiv:2404.03001, 2024b.doi: 10.48550/arXiv.2404.03001.
  • DESI Collaboration [2024b]DESI Collaboration.DESI 2024 V: Analysis of the full shape of two-point clusteringstatistics from galaxies and quasars.in preparation, 2024b.
  • DESI Collaboration etal. [2024c]DESI Collaboration etal.DESI 2024 VI: Cosmological Constraints from the Measurements ofBaryon Acoustic Oscillations.arXiv e-prints, page arXiv:2404.03002, 2024c.doi: 10.48550/arXiv.2404.03002.
  • DESI Collaboration [2024c]DESI Collaboration.DESI 2024 VII: Cosmological constraints from full-shape analyses ofthe two-point clustering statistics measurements.in preparation, 2024c.
  • DESI Collaboration [2024d]DESI Collaboration.DESI 2024 VIII: Constraints on Primordial Non-Gaussianities.in preparation, 2024d.
  • Allende Prieto etal. [2020]Carlos Allende Prieto, AndrewP. Cooper, Arjun Dey, etal.Preliminary Target Selection for the DESI Milky Way Survey (MWS).Research Notes of the American Astronomical Society,4(10):188, October 2020.doi: 10.3847/2515-5172/abc1dc.
  • Ruiz-Macias etal. [2020]Omar Ruiz-Macias, Pauline Zarrouk, Shaun Cole, etal.Preliminary Target Selection for the DESI Bright Galaxy Survey(BGS).Research Notes of the American Astronomical Society,4(10):187, October 2020.doi: 10.3847/2515-5172/abc25a.
  • Zhou etal. [2020]Rongpu Zhou, JeffreyA. Newman, KyleS. Dawson, etal.Preliminary Target Selection for the DESI Luminous Red Galaxy (LRG)Sample.Research Notes of the American Astronomical Society,4(10):181, October 2020.doi: 10.3847/2515-5172/abc0f4.
  • Raichoor etal. [2020]Anand Raichoor, DanielJ. Eisenstein, Tanveer Karim, etal.Preliminary Target Selection for the DESI Emission Line Galaxy (ELG)Sample.Research Notes of the American Astronomical Society,4(10):180, October 2020.doi: 10.3847/2515-5172/abc078.
  • Yèche etal. [2020]Christophe Yèche, Nathalie Palanque-Delabrouille, Charles-AntoineClaveau, etal.Preliminary Target Selection for the DESI Quasar (QSO) Sample.Research Notes of the American Astronomical Society,4(10):179, October 2020.doi: 10.3847/2515-5172/abc01a.
  • Dey etal. [2019]Arjun Dey, DavidJ. Schlegel, Dustin Lang, etal.Overview of the DESI Legacy Imaging Surveys.AJ, 157(5):168, May 2019.doi: 10.3847/1538-3881/ab089d.
  • DESI Collaboration etal. [2023a]DESI Collaboration etal.Validation of the Scientific Program for the Dark EnergySpectroscopic Instrument.arXiv e-prints, art. arXiv:2306.06307, June2023a.doi: 10.48550/arXiv.2306.06307.
  • Guy etal. [2023]J.Guy, S.Bailey, A.Kremin, etal.The Spectroscopic Data Processing Pipeline for the Dark EnergySpectroscopic Instrument.AJ, 165(4):144, April 2023.doi: 10.3847/1538-3881/acb212.
  • DESI Collaboration etal. [2023b]DESI Collaboration etal.The Early Data Release of the Dark Energy Spectroscopic Instrument.arXiv e-prints, art. arXiv:2306.06308, June2023b.doi: 10.48550/arXiv.2306.06308.
  • Hardin, R. H., Sloane, N. J. A., and Smith, W. D. 2001, Tables ofSpherical Codes with Icosahedral Symmetry [Florham Park: AT&T ShannonLab.]Hardin, R. H., Sloane, N. J. A., and Smith, W. D. 2001, Tables of SphericalCodes with Icosahedral Symmetry (Florham Park: AT&T Shannon Lab.).Accessed 02-12-2024.
  • C.Zhao et al. [2024]C.Zhao et al.Mock catalogues with survey realism for the DESI DR1.in preparation, 2024.
  • D.Bianchi et al. [2024]D.Bianchi et al.Characterization of DESI fiber assignment incompleteness effect on2-point clustering and mitigation methods for 2024 analysis.in preparation, 2024.
  • Maksimova etal. [2021]NinaA Maksimova, LehmanH Garrison, DanielJ Eisenstein, etal.AbacusSummit: a massive set of high-accuracy, high-resolution N-bodysimulations.Monthly Notices of the Royal Astronomical Society,508(3):4017–4037, 09 2021.ISSN 0035-8711.doi: 10.1093/mnras/stab2484.URL https://doi.org/10.1093/mnras/stab2484.
  • Garrison etal. [2021]LehmanH Garrison, DanielJ Eisenstein, Douglas Ferrer, etal.The abacus cosmological N-body code.Monthly Notices of the Royal Astronomical Society,508(1):575–596, 09 2021.ISSN 0035-8711.doi: 10.1093/mnras/stab2482.URL https://doi.org/10.1093/mnras/stab2482.
  • Planck Collaboration etal. [2016]Planck Collaboration etal.Planck 2015 results - xiii. cosmological parameters.A&A, 594:A13, 2016.doi: 10.1051/0004-6361/201525830.URL https://doi.org/10.1051/0004-6361/201525830.
  • Rocher etal. [2023]Antoine Rocher, Vanina Ruhlmann-Kleider, Etienne Burtin, etal.The desi one-percent survey: exploring the halo occupationdistribution of emission line galaxies with abacussummit simulations.Journal of Cosmology and Astroparticle Physics, 2023(10):016, oct 2023.doi: 10.1088/1475-7516/2023/10/016.URL https://dx.doi.org/10.1088/1475-7516/2023/10/016.
  • Yuan etal. [2023c]Sihan Yuan, Hanyu Zhang, AshleyJ. Ross, etal.The DESI One-Percent Survey: Exploring the Halo OccupationDistribution of Luminous Red Galaxies and Quasi-Stellar Objects withAbacusSummit.arXiv e-prints, art. arXiv:2306.06314, June2023c.doi: 10.48550/arXiv.2306.06314.
  • Bianchi etal. [2018]Davide Bianchi, Angela Burden, WillJ. Percival, etal.Unbiased clustering estimates with the DESI fibre assignment.MNRAS, 481(2):2338–2348, December 2018.doi: 10.1093/mnras/sty2377.
  • Smith etal. [2019]Alex Smith, Jian-hua He, Shaun Cole, etal.Correcting for fibre assignment incompleteness in the DESI BrightGalaxy Survey.MNRAS, 484(1):1285–1300, March 2019.doi: 10.1093/mnras/stz059.
  • Myers etal. [2023]AdamD. Myers, John Moustakas, Stephen Bailey, etal.The Target-selection Pipeline for the Dark Energy SpectroscopicInstrument.AJ, 165(2):50, February 2023.doi: 10.3847/1538-3881/aca5f9.
  • Raichoor et al. [2024]Raichoor et al.2024.in preparation.
  • Landy and Szalay [1993]StephenD. Landy and AlexanderS. Szalay.Bias and Variance of Angular Correlation Functions.ApJ, 412:64, July 1993.doi: 10.1086/172900.
  • Keihänen etal. [2019]E.Keihänen, H.Kurki-Suonio, V.Lindholm, etal.Estimating the galaxy two-point correlation function using a splitrandom catalog.A&A, 631:A73, November 2019.doi: 10.1051/0004-6361/201935828.
  • Sinha and Garrison [2020]Manodeep Sinha and LehmanH. Garrison.CORRFUNC - a suite of blazing fast correlation functions on theCPU.MNRAS, 491(2):3022–3041, Jan 2020.doi: 10.1093/mnras/stz3157.
  • Sinha and Garrison [2019]Manodeep Sinha and Lehman Garrison.Corrfunc: Blazing fast correlation functions with avx512f simdintrinsics.In Amit Majumdar and Ritu Arora, editors, Software Challengesto Exascale Computing, pages 3–20, Singapore, 2019. Springer Singapore.ISBN 978-981-13-7729-7.URL https://doi.org/10.1007/978-981-13-7729-7_1.
  • Bianchi and Verde [2020]Davide Bianchi and Licia Verde.Confronting missing observations with probability weights: Fourierspace and generalized formalism.MNRAS, 495(1):1511–1529, June 2020.doi: 10.1093/mnras/staa1267.
  • Feldman etal. [1994]HumeA. Feldman, Nick Kaiser, and JohnA. Peaco*ck.Power-Spectrum Analysis of Three-dimensional Redshift Surveys.ApJ, 426:23, May 1994.doi: 10.1086/174036.
  • A.Ross et al. [2024]A.Ross et al.Construction of Large-scale Structure Catalogs for the Dark EnergySpectroscopic Instrument.in preparation, 2024.
  • Astropy Collaboration etal. [2013]Astropy Collaboration etal.Astropy: A community Python package for astronomy.A&A, 558:A33, October 2013.doi: 10.1051/0004-6361/201322068.
  • Astropy Collaboration etal. [2018]Astropy Collaboration etal.The Astropy Project: Building an Open-science Project and Status ofthe v2.0 Core Package.AJ, 156(3):123, September 2018.doi: 10.3847/1538-3881/aabc4f.
  • Astropy Collaboration etal. [2022]Astropy Collaboration etal.The Astropy Project: Sustaining and Growing a Community-orientedOpen-source Project and the Latest Major Release (v5.0) of the Core Package.ApJ, 935(2):167, August 2022.doi: 10.3847/1538-4357/ac7c74.

Appendix A Author Affiliations

{hangparas}

.5cm1

1Department of Physics, Southern Methodist University, 3215 Daniel Avenue, Dallas, TX 75275, USA

2Departamento de Astrofísica, Universidad de La Laguna (ULL), E-38206, La Laguna, Tenerife, Spain

3Instituto de Astrofísica de Canarias, C/ Vía Láctea, s/n, E-38205 La Laguna, Tenerife, Spain

4Department of Physics & Astronomy, University of Wyoming, 1000 E. University, Dept.3905, Laramie, WY 82071, USA

5Center for Cosmology and AstroParticle Physics, The Ohio State University, 191 West Woodruff Avenue, Columbus, OH 43210, USA

6Department of Astronomy, The Ohio State University, 4055 McPherson Laboratory, 140 W 18th Avenue, Columbus, OH 43210, USA

7The Ohio State University, Columbus, 43210 OH, USA

8Dipartimento di Fisica “Aldo Pontremoli”, Università degli Studi di Milano, Via Celoria 16, I-20133 Milano, Italy

9University of Michigan, Ann Arbor, MI 48109, USA

10IRFU, CEA, Université Paris-Saclay, F-91191 Gif-sur-Yvette, France

11Department of Physics and Astronomy, University of Waterloo, 200 University Ave W, Waterloo, ON N2L 3G1, Canada

12Perimeter Institute for Theoretical Physics, 31 Caroline St. North, Waterloo, ON N2L 2Y5, Canada

13Waterloo Centre for Astrophysics, University of Waterloo, 200 University Ave W, Waterloo, ON N2L 3G1, Canada

14Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA 94720, USA

15Physics Dept., Boston University, 590 Commonwealth Avenue, Boston, MA 02215, USA

16Institute for Computational Cosmology, Department of Physics, Durham University, South Road, Durham DH1 3LE, UK

17Department of Physics & Astronomy, University College London, Gower Street, London, WC1E 6BT, UK

18Instituto de Física, Universidad Nacional Autónoma de México, Cd. de México C.P. 04510, México

19Department of Astronomy, School of Physics and Astronomy, Shanghai Jiao Tong University, Shanghai 200240, China

20Kavli Institute for Particle Astrophysics and Cosmology, Stanford University, Menlo Park, CA 94305, USA

21SLAC National Accelerator Laboratory, Menlo Park, CA 94305, USA

22Departamento de Física, Universidad de los Andes, Cra. 1 No. 18A-10, Edificio Ip, CP 111711, Bogotá, Colombia

23Observatorio Astronómico, Universidad de los Andes, Cra. 1 No. 18A-10, Edificio H, CP 111711 Bogotá, Colombia

24Institut d’Estudis Espacials de Catalunya (IEEC), 08034 Barcelona, Spain

25Institute of Cosmology and Gravitation, University of Portsmouth, Dennis Sciama Building, Portsmouth, PO1 3FX, UK

26Institute of Space Sciences, ICE-CSIC, Campus UAB, Carrer de Can Magrans s/n, 08913 Bellaterra, Barcelona, Spain

27Fermi National Accelerator Laboratory, PO Box 500, Batavia, IL 60510, USA

28Department of Physics, The Ohio State University, 191 West Woodruff Avenue, Columbus, OH 43210, USA

29School of Mathematics and Physics, University of Queensland, 4072, Australia

30NSF NOIRLab, 950 N. Cherry Ave., Tucson, AZ 85719, USA

31Sorbonne Université, CNRS/IN2P3, Laboratoire de Physique Nucléaire et de Hautes Energies (LPNHE), FR-75005 Paris, France

32Departament de Física, Serra Húnter, Universitat Autònoma de Barcelona, 08193 Bellaterra (Barcelona), Spain

33Institut de Física d’Altes Energies (IFAE), The Barcelona Institute of Science and Technology, Campus UAB, 08193 Bellaterra Barcelona, Spain

34Institució Catalana de Recerca i Estudis Avançats, Passeig de Lluís Companys, 23, 08010 Barcelona, Spain

35Department of Physics and Astronomy, Siena College, 515 Loudon Road, Loudonville, NY 12211, USA

36Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, U.K

37National Astronomical Observatories, Chinese Academy of Sciences, A20 Datun Rd., Chaoyang District, Beijing, 100012, P.R. China

38Departamento de Física, Universidad de Guanajuato - DCI, C.P. 37150, Leon, Guanajuato, México

39Instituto Avanzado de Cosmología A.C., San Marcos 11 - Atenas 202. Magdalena Contreras, 10720. Ciudad de México, México

40Korea Astronomy and Space Science Institute, 776, Daedeokdae-ro, Yuseong-gu, Daejeon 34055, Republic of Korea

41Space Sciences Laboratory, University of California, Berkeley, 7 Gauss Way, Berkeley, CA 94720, USA

42University of California, Berkeley, 110 Sproul Hall #5800 Berkeley, CA 94720, USA

43Instituto de Astrofísica de Andalucía (CSIC), Glorieta de la Astronomía, s/n, E-18008 Granada, Spain

44Department of Physics, Kansas State University, 116 Cardwell Hall, Manhattan, KS 66506, USA

45Department of Physics and Astronomy, Sejong University, Seoul, 143-747, Korea

46CIEMAT, Avenida Complutense 40, E-28040 Madrid, Spain

47Department of Physics, University of Michigan, Ann Arbor, MI 48109, USA

48Department of Physics & Astronomy, Ohio University, Athens, OH 45701, USA

Production of Alternate Realizations of DESI Fiber Assignment for Unbiased Clustering Measurement in Data and Simulations (2024)
Top Articles
Latest Posts
Article information

Author: Greg Kuvalis

Last Updated:

Views: 6659

Rating: 4.4 / 5 (75 voted)

Reviews: 90% of readers found this page helpful

Author information

Name: Greg Kuvalis

Birthday: 1996-12-20

Address: 53157 Trantow Inlet, Townemouth, FL 92564-0267

Phone: +68218650356656

Job: IT Representative

Hobby: Knitting, Amateur radio, Skiing, Running, Mountain biking, Slacklining, Electronics

Introduction: My name is Greg Kuvalis, I am a witty, spotless, beautiful, charming, delightful, thankful, beautiful person who loves writing and wants to share my knowledge and understanding with you.