2019A&A...627A..84L

With a total integration time of 168h and a narrowband filter tuned to Lyα emission from z=8.8, the UltraVISTA survey has set out to find some of the most distant galaxies, on the verge of the epoch of reionization. Previous calculations of the expected number of detected Lyα-emitting galaxies (LAEs) at this redshift based for example on extrapolation of lower-redshift luminosity functions did not explicitly take into account the radiative transfer of Lyα. In this work we have combined a theoretical model for the halo mass function, that is, the expected number of haloes per volume, with numerical results from high-resolution cosmological hydro-simulations post-processed with radiative transfer of ionizing UV and Lyα radiation, assessing the visibility of LAEs residing in these haloes. We have taken into account uncertainties such as cosmic variance and the anisotropic escape of Lyα, and predict that once the survey has finished, the probabilities of detecting none, one, or more than one are roughly 90%, 10%, and 1%, respectively. This is a significantly smaller success rate than in earlier predictions, due to the combined effect of a highly neutral intergalactic medium (IGM) scattering Lyα to such large distances from the galaxy that they fall outside the observational aperture, and to the actual depth of the survey being less than predicted. Because the IGM affects narrowband (NB) and broadband (BB) magnitudes differently, we argue for a relaxed colour selection criterion of m_NB-m_BB~=+0.85 in the AB system. Since the flux is dominated by the continuum, however, even if a galaxy is detectable in the NB, its probability of being selected as a narrowband excess object is ≤35%. Various additional properties of galaxies at this redshift are predicted, for example, the Lyα and UV luminosity functions, the stellar mass-halo mass relation, the spectral shape, the optimal aperture, as well as the anisotropic escape of Lyα through both the dusty, interstellar medium and through the partly neutral IGM. Finally, we describe and make public a fast numerical code for adding numbers with asymmetric uncertainties ("x^+σ+_–σ–") which proves significantly more precise than the standard, but wrong, way of separately adding upper and lower uncertainties in quadrature.