Kilonova Luminosity Function Constraints Based on Zwicky Transient Facility Searches for 13 Neutron Star Merger Triggers during O3

We present a systematic search for optical counterparts to 13 gravitational wave (GW) triggers involving at least one neutron star during LIGO/Virgo's third observing run (O3). We searched binary neutron star (BNS) and neutron star black hole (NSBH) merger localizations with the Zwicky Transient Facility (ZTF) and undertook follow-up with the Global Relay of Observatories Watching Transients Happen (GROWTH) collaboration. The GW triggers had a median localization area of 4480 deg(2), a median distance of 267 Mpc, and false-alarm rates ranging from 1.5 to 10(-25) yr(-1). The ZTF coverage in the g and r bands had a median enclosed probability of 39%, median depth of 20.8 mag, and median time lag between merger and the start of observations of 1.5 hr. The O3 follow-up by the GROWTH team comprised 340 UltraViolet/Optical/InfraRed (UVOIR) photometric points, 64 OIR spectra, and three radio images using 17 different telescopes. We find no promising kilonovae (radioactivity-powered counterparts), and we show how to convert the upper limits to constrain the underlying kilonova luminosity function. Initially, we assume that all GW triggers are bona fide astrophysical events regardless of false-alarm rate and that kilonovae accompanying BNS and NSBH mergers are drawn from a common population; later, we relax these assumptions. Assuming that all kilonovae are at least as luminous as the discovery magnitude of GW170817 (-16.1 mag), we calculate that our joint probability of detecting zero kilonovae is only 4.2%. If we assume that all kilonovae are brighter than -16.6 mag (the extrapolated peak magnitude of GW170817) and fade at a rate of 1 mag day(-1) (similar to GW170817), the joint probability of zero detections is 7%. If we separate the NSBH and BNS populations based on the online classifications, the joint probability of zero detections, assuming all kilonovae are brighter than -16.6 mag, is 9.7% for NSBH and 7.9% for BNS mergers. Moreover, no more than <57% (<89%) of putative kilonovae could be brighter than -16.6 mag assuming flat evolution (fading by 1 mag day(-1)) at the 90% confidence level. If we further take into account the online terrestrial probability for each GW trigger, we find that no more than <68% of putative kilonovae could be brighter than -16.6 mag. Comparing to model grids, we find that some kilonovae must have M-ej M, X-lan > 10(-4), or > 30 degrees to be consistent with our limits. We look forward to searches in the fourth GW observing run; even 17 neutron star mergers with only 50% coverage to a depth of -16 mag would constrain the maximum fraction of bright kilonovae to <25%.