Planets around low-mass stars (PALMS). IV. the outer architecture of M dwarf planetary systems

We present results from a high-contrast adaptive optics imaging search for giant planets and brown dwarfs (≳1 M _Jup) around 122 newly identified nearby (≲40 pc) young M dwarfs. Half of our targets are younger than 135 Myr and 90% are younger than the Hyades (620 Myr). After removing 44 close stellar binaries (implying a stellar companion fraction of >35.4% ± 4.3% within 100 AU), 27 of which are new or spatially resolved for the first time, our remaining sample of 78 single M dwarfs makes this the largest imaging search for planets around young low-mass stars (0.1-0.6 M _⊙) to date. Our H- and K-band coronagraphic observations with Keck/NIRC2 and Subaru/HiCIAO achieve typical contrasts of 12-14 mag and 9-13 mag at 1″, respectively, which correspond to limiting planet masses of 0.5-10 M _Jup at 5-33 AU for 85% of our sample. We discovered four young brown dwarf companions: 1RXS J235133.3+312720 B (32 ± 6 M _Jup;L0; 120 ± 20 AU), GJ 3629 B (64 M _Jup; M7.5 ± 0.5; 6.5 ± 0.5 AU), 1RXS J034231.8+121622 B (35 ± 8 M _Jup; L0 ± 1; 19.8 ± 0.9 AU), and 2MASS J15594729+4403595 B (43 ± 9 M _Jup; M8.0 ± 0.5; 190 ± 20 AU). Over 150 candidate planets were identified; we obtained follow-up imaging for 56% of these but all are consistent with background stars. Our null detection of planets enables strong statistical constraints on the occurrence rate of long-period giant planets around single M dwarfs. We infer an upper limit (at the 95% confidence level) of 10.3% and 16.0% for 1-13 M _Jup planets between 10-100 AU for hot-start and cold-start (Fortney) evolutionary models, respectively. Fewer than 6.0% (9.9%) of M dwarfs harbor massive gas giants in the 5-13 M _Jup range like those orbiting HR 8799 and β Pictoris between 10-100 AU for a hot-start (cold-start) formation scenario. The frequency of brown dwarf (13-75 M _Jup) companions to single M dwarfs between 10-100 AU is 2.8%. Altogether we find that giant planets, especially massive ones, are rare in the outskirts of M dwarf planetary systems. Although the first directly imaged planets were found around massive stars, there is currently no statistical evidence for a trend of giant planet frequency with stellar host mass at large separations as predicted by the disk instability model of giant planet formation.