Germline mutations in BRCA1 and BRCA2 (BRCA1/2) genes considerably increase breast and ovarian cancer risk. Given that tumors with these mutations have elevated genomic instability, they exhibit relative vulnerability to certain chemotherapies and targeted treatments based on poly (ADP-ribose) polymerase (PARP) inhibition. However, the molecular mechanisms that influence cancer risk and therapeutic benefit or resistance remain only partially understood. BRCA1 and BRCA2 have also been implicated in the suppression of R-loops, triple-stranded nucleic acid structures composed of a DNA:RNA hybrid and a displaced ssDNA strand. Here, we report that loss of RNF168, an E3 ubiquitin ligase and DNA double-strand break (DSB) responder, remarkably protected Brca1-mutant mice against mammary tumorigenesis. We demonstrate that RNF168 deficiency resulted in accumulation of R-loops in BRCA1/2-mutant breast and ovarian cancer cells, leading to DSBs, senescence, and subsequent cell death. Using interactome assays, we identified RNF168 interaction with DHX9, a helicase involved in the resolution and removal of R-loops. Mechanistically, RNF168 directly ubiquitylated DHX9 to facilitate its recruitment to R-loop–prone genomic loci. Consequently, loss of RNF168 impaired DHX9 recruitment to R-loops, thereby abrogating its ability to resolve R-loops. The data presented in this study highlight a dependence of BRCA1/2-defective tumors on factors that suppress R-loops and reveal a fundamental RNF168-mediated molecular mechanism that governs cancer development and vulnerability.