The son of a glassware maker, Peter Joachim Fröhlich grew up in Germany as the Nazis rose to power. Escaping in 1938 with the rest of his family on the last boat of refugees admitted to Cuba, he gained entry with them to the United States two years later, whereupon he changed his name to Gay. He graduated from the University of Denver in 1946 and earned a master's and doctorate in history from Columbia University.

Gay taught at Columbia from 1947 until 1969. In 1969 he joined the faculty at Yale University, where he taught until he retired as Sterling Professor of History in 1993. He was a former director of the New York Public Library's Center for Scholars and Writers from 1997 until 2003. Gay was the recipient of numerous awards, including the National Book Award and the received the American Historical Association's (AHA) Award for Scholarly Distinction.