Ossie Davis was born Raiford Chatman Davis, a son of Kince Charles Davis, a railway construction engineer, and his wife Laura Cooper. The name Ossie came from a county clerk who misheard his mother's pronunciation of his initials "R.C." when he was born. Following the wishes of his parents, he attended Howard University but dropped out in 1939 to fulfill his acting career in New York; he later attended Columbia University School of General Studies. His acting career, which spanned seven decades, began in 1939 with the Rose McClendon Players in Harlem. He made his film debut in 1950 in the Sidney Poitier film No Way Out. He voiced Anansi the spider on the PBS children's television series Sesame Street in its animation segments.

Davis experienced many of the same struggles that most African American actors of his generation underwent; he wanted to act but he did not want to play stereotypical subservient roles, such as a butler, that was the standard for black actors of his generation. Instead, he tried to follow the example of Sidney Poitier and play more distinguished characters. When he found it necessary to play a Pullman porter or a butler, he tried to inject the role with a certain degree of dignity.

In 2003, both Ossie Davis and his wife Ruby Dee starred and narrated in the HBO film Unchained Memories, a tribute to the WPA slave narratives.