Carr was born in Thirsk Junction, Carlton Miniott, Yorkshire, into a Wesleyan Methodist family. His father Joseph, the eleventh son of a farmer, went to work for the railways, eventually becoming a station master for the North Eastern Railway. Carr was given the same Christian name as his father and the middle name Lloyd, after David Lloyd George, the Liberal Chancellor of the Exchequer. He adopted the names Jim and James in adulthood. His brother Raymond, who was also a station master, called him Lloyd.

Carr's early life was shaped by failure. He attended the village school at Carlton Miniott. He failed the scholarship exam, which denied him a grammar school education, and on finishing his school career he also failed to gain admission to teacher training college. Interviewed at Goldsmiths' College, London, he was asked why he wanted to be a teacher. Carr answered: "Because it leaves so much time for other pursuits." He was not accepted. Over forty years later, after his novel The Harpole Report was a critical and popular success, he was invited to give a talk at Goldsmiths'. He replied that the college once had its chance of being addressed by him.
He worked for a year as an unqualified teacher — one of the lowest of the low in English education — at South Milford Primary School, where he became involved in a local amateur football team which was startlingly successful that year. This experience he developed into the novel How Steeple Sinderby Wanderers Won the F.A. Cup. He then successfully applied to a teacher training college in Dudley. In 1938 he took a year out from his teaching career to work as an exchange teacher in Huron, South Dakota in the Great Plains. Much of the year was a struggle to survive in what was a strangely different culture to him; his British salary converted into dollars was pitifully inadequate to meet American costs of living. This experience gave rise to his novel The Battle of Pollocks Crossing.

At the end of his year in the USA Carr continued his journey westward and found himself travelling through the Middle East and the Mediterranean as the Second World War loomed. He arrived in France in September 1939 and reached England, where he volunteered for service in the Royal Air Force. He was trained as an RAF photographer and stationed in West Africa, later serving in Britain as an intelligence officer, an experience he translated into fiction with A Season in Sinji.

At the end of the War he married Sally (Hilda Gladys Sexton) and returned to teaching. He was appointed headmaster of Highfields Primary School in Kettering, Northamptonshire, a post he filled from 1952 to 1967 in a typically idiosyncratic way which earned the devotion of staff and pupils alike. He returned to Huron, South Dakota, in 1957 to teach again on an exchange visit, when he wrote and published himself a social history of The Old Timers of Beadle County.

In 1967, having written two novels, he retired from teaching to devote himself to writing. He produced and published from his own Quince Tree Press a series of 'small books' designed to fit into a pocket: some of them selections from English poets, others brief monographs about historical events, or works of reference. In order to encourage children to read, each of the "small books" was given two prices, the lower of which applied only to children. As a result, Carr received several letters from adults in deliberately childish writing in an attempt to secure the discount.

He also carried on a single-handed campaign to preserve and restore the parish church of Saint Faith at Newton in the Willows, which had been vandalised and was threatened with redundancy. Carr, who appointed himself its guardian, came into conflict with the vicar of the benefice, and higher church authorities, in his attempts to save the church. The building was saved, but his crusade was also a failure in that redundancy was not averted and the building is now a scientific study centre.