Gernsheim was born in Worms. He was given his first musical training at home under his mother's care, then starting from the age of seven under Worms' musical director, Louis Liebe, a former pupil of Louis Spohr. His father, a prominent Jewish physician, moved the family to Frankfurt am Main in the aftermath of the year of revolutions, 1848, where he studied with Edward Rosenhain, brother of Jakob Rosenhain. He made his first public appearance as a concert pianist in 1850 and toured for two seasons, then settled with his family in Leipzig, where he studied piano with Ignaz Moscheles from 1852. He spent the years 1855–1860 in Paris, meeting Gioachino Rossini, Édouard Lalo and Camille Saint-Saëns.