Pedersen was born in Osted, near Roskilde, on the Danish island of Zealand, the son of a church organist. As a child, Ørsted Pedersen played piano, but from the age of 13, he started learning to play upright bass and at the age of 14, while studying, he began his professional jazz career in Denmark with his first band, Jazzkvintet 60 (Danish for Jazz Quintet 60). By the age of fifteen, he had the ability to accompany leading musicians at nightclubs, working regularly at Copenhagen's Jazzhus Montmartre, after his debut there on New Year's Eve 1961, when he was only 15. When seventeen, he had already turned down an offer to join the Count Basie orchestra, mainly because he was too young to get legal permission to live and work as a musician in the United States.

The Montmartre was a regular stop-off for touring American Jazz stars, and as a member of the house band, the young Ørsted Pedersen performed with saxophonists such as Booker Ervin, Sonny Rollins, Dexter Gordon, Rahsaan Roland Kirk and Stan Getz, and pianist Bill Evans, with whom he toured in Europe in 1965. During the 1960s, Pedersen played with a series of other important American jazzmen who were touring or resident in Denmark, including Ben Webster, Brew Moore, Bud Powell, Count Basie, Roy Eldridge, Dizzy Gillespie, Jackie McLean, and vocalist Ella Fitzgerald.

He is perhaps best known for his extensive collaboration with Oscar Peterson from 1972 to 1987. His predecessor, Ray Brown, thought highly of the Dane and regarded him as the only upright bassist equal to the task of keeping up with the pianist. He was awarded Best Bass Player of the Year by DownBeat Critics' Poll in 1981.

Ørsted Pedersen worked in duo and trio arrangements with pianist Kenny Drew, recording over 50 albums together. He also worked with Stéphane Grappelli and Joe Pass and recorded extensively as a leader. His best known songs are "My Little Anna", "Jaywalkin'", and "The Puzzle", as well as jazz arrangements of traditional Danish folk songs. A duo performance with Rune Gustafsson at Vossajazz 1980, concluded on the album Just The Way You Are on the label Sonet Gramofon, recorded half a year after this first meeting. He was awarded the Nordic Council Music Prize in 1991.

Pedersen had a particular ability to interpret Danish songs and folk melodies. He often played within trio ensembles, partly collectively with the trumpeter Palle Mikkelborg and the keyboard player Kenneth Knudsen, and partly under his own name, usually with guitarists like Philip Catherine and Ulf Wakenius. In 1999, he co-led a duo with pianist Mulgrew Miller, touring Europe, Japan, Australia, and Korea. This format was later enlarged into a trio featuring drummer Alvin Queen. This trio remained intact until Pedersen's death. Ørsted Pedersen died of heart failure in 2005 at the age of 58 in Copenhagen, Denmark. He was survived by his wife, Solveig, and his three children.