Quincy Jones, a titan of American entertainment who worked with stars from Frank Sinatra to Michael Jackson and Will Smith, has died aged 91.

Jones’ publicist, Arnold Robinson, said he died on Sunday night at his home in the Bel Air section of Los Angeles, surrounded by his family.

“Tonight, with full but broken hearts, we must share the news of our father and brother Quincy Jones’ passing,” the family said in a statement. “And although this is an incredible loss for our family, we celebrate the great life that he lived and know there will never be another like him.”

Jones was arguably the most versatile pop cultural figure of the 20th century, perhaps best known for producing the albums Off the Wall, Thriller and Bad for Michael Jackson in the 1980s, which made the singer the biggest pop star of all time. Jones also produced music for Sinatra, Aretha Franklin, Donna Summer and many others.

He was also a successful composer of dozens of film scores, and had numerous chart hits under his own name. Jones was a bandleader in big band jazz, an arranger for jazz stars including Count Basie, and a multi-instrumentalist, most proficiently on trumpet and piano. His TV and film production company, founded in 1990, had major success with the sitcom The Fresh Prince of Bel-Air and other shows, and he continued to innovate well into his 80s, launching Qwest TV in 2017, an on-demand music TV service. Jones is third only to Beyoncé and Jay-Z for having the most Grammy award nominations of all time – 80 to their 88 each – and is the awards’ third most-garlanded winner, with 28.

Among those paying tribute to Jones was Joe Biden, who called him a “musical genius who transformed the soul of America – one beat, one rhythm, and one rhyme at a time … Quincy Jones solidified Black culture as American culture”.

Playwright and actor Jeremy O Harris paid tribute to Jones’s “limitless” contributions to US culture, writing: “What couldn’t he do? Quincy Jones, literally born when the limits on how big a black boy could dream were unfathomably high, taught us that the limit does not exist.”

Actor Michael Caine was born on the same day as Jones – 14 March 1933 – and wrote: “My celestial twin Quincy was a titan in the musical world. He was a wonderful and unique human being, lucky to have known him.”

Jones was born in Chicago. His half-white father had been born to a Welsh slave owner and one of his female slaves, while his mother’s family were also descended from slave owners. His introduction to music came through the walls of his childhood home from a piano played by a neighbour, which he started learning aged seven, and via his mother’s singing.

His parents divorced and he moved with his father to Washington state, where Jones learned drums and a host of brass instruments in his high-school band. At 14, he started playing in a band with a 16-year-old Ray Charles in Seattle clubs, once, in 1948, backing Billie Holiday. He studied music at Seattle University, transferring east to continue in Boston, and then moved to New York after being rehired by the jazz bandleader Lionel Hampton, with whom he had toured as a high-schooler (a band for which Malcolm X was a heroin dealer when they played in Detroit).

In New York, one early gig was playing trumpet in Elvis Presley’s band for his first TV appearances, and he met the stars of the flourishing bebop movement including Charlie Parker and Miles Davis. (Years later, in 1991, Jones conducted Davis’s last performance, two months before he died.)

Jones toured Europe with Hampton, and spent much time there in the 1950s, including a period furthering his studies in Paris, where he met luminaries including Pablo Picasso, James Baldwin and Josephine Baker. At the age of 23, he also toured South America and the Middle East as Dizzy Gillespie’s musical director and arranger. He convened a crack team for his own big band, touring Europe as a way to test Free and Easy, a jazz musical, but the disastrous run left Jones, by his own admission, close to suicide and with $100,000 of debt.

He secured a job at Mercury Records and slowly paid off the debt with plenty of work as a producer and arranger for artists including Ella Fitzgerald, Dinah Washington, Peggy Lee, Sarah Vaughan and Sammy Davis Jr. He also began scoring films, his credits eventually including The Italian Job, In the Heat of the Night, The Getaway and The Color Purple. (He produced the last of these, which was nominated for 11 Oscars, three for Jones himself.) In 1968, he became the first African American to be nominated for best original song at the Oscars, for The Eyes of Love from the film Banning (alongside songwriter Bob Russell); he had seven nominations in total. For TV, he scored programmes such as The Bill Cosby Show, Ironside and Roots.

His work with Sinatra began in 1958 when he was hired to conduct and arrange for Sinatra and his band by Grace Kelly, princess consort of Monaco, for a charity event. Jones and Sinatra continued working on projects until Sinatra’s final album, LA Is My Lady, in 1984. Jones’s solo musical career took off in the late 1950s, recording albums under his own name as bandleader for jazz ensembles that included luminaries such as Charles Mingus, Art Pepper and Freddie Hubbard.

Jones once said of his time in Seattle: “When people write about the music, jazz is in this box, R&B is in this box, pop is in this box, but we did everything,” and his catholic tastes served him well as modern pop mutated out of the swing era. He produced four million-selling hits for the New York singer Lesley Gore in the mid-60s, including the US No 1 It’s My Party, and later embraced funk and disco, producing hit singles including George Benson’s Give Me the Night and Patti Austin and James Ingram’s Baby Come to Me, along with records by the band Rufus and Chaka Khan, and the Brothers Johnson. Jones also released his own funk material, scoring US Top 10 albums with Body Heat (1974) and The Dude (1981).

His biggest success in this style was his work with Michael Jackson: Thriller remains the biggest selling album of all time, while Jones’s versatility between Off the Wall and Bad allowed Jackson to metamorphose from lithe disco to ultra-synthetic funk-rock. He and Jackson (along with Lionel Richie and producer Michael Omartian) also helmed We Are the World, a successful charity single that raised funds for famine relief in Ethiopia in 1985. “I’ve lost my little brother today, and part of my soul has gone with him,” Jones said when Jackson died in 2009. In 2017, Jones’s legal team successfully argued that he was owed $9.4m in unpaid Jackson royalties, though he lost on appeal in 2020 and had to return $6.8m.

After the success of The Color Purple in 1985, he formed the film and TV production company Quincy Jones Entertainment in 1990. His biggest screen hit was the sitcom The Fresh Prince of Bel-Air, which ran for 148 episodes and launched the career of Will Smith; other shows included the LL Cool J sitcom In the House and the long-running sketch comedy show MadTV.

He also created the media company Qwest Broadcasting and in 1993, the Black music magazine Vibe in partnership with Time Inc. Throughout his career he supported numerous charities and causes, including the , National Association for the Advancement of Colored People, the Jazz Foundation of America and others, and mentored young musicians including the British multiple Grammy winner Jacob Collier.

Jones’ illustrious career was twice nearly cut short: he narrowly avoided being killed by Charles Manson’s cult in 1969, having planned to go to Sharon Tate’s house on the night of the murders there, but Jones forgot the appointment. He also survived a brain aneurysm in 1974 that prevented him from playing the trumpet again in case the exertion caused further harm.

Jones was married three times, first to his high-school girlfriend Jeri Caldwell, for nine years until 1966, fathering his daughter Jolie. In 1967, he married Ulla Andersson and had a son and daughter, divorcing in 1974 to marry actor Peggy Lipton, best known for roles in The Mod Squad and Twin Peaks. They had two daughters, including the actor Rashida Jones, before divorcing in 1989. He had two further children: Rachel, with a dancer, Carol Reynolds, and Kenya, his daughter with actor Nastassja Kinski.

He never remarried, but continued to date a string of younger women, raising eyebrows with his year-long partnership with 19-year-old Egyptian designer Heba Elawadi when he was 73. He has also claimed to have dated Ivanka Trump and Juliette Gréco. He is survived by his seven children.

Other artists paying tribute included LL Cool J, who wrote: “You were a father and example at a time when I truly needed a father and example. Mentor. Role model. King. You gave me opportunities and shared wisdom. Music would not be music without you.” Femi Koleoso, bandleader with Mercury prize-winning jazz group Ezra Collective, called Jones a “masterful musician and beautiful soul”.

Nile Rodgers of Chic wrote “rest in power”, describing Jones as a “leader, teacher, spirit, pioneer”.