Infusing taste into LLM applications

Taste can be defined as the curatorial filter built from your experiences, preferences, empathy, and cultural consumption over a (normally) long period.

I have been working at the intersection of taste and Large Language Modoels and how to infuse taste into applications that integrate LLMs in their core product.

The task at hand is simple (at least that’s how it seemed at the start):

• A user signs up.
• Tells you about themselves.
• And then, you can recommend music based on what they have told you.

But this introduces new problems. For example,

If I listen to a Kendrick Lamar album today, does it mean that I only like hip-hop?

In reality, the above is never true, (even by looking at the music I listen to).

The problem remains, how do I make sure that the recommendations I’m going to send to the user fall within their “taste profile” (essentially, the kind of music they like to listen to)?

You can solve that by looking at the attributes of the 2 main entities - a song and a user.

Take an instance of a song like Alright by Kendrick Lamar.

The song is associated with hip-hop, political commentary, social issues, and a particular sound and production style.

But, this does not really tell us the kind of listener that a user is. By looking at the song, you can’t:

• Tell if the user cares about the message behind the lyrics.
• Tell if they’re listening because they enjoy the production, the flow, or the beat.
• Tell if they’re just listening because their friends are playing it. Come on! It’s Kendrick Lamar we’re talking about…!
• Tell if the user’s taste extends to other genres, artists, or completely different sounds.

Take another example. Andy Blueman is known for his uplifting trance, often blending electronic music with classical and orchestral elements.

Someone who listens to his music might enjoy the genre itself. But they might also be drawn to the emotional breakdowns, the orchestral arrangements, the gradual build-ups, or the feeling of listening to something that sounds almost cinematic.

These are different reasons for liking the same music.

And, they matter.

If I recommend another uplifting trance producer to this listener, I might get it right. But what if what they really enjoy is the orchestral composition? Perhaps they would enjoy a film score. Or a classical piece. Or an entirely different genre that produces the same emotional response.

A recommendation system that only looks at genres and artists would probably miss these connections.

This is where I think LLMs can be useful. They can help us extract the attributes of a song beyond its genre, identify patterns across a user’s listening history, and connect those patterns to music the user hasn’t discovered yet.

Of course, listening history alone is not enough. Listening to a song once doesn’t mean someone likes it. And listening to something repeatedly doesn’t necessarily mean it’s their favourite. Context matters, too.

The goal isn’t to put users into boxes. It’s to build a model of their preferences that changes as they discover new music, revisit old favourites, and develop different interests.

Ultimately, I don’t think the problem with music recommendations is that we don’t have enough songs to recommend. We have millions of them.

The problem is understanding why someone might like a song they’ve never heard before.

And that’s the intersection of data and taste. In this case, the more data you have about what someone has listened to, the more accurately you can recommend precise recommendations that fall within their taste profile.