SECRET OF CSS

Automate Your Music Collection Using This Python Package | by Fabio Manganiello | Sep, 2022


You don’t need a big music cloud service to keep track of your listening habits and discover new music

1*OsAJt 7VgJiFGM6No pieg

I have been an enthusiastic user of mpd and mopidy for nearly two decades.

I have already written an article on how to leverage mopidy (with its tons of integrations, including Spotify, Tidal, YouTube, Bandcamp, Plex, TuneIn, SoundCloud, etc.), Snapcast (with its multi-room listening experience out of the box), and Platypush (with its automation hooks that allow you to create if-this-then-that rules for your music events easily) to take your listening experience to the next level while using open protocols and easily extensible open-source software.

There is a feature that I haven’t yet covered in my previous articles: the automation of your music collection.

Spotify, Tidal, and other music streaming services offer you features such as Discovery Weekly or Release Radar playlists, respectively, filled with tracks that you may like or newly released tracks that you may be interested in.

The problem is that these services come with heavy trade-offs:

  1. Their algorithms are closed. You don’t know how Spotify figures out which songs should be picked in your smart playlists. In the past months, Spotify would often suggest me tracks from the same artists that I had already listened to or skipped in the past. There’s no transparent way to tell the algorithm, “hey, actually, I’d like you to suggest me more this kind of music — or maybe calculate suggestions only based on the music I’ve listened to in this time range, or maybe weigh this genre more.”
  2. Those features are tightly coupled with the service you use. If you cancel your Spotify subscription, you also lose those smart features. Companies like Spotify use such features as a lock-in mechanism — you can check out any time you like, but if you do, nobody else will provide you with their clever suggestions.

After migrating from Spotify to Tidal in the past couple of months (TL;DR: Spotify f*cked up their developer experience multiple times over the past decade, and their killing off libspotify without providing any alternatives was the last nail in the coffin for me), I felt like missing their smart mixes, discovery and new releases playlists.

But, on the other hand, Tidal took a while to learn my listening habits, and even when it did, it often generated smart playlists that were an inch below Spotify’s. I asked myself why my music discovery experience should be so tightly coupled to one cloud service on earth. And I decided that the time had come for me to generate my service-agnostic music suggestions automatically: it’s not rocket science anymore, there are plenty of services that you can piggyback on to get artists or tracks similar to some music given as input, and there’s just no excuses to feel locked in by Spotify, Google, Tidal or some other cloud music provider.

In this article, we’ll cover how to:

  1. Use Platypush to keep track of the music automatically you listen to from any of your devices
  2. Calculate the suggested tracks that may be similar to the music you’ve recently listened to by using the Last.FM API
  3. Generate a Discover Weekly playlist similar to Spotify’s without relying on Spotify
  4. Get the newly released albums and singles by subscribing to an RSS feed
  5. Generate a weekly playlist with the new releases by filtering those from artists that you’ve listened to at least once

We will use Platypush to handle the following features:

  1. Store our listening history to a local database, or synchronize it with a scrobbling service like last.fm.
  2. Periodically inspect our newly listened tracks, and use the last.fm API to retrieve similar tracks.
  3. Generate a discover weekly playlist based on a simple score that ranks suggestions by match score against the tracks listened to on a certain period, and increases the weight of suggestions that occur multiple times.
  4. Monitor new releases from the newalbumreleases.net RSS feed, and create a weekly Release Radar playlist containing the items from artists that we have listened to at least once.

This tutorial will require:

  1. A database to store your listening history and suggestions. The database initialization script has been tested against Postgres, but it should be easy to adapt to MySQL or SQLite with minimal modifications.
  2. A machine (a RaspberryPi, a home server, a VPS, an unused tablet, etc.) to run the Platypush automation.
  3. A Spotify or Tidal account. The reported examples will generate the playlists on a Tidal account by using the music.tidal Platypush plugin, but it should be straightforward to adapt them to Spotify by using themusic.spotify plugin, or even to YouTube by using the YouTube API, or even to local M3U playlists.

Start by installing Platypush with the Tidal, RSS, and Last.fm integrations:

[sudo] pip install 'platypush[tidal,rss,lastfm]'

If you want to use Spotify instead of Tidal then just remove tidal from the list of extra dependencies — no extra dependencies are required for the Spotify plugin.

If you are planning to listen to music through mpd/mopidy, then you may also want to include mpd in the list of extra dependencies, so Platypush can directly monitor your listening activity over the MPD protocol.

Let’s then configure a simple configuration under ~/.config/platypush/config.yaml:

Start Platypush by running the platypush command. The first time it should prompt you with a tidal.com link required to authenticate your user. Open it in your browser and authorize the app. The next runs should no longer ask you to authenticate.

Once the Platypush dependencies are in place, let’s move to configure the database.

I’ll assume that you have a Postgres database running somewhere, but the script below can be easily adapted also to other DBMSs.

Here’s the database initialization script:

Run the script on your database — if everything went smoothly, then all the tables should be successfully created.

Now that all the dependencies are in place, it’s time to configure the logic to store your music activity in your database.

If most of your music activity happens through mpd/mopidy, then storing your activity to the database is as simple as creating a hook on NewPlayingTrackEvent events that inserts any new played track on tmp_music. Paste the following content to a new Platypush user script (e.g.~/.config/platypush/scripts/music/sync.py):

Alternatively, if you also want to sync music activity that happens on other clients (such as the Spotify/Tidal app or web view, or over mobile devices), you may consider leveraging Last.fm. Last.fm (or its open alternative Libre.fm) is a scrobbling service compatible with most of the music players out there.

Both Spotify and Tidal support scrobbling, the Android app can grab any music activity on your phone and scrobble it. There are even browser extensions that allow you to keep track of any music activity from any browser tab.

So an alternative approach may be to send both your mpd/mopidy music activity, as well as your in-browser or mobile music activity, to last.fm / libre.fm. The corresponding hook would be:

If you go for the scrobbling way, then you may want to periodically synchronize your scrobble history to your local database — for example, through a cron that runs every 30 seconds:

This cron will synchronize your scrobbling history to your local database, so we can use the local database as the source of truth for the next steps — no matter where the music was played from.

To test the logic, simply restart Platypush, play some music from your favourite player(s), and check that everything gets inserted into the database — even if we are inserting tracks on the tmp_music table, the listening history should be automatically normalized on the appropriate tables by the trigger we created at initialization time.

Now that all the plumbing to get your listening history in one data source is in place, let’s move to the logic that recalculates the suggestions based on your listening history.

We will again use the last.fm API to get tracks that are similar to those we listened to recently — I find last.fm suggestions often more relevant than those of Spotify.

For the sake of simplicity, let’s map the database tables to some SQLAlchemy ORM classes so that the upcoming SQL interactions can be notably simplified. The ORM model can be stored under ~/.config/platypush/music/db.py:

Then create a new user script under ~/.config/platypush/scripts/music/suggestions.py with the following content:

Restart Platypush and let it run for a bit. The cron will operate in batches of ten items each (it can be easily customized), so your music_suggestions table should start getting populated after a few minutes.

So far, we have achieved the following targets:

  • We have a piece of logic that synchronizes our listening history to a local database.
  • We have a way to synchronize last.fm / libre.fm scrobbles to the same database as well.
  • We have a cronjob that periodically scans our listening history and fetches the suggestions through the last.fm API.

Now let’s put it all together with a cron that runs every week (or daily, or at whatever interval we like) that does the following:

  • It retrieves our listening history over the specified period.
  • It retrieves the suggested tracks associated with our listening history.
  • It excludes the tracks that we’ve already listened to, or that have already been included in previous discovery playlists.
  • It generates a new discovery playlist with those tracks, ranked according to a simple score:
formula: p(i) = Σ to the mij. Under the sigma is j∈Li

Where ρ(i) is the ranking of the suggested i-th suggested track, L(i) is the set of listened tracks with the i-th track among its similarities, and m(i, j) is the match score between i and j as reported by the last.fm API.

Let’s put all these pieces together in a cron defined in ~/.config/platypush/scripts/music/discovery.py:

You can test the cronjob without having to wait for the next Monday through your Python interpreter:

If everything went well, you should soon see a new playlist in your collection named Discover Weekly [date]. Congratulations!

Another great feature of Spotify and Tidal is the ability to provide “release radar” playlists containing new releases from artists we may like.

We now have a powerful way of creating such playlists ourselves, though. We previously configured Platypush to subscribe to the RSS feed from newalbumreleases.net. Populating our release radar playlist involves the following steps:

  1. Creating a hook that reacts to NewFeedEntryEvent events on this feed.
  2. The hook will store new releases that match artists in our collection on the new_release table that we created when we initialized the database.
  3. A cron will scan this table every week, search the tracks on Spotify/Tidal, and populate our playlist just like we did for Discover Weekly.

Let’s put these pieces together in a new user script stored under e.g. ~/.config/platypush/scripts/music/releases.py:

Just like in the previous case, it’s quite easy to test that it works by simply running refresh_release_radar_cron in the Python interpreter. Just like in the case of the discovery playlist, things will also work if you use Spotify instead of Tidal – just replace the music.tidal plugin references with music.spotify.

If it all goes as expected, you will get a new playlist named New Releases [date] every Monday with the new releases from artists that you have listened.

Music junkies can discover many new music today without leaving their music app. However, smart playlists provided by the major music cloud providers are usually implicit lock-ins, and the way they select the tracks that should end up in your playlists may not be transparent or modifiable.

After reading this article, you should be able to generate your discovery and new release playlists without relying on suggestions from a specific music cloud. This could also make it easier to change your music provider: even if you drop Spotify or Tidal, your music suggestions logic will follow you whenever you decide to go.



News Credit

%d bloggers like this: