The glory day of cassettes and CDs to enjoy music were almost gone in public. Many stores were out of business while audio technology had come such a long way and continue to evolve. Thanks to technology, we become closer to music with many access whether on desktop or mobile. The on-demand music streaming service is one of them that help us do so. You might not recognize it but listening to music through an online streaming service doesn’t require much effort. Just open the app, click the music you want to listen, and voila!
There are many on-demand music streaming services these days but there’s no doubt that Spotify leads the business. Launched in 2008, Spotify has gained over 100 million active users, where 70 million of which are premium users. There are over 20 million songs available online on Spotify and 20.000 new songs are added every day to the database.
With millions of people listening to music every minute, Spotify has received a huge amount of data information, such as which songs are the most popular, where listeners are tuning in from, and what devices are using to access Spotify. They harness this data in every part of the organization to drive decisions which resulted in something valuable and insightful both for fans and musicians.
Behind The Big Data Technology of Spotify
On the surface, collecting the data seems simple. First, you provide the service. After that, people use your service for their own. Then, you get the data from the people’s usages. Finally, you use the data as information to make important business decisions. That’s the simple way people mostly think about how brands collect the data.
In reality, it’s actually a complicated process because it relies on many different systems. Not to mention that it needs a huge amount of physical storage space. Each day, Spotify’s users have created over 600GB of listening data, while Spotify has created more than 4TB of a data store for new music and others.
So, how do people at Spotify utilize that extraordinary amount of data?
Well, according to Jason Palmer, an Engineering Lead at Spotify, says that every co-worker has access to three tools: Dashboards, Data Warehouse, and Luigi. Dashboards have a similar interface to Google Analytics which allows the user to create their own custom screens containing data they are interested in from their pipeline.
Meanwhile, Data Warehouse is the more complex system that allows everyone to access the data-set directly. If a person more into complex operations, they’ll welcome to use Luigi which is used to run machine learning algorithm and provide daily reports. Palmer also wrote about the Spotify’s analytics infrastructure on the official page of Spotify Labs. He said:
“Most of our recurring data is added to our analytics pipeline by a set of daemons that constantly parse the syslog on production machines looking for messages we have defined along with the associated data for each message. Matching data is compressed and periodically synced to HDFS. Typically data is available in our Data Warehouse and Dashboards within 24 hours, but in some cases, data is available within a few hours or even instantly through tools like Storm.”
So, it is indeed complicated but Spotify isn’t alone in doing this. They’ve acquired Niland, the audio analysis company, to provide the data with better search and recommendations. In early 2017, Spotify also had acquired the blockchain startup, Mediachain Labs, to help them develop solutions via a decentralized database. The aim is to have a better connection between artists and licensing agreements with the tracks on Spotify’s service.
Spotify’s Big data Products
If you use it on daily basis, you must know that Spotify releases a playlist called “Discover Weekly” every Monday. In this playlist, you’ll find a list of songs that were chosen by Spotify based on your listening history. Basically, it gives you a recommendation of new music to listen. This is truly amazing as Spotify has millions of users and each of them has different playlist content so that they’ll feel special.
“Discover Weekly” is just one of many products that Spotify creates based on the big data technology. Here are others more of them:
Similar to “Discover Weekly” but the difference is it releases every Friday and only focus on the new-released tracks, especially from bands and singers that you listen to the most. It’s a good way to find a music you might miss but actually fits you.
Please bear in mind! We’re not talking about the conventional radio. The Radio feature on Spotify allows you to create a playlist with similar artists or songs. What’s more amazing is that the feature comes automatically every time based on what you’re listening to, whether it’s a playlist or an album. When you’re trying to figure out what songs to add to your playlist that fits a certain mood, this feature will come in handy.
Musical Map of the World
Spotify believes that music is a universal language. In order to help them better understand the nuances, they created a “Musical Map of the World” to see and hear how people around the world listen to music differently. In this page, you can see the world map in which cities and countries have green dots. When you hover the cursor to that green dot, the name of the city will appear and if you click it, it will show you a playlist of 100 tracks with the most streams in that certain area.
Not Just for Fans But Artists Also.
Spotify’s big data analytics go both ways for fans and artists. They make sure that its amount of data also available to musicians and managers. Last year, Spotify launched the Spotify for Artist app that provides mobile access to analytics. This covers everything from how many streams they are getting to which playlists are generating new fans. Basically, it’s a Google Analytics for musicians.
Moreover, the new mobile app for artists also allows musicians to have more control over their presence on Spotify, including the “artist’s pick” selection. They can update their bios and post playlists through this app, too. The geographic streaming data from the mobile app can even be instrumental for both musicians and their teams which allow them to plan tours more effectively.
Spotify for Artists app isn’t the only feature Spotify created for the musicians. There’s also a program called Fans First, which uses data to find an artist’s most passionate fans and target them with special offers. Spotify wants to make people notice that they make a serious effort to empower artists and make them less sceptical of the company. On the other hand, musicians can be more interactive and engage with their fans which prevent relation gap between them.
So, what’s next for Spotify?
We might not know what innovations Spotify will create next but we will be watching them. Hopefully, they will be able to continue humanizing data in creative ways and provide us with much more relatable music to our preference. More people will enjoy the music and music will never die in public.