Spotify developers meet 217 million monthly listeners where they are with seamless collaboration, continuous integration, and GitHub Enterprise.

You might remember when listening to your favorite music on the go meant tuning a radio, syncing an MP3 player, or adjusting a skipping portable CD player. It’s easy to forget this near-past when so much of the world’s music is available everywhere, instantly—only a search and a tap away. Music, audio books, podcasts and more can be streamed across every platform and across the globe. Behind the streaming industry’s incredible advances are decades of technical achievements in internet access, file formats, algorithms, and more.

In the last 10 years, Spotify has grown in step with these achievements to 217 million active users. From a Swedish startup to one of the largest music streaming services, the company is now a benchmark for integrated user experiences. Listeners tune in from iOS or Android on the go. At their desks they can listen from their browsers or from an app on Mac, Windows, or Linux. And from their homes they can connect to a smart device: speakers, virtual assistants, sound bars, and more. No matter where their users listen, Spotify’s software has to work seamlessly—and so do all employees working in offices around the world.

Product differentiation and user experience are key in an increasingly crowded streaming industry. With a growing and set of features, Spotify’s development team needs to collaborate to ensure everything from the client to the back-end infrastructure plays nicely together. And, for a user centric brand, it’s critical that nothing breaks in the process of pushing regular updates. Product Manager Laurent Ploix works to implement tools, processes, and systems that keep developers (and their code) running smoothly.

Ploix’s team and developers across Spotify use GitHub Enterprise Server for innersource projects and collaboration. They also build with GitHub Enterprise Cloud to securely open up their code, work with external partners and participate in the open source community. With open source close to the team’s process, they’ve been able to learn from the larger developer community.

Spotify’s experience in open source is first hand. It’s the foundation of some of the company’s most popular features. To power the recommendation-driven “Discover Weekly” playlists, the team builds and maintains Scio, an open source project built on Apache Beam. The technology allows them to compute recommendations for hundreds of millions of users and run complex processing jobs on thousands of machines in parallel. Scio is an important part of Spotify’s operation—still, it’s open source because the team believes in the model. Thousands of contributors mean thousands of ideas, greater diversity of thought, and ultimately, more robust ideas.

To bring open source innovation to proprietary projects, Spotify uses innersource. Innersource allows developers to run internal projects as if they were open source: working openly, learning from each other, and reusing code across the company. Ploix explained, “We encourage people to contribute to someone else’s code. Instead of submitting a JIRA ticket and waiting for a response, anyone can contribute.”

Ploix sees shared ownership as a route to higher quality and speedier delivery. “You need both openness and ownership,” he said. “When developers own code, they don’t change it for others or work on it exclusively. It means they feel strongly about it. They care about the quality, and they’re proud of it.” And to Spotify developers looking to contribute, Ploix said, “Pull requests are welcome. Someone out there might find a better solution than I can.”

Ultimately teams who own projects at Spotify take on roles similar to some open source maintainers. They receive and triage new bugs, ideas, and code. They also might be responsible for deprecating or even archiving their projects. According to Ploix, “They become maintainers, for sure. And with strong ownership, we avoid code piling up where no one knows whether it’s being used or not.”

We’ve focused on reducing information overload. Developers can get too much information from CI, but they need more than a pass or fail status to make informed decisions. We want developers to know exactly how a change impacts the code base.

Continuous integration (CI) is a central focus of Spotify’s ecosystem. Like many organizations, Spotify depends on CI as an internal process to reduce the time and effort required by each feature integration—and to successfully deliver a product version suitable for release at any moment. “CI is important from a business development angle,” he said.

To ensure they’re following best practices for development—fast and incremental iterations—the team built CI systems that integrate with GitHub Enterprise and provide developers with the exact information they need. “Our builds generate a lot of data, and it’s sometimes difficult to find the relevant feedback. Our integrations with GitHub enable us to surface it in the pull request and shorten the feedback loop,” says Marcus Forsell Stahre, Senior Engineer at Spotify.

This is where custom tools and webhooks come in—but Ploix sees room to further streamline their review process with the GitHub API. He explained, “The API has the potential to transform the developer experience. We use a lot of bots to provide insight into the impact a pull request will make—but it tends to pollute the conversation. We want to make this information easier to take in and act on. That’s where the Checks API is going to help.”

For Ploix, process can always improve, and even small enhancements can make a big impact on quality at the scale of Spotify’s operation. The team has a large number of machines running builds and tests for CI. Ploix notes that the team works with these systems on a vast amount of builds every day. And this number is only growing as more developers make more changes.

When Spotify developers need help, they turn to the GitHub Enterprise Support Team. “We always compare the types of support we get from vendors,” explained Ploix. “We hear back quickly from GitHub with insight into what’s going on—we’ve been really happy with it.” The Spotify team also meets regularly with GitHub Solutions Engineers to get more in depth support—from questions on using an API to larger strategic issues.

Fast, thorough support has become even more important as the company grows, stepping up developer recruitment. Even among a fast-growing team, Ploix has found that most developers know GitHub. He noted, “People know what a pull request is because it’s how they contribute to open source projects. We have many developers who are well-versed with GitHub, either for personal development or previous roles. With GitHub Enterprise, no one has to relearn the wheel.”

Ploix sees new hires and seasoned employees alike benefit from using a version control system they know and understand. He also appreciates the freedom teams have to fork (or copy) an existing project and work on it without touching the original source code. He said, “The sandbox environment GitHub provides allows our developers to experiment with new ideas. They don’t have to worry about breaking anything fundamental. This sort of creativity is encouraged, and it’s helped make the Spotify client what it is today.”