Skip to content
dub.digger (2.2 Beta)

Info

About this project.

Dub Digger is a tool for digging. Dub techno first. Type an artist, and you see who they worked with and which labels they released on. Every name is a link, so you can pick one and keep going.

The idea

Every record is connected to another. A producer, a mixing engineer who turns up on four other records you already enjoy. An iconic, minor label you discover by picking up an obscure release gives you access to a goldmine of tunes. That network of information is how crate diggers actually find music, and it all sits in Discogs. It just isn't optimised for digging this way.

Dub Digger reshapes it. Type an artist, see who they worked with and what labels they released on. Type a label, see the roster and what came out on it. Then click any artist or label and keep going. A map of the extended scene, drawn from credits. No unwanted noise.

Who it is for

As a dub techno enthusiast and record collector, I built the tool I always wanted: one that finds new acts starting from a core set of artists and labels.

So the intended audience is people who already read Discogs pages, whether for fun or for work. There are no recommendations here, no playlists, nothing trying to guess your taste apart from the fact that you love dub techno. The site assumes you either know what you are looking for, or you will know it when you see it.

That is why the pages are dense, and why almost everything on them is a link. Digging is a series of small jumps: artist, label, another artist, a quick listen to a preview. One click each time, no query to write.

What is in it

Not all of Discogs. A slice centred on dub techno, plus one step out from it. Right now that is 1,095,302 releases, 325,942 artists and 114,226 labels.

The slice starts with records tagged dub techno, deep techno, dub, ambient or minimal. Everyone credited on those records becomes the core. Then the tool takes one step out, in two directions: people who worked with someone in the core, and people who released on the same labels without ever meeting them.

It stops there. More steps out and you end up with most of techno, which would be a copy of Discogs and not a map of anything. So an artist you expected to find may simply be outside the boundary. That is how it is supposed to work, not a gap.

How it is made

Discogs publishes its whole catalogue every month as XML files, free to use. The releases file is over 100 GB once unpacked, and it is where all the information I needed lives.

All the heavy lifting happens once on my own machine. A script reads the dump piece by piece, keeps only the few fields the tool needs, picks the slice described above, and counts everything in advance. Two passes over 19 million releases.

The result is one lightweight database file. The website only reads it. No live API, no database server, nothing fetched while you browse (unless you press play on a preview: they stream straight from Apple). Also, there are no cover images: the dumps are free to use, the pictures are not. They are in the pipeline for a later version.

The editorial twist

Everything up to here is rules. I defined the logic, the script applied it, and nothing was decided by hand. Every artist carries a relevance grade, measured from how much of their work sits in the dub techno core. That measure works well, until it does not.

Quick example: King Tubby invented dub, and the tool graded him the same as the Spice Girls. His records are tagged as reggae, and the core rule only counts "dub" when it's paired with "electronic". I looked for a better number but I couldn't find anything. Madonna has more records inside the core than King Tubby has, on every measure the data can produce. As you may imagine, I wasn't happy to see that.

Inspirations and cultural factors are a matter of history, and genre tags cannot express them. So it is curated by hand instead. As a starting point I added seven "traditions/influences" in the config, among them roots dub, reggae, Detroit techno, afrobeat, dubstep and UK garage.

Belonging to one of them raises an artist to a fixed grade and no higher, and the page always says the grade was raised and which "tradition/influence" did it. This curation is my own editorial choice rather than a data-driven line of descent, and it is the fluid, human aspect of the tool.

What to expect

I applied Gall's Law as the founding principle of this project.

Gall's Law is a rule of thumb stating that all working complex systems evolve from simple systems that worked, while complex systems designed from scratch inevitably fail.

That's why I decided to start with the bare minimum of features so that the system can scale up easily if I ever decide to expand it. So if you expect a feature and it's not there, this may be the reason.

Discogs calls everyone credited on a record an artist, so engineers, designers and photographers have pages here too.

Credits on Discogs are entered by volunteers, so they may be uneven and patchy.

This is a beta, and the ranked lists are the most experimental part of it. If a page looks wrong, it is either the data or one of my judgements. Both can be argued with. So feel free to get in touch to discuss it.

Happy digging.
Simone

Apple and Apple Music are trademarks of Apple Inc., registered in the U.S. and other countries.

Let's talk.

Drop me a message at

simone.ferraro@nutek.it