A few days ago, on a routine end-of-season wrap-up and reflection session with one of Message Heard’s clients, someone on the call said something in passing about reaching audiences outside of London. I took a note to follow up with some stats and caveats, and as I prepared my note, I stopped to check how, say, Apple and Spotify know where someone is while they’re listening through their apps.
The short answer is: I couldn’t find the information. Oh, they’d confidently declaim which cities people are listening in, but not how they knew. And that’s a problem.
It’s a problem for our client, who has a sector-specific reason for wanting to reach audiences outside the capital, but it’s also emblematic of a broader problem in podcasting: analytics.
We all know this. As a podcast listener and citizen, I love and champion the open nature of the formal podcasting specification, its privacy and its lack of lock-in or even ownership. As a podcast exec, however, one of the least fun parts of my job is spinning up my various spiels about the lack of richness in podcast data. “Well, a download is…”. “You can extrapolate some demographic data from Spotify, but…” “There’s no provision at the spec level in podcasting for inbound referrer links, so…”
But I’m not talking here about the big, intractable technical problem of analytics in podcasting, the stuff which frustrates the clients of a production house like Message Heard who are more used to web and social marketing, with data which is granular almost to a fault. The thing that concerns me is, I don’t really trust podcast analytics either.
There’s a huge caveat to that provocative line; I don’t think anyone’s actively lying, and if your podcast host is IAB-certified, I have no reason to think the basics aren’t sound. My concern is we’re often just being asked to accept what we’re given at face value.
“Is my podcast being listened to outside of London?” is a perfectly reasonable question to ask, and location is one of the few data points we have! Yet I don’t know how it is derived.
Stay with me for the next two paragraphs even if you’re not a technophile. A typical way digital services locate you is through your IP address – the numerical string your computer reports out to the internet to route traffic. If your IP address is between 104.166.32.0 and 104.166.47.255, say, then you’re in the Bahamas and I’m a little envious.
These lookups, though, are imperfect for some reasons I know (imprecision, VPNs, addresses for certain providers surfacing in locations hundred of miles away from the user) and likely for some reasons I don’t. Your podcast host will report listener location by IP address inference, and it’s generally thought to be good enough for country-level breakdown.
The apps for Apple Podcasts and Spotify themselves also gather location data, though, so long as the user has granted permission, and so presumably since your phone has GPS, the location data you see in their dashboards (rather than your podcast host’s) should be super-accurate. Except… I can’t find any information anywhere about how this location is recorded (although if you know, please tell me).
It could be from GPS or similar satellite technologies, but phones can use a range of other techniques, and all the app does is ask the operating system ‘where am I?’, with all the heavy lifting for coming up with the answer handed off to the phone. Which technique is used? We don’t know. How robust is that data? We don’t know that either.
Here we have a reasonable question – which it should actually be possible to answer even with our scant data – and yet I’m not confident enough about its provenance and methodologies for me to advise our client in a simple and unambiguous way. And this is a tiny example of a problem anyone, especially those in the professional areas of podcasting, will face dozens of times a day.
The data in this example about location may be robustly arrived at for Apple and Spotify, but they haven’t told us, I don’t think, so we can’t check; that lack of transparency across many aspects of podcast analytics smears an unfortunate stench of charlatanism and chicanery across the whole industry.
There are lots of interesting discussions happening about the future of podcasting – even the future of open podcasting. We know that as podcasting becomes professionalised, for good and ill, companies will demand more granularity, more accountability and more insight, even if just through force of habit.
I’m just worried that even what we have now is opaque; that even in cases where there’s no political or commercial advantage I can see to keeping methodologies private, companies still do; and that in a sector that necessarily mixes creativity with technology, most of us pour energy and rigour into the former while leaving the latter foolishly underinterrogated.
Podcasting’s analytics need to get better in a commercial context, but so do we. Better at probing the scraps of data we’re given to understand what it means and if it’s trustworthy, rather than blithely accepting pretty graphs and reassuringly precise numbers on a dashboard. And better at expecting more transparency and collegiality from the companies which populate them.
