On music apps, wellness, and the many types of “tech story.”
Since Technology was invented in the late 1990s, there have emerged a handful of species of stories in the tech press:
“When Will That Small Thing With the Blinking Light Kill Us?”
“Finally, a Thin Caucasian Fellow Has Invented a New Milk”
“You’ll Never Guess Who’s Going to Space Now”
“Billions of Dollars Later, It’s Still Racist/Sexist”
“Yeah, But Can It See Me When I Pee?”
This story is going to follow another popular theme: “AI Knows You Better Than You Know Yourself.” In fact, I was actually going to title it something like “AI Knows You Better Than You Know Yourself,” but when I searched for that combination of words, I found a long, proud tradition of writing on the topic.
These stories tend to explore how new technologies record data on our behavior or bodily functions and run that data through a powerful artificial intelligence. The AI then, possibly, comes up with all sorts of deep insights about us, which can then, allegedly, be used to shape our behavior. These stories weigh how this behavior modification might be used for our benefit — like getting us to walk more — or to create some negative outcome — like getting us to walk into traffic because we’re revealed as being undesirable.
It is exciting to write (and read) this type of journalism in the same way it is exciting to write (and read) science fiction. This is how things may turn out! Wouldn’t that be exciting! And if it doesn’t turn out that way, no one will remember!
I am not immune to the seductions of a good tech story. So here is one.
This story is about how music might be used by AI as a kind of therapy, or medicine, which I suppose is a subcategory of the AI story, something along the lines of:
“Now the Robots Are Doing Stuff With Our Precious Art?”
(I initially was curious about how new music apps might change the way our minds work. That story is here.)
Companies, most notably Spotify (but also Apple, Pandora, and Amazon), build their business model in part off of figuring out what we like to listen to and giving us suggestions about what else we might like to listen to, the goal being to keep us listening long enough to hear ad spots or just get a subscription already. But what if the end goal wasn’t endless jams but, rather, not dying of a stroke someday?
This is the premise behind Rubato, which is an app that purports to reduce stress via the music from your own, living playlist. Co-founder and CEO Amit Sternberg explained to me that, when you’re listening to your favorite music on a playlist, his app correlates it with your heart-rate activity, recorded via an Apple Watch or Garmin or some other wearable tech.
“We build the personal profile for each listener, and we understand which musical attributes impact his stress level and anxiety level or make a change in his cardiac indications,” he said, “and the AI then understands and prioritizes these attributes, and we are capable of recommending music that would scientifically reduce your stress.”
“Stress” here is determined by something called heart rate variability (HRV). HRV just means how much your heart rate changes from moment to moment. It’s controlled by the autonomic nervous system (which also regulates basic functions like breathing), which in turn runs two other systems: the sympathetic nervous system (responsible for fight-or-flight reactions) and the parasympathetic nervous system (responsible for, among other things, causing the body to, in scientific terms, chillax.)
When you’re stressed, your fight-or-flight is engaged, which keeps your heart rate steady, meaning your HRV is low. When you’re relaxed, your HRV is high — which you want, because it shows you’re adaptive. Here’s Harvard on the subject:
In other words, the healthier the [autonomic nervous system] the faster you are able to switch gears, showing more resilience and flexibility. Over the past few decades, research has shown a relationship between low HRV and worsening depression or anxiety. A low HRV is even associated with an increased risk of death and cardiovascular disease.
Rubato’s AI — still in beta — looks for what the music is doing to HRV based on 15 attributes: things like tempo, key, “speechiness,” “liveness,” “danceability.” For the few dozen users they’ve studied (mostly Rubato employees and students at the San Francisco Conservatory of Music and University of San Francisco), different attributes bring out different reactions.
“We see people that are very sensitive to the tempo of the music, but we see people that are in a way agnostic to the tempo,” said Sternberg. The AI figures out “which attributes reduce your stress,” and then it assembles a playlist of songs that fit those particular attributes.
First Rubato learns what you like to listen to; then, when you’re stuck in traffic, it might offer you a prompt: Wanna listen to a playlist to ease your woes?
“But,” you might say, “wouldn’t I already know which songs whizzle me down versus which songs get me jonked up for an evening of fist-fights?” Here is where this story’s theme really comes into play.
Because, said Sternberg, “Being excited for a song is not necessarily being relaxed by the song. Usually it’s the other way around.” Songs that, for example, evoke memories may seem to be relaxing you, but in fact are elevating your stress levels. So says the data, anyway.
And this is the big question that these sorts of tech stories weigh: Are there essential truths about ourselves floating along some dark invisible river that would make our lives better, if only we could get at them?
It is this possibility — that there’s a hidden world of Us that only technology can get at — that drives these stories, and in fact much of the tech industry generally. The corollary is the fear that this technology will use these insights against us.
Media professor Joseph Turow wrote an op-ed for The New York Times about voice analysis technology that companies (like Spotify) might use to infer everything from age and gender to personality and emotional state:
The technology is based on the idea that voice is biometric — a part of the body that can be used to identify and evaluate us either instantly or permanently. Businesses using this voice technology to offer us better pricing sounds great, unless you’re in the camp that loses the discount. What if you end up being refused insurance or having to pay much more for it? What if you find yourself turned away during early job screenings or have your cultural tastes prejudged as you surf the internet?
Tech journalism is a lot like tech: Both are forms of speculative fiction. An industry, inspired by The Jetsons, spends billions developing the flying car, drawing from visions from the past to conjure a possible vision for the future. Similarly, journalists draw on fears from the sci-fi and autocracies of yore to conjure their critiques of present-day tech. This is not to say that the journalists are wrong to do so, or that the industry is wrong to innovate. Only that — there’s so much we don’t know and have no real means (yet) of testing. It all can end up making the tech seem frivolous and the journalism seem flaccid.
Which is of course its own category of tech story:
“Sure, We Were Wrong, But 11 Years Ago Is So Ancient People Were Still Making Their Own Sandals”
“There’s a lot of different mental states that map onto the same physiological states. It’s not like there’s a clear path from those measurements to what’s happening in your mind.” – Jonas Kaplan
With that in mind, what can we say about Rubato? Well, it will be pretty great if it works as advertised: a non-invasive, non-pharmaceutical intervention that you can dance to. It will be less great if it sells your data or gets co-opted by Amazon or some other behavior-directing monolith. But maybe it will know us better than we know ourselves, and that knowledge will benefit us.
Sternberg says they’ve seen positive results as compared to randomly generated playlists. He imagines Rubato as a new kind of medicine, an alternative to a pill. They plan to test the app on patients undergoing chemotherapy or dialysis, where a lowered stress response could facilitate better healing.
If, that is, the data means what they think it means.
“There’s some basic unknowns and difficulties with matching music to your mood through those measurements, because those measurements are pretty crude with respect to how they map onto your actual mood,” said Jonas Kaplan, an associate professor at the University of Southern California’s Brain and Creativity Institute and co-director of the school’s Dornsife Cognitive Neuroimaging Center.
Kaplan said that, yes, you can get good data on heart rate or skin conductance, and “those things might have some broad relationship to what’s going on in your body. But there’s a lot of different mental states that map onto the same physiological states. It’s not like there’s a clear path from those measurements to what’s happening in your mind, or at least if there is, it’s pretty coarse.”
And yet — I would really like to know the secret combination of songs I have, here in my hands, that will put me at ease, provide some biometrically provable solace. What is the music that will make me whole?
The machines are trying to figure it out, and the writers are trying to figure out the machines. Round and round we dance.
