Insights · June 2nd, 2021
Do you feel like your digital life has been eclipsed by your own preferences? That you keep going in circles with the same people, same kinds of movie recommendations, same ads and same controversies? Or vice versa, does it seem like your old friends have been swooped up by aliens and transferred to an alternative reality? This could be because these platforms have been designed with the belief that personalizing everything is the best way to reel you in for as long as possible. Movies like The Social Dilemma and The Great Hack portray a chilling reality where the slot machines in our pockets predict our weaknesses and addictions so our attention can be sold to the highest bidder. Where the algorithms lead us into digital booby traps of misinformation and distorted ideas about reality in the pursuit of profit.
These movies are good at exposing the power of algorithms and the companies that use them, but they don’t really explain the models themselves, which may lead the viewers to think that recommender systems are a one-trick pony.
In this article I will try to explain the types of algorithms that cause these filter bubbles, how they are currently optimized and what this has to do with – the universe. In my next piece I’ll cover why the current iteration of recommender systems might reach a point of diminishing returns and how the algorithms might be optimized in the future to benefit all stakeholders. Yes, a win-win solution!
If you buy diapers you might want to buy – beer?
In 1992 a business manager for a retail chain analyzed point-of-sales data and found an odd correlation between the sale of diapers and beer. This was such an unusual product combination that the beer-diaper parable later became hugely influential for developing more targeted marketing approaches. If you can discover apparently idiosyncratic human behavior in unstructured data, you can suggest unique product combinations that the users may not even realize they desire.
Cue modern recommender systems
Almost 30 years later recommender systems based on these principles have become so powerful that they can affect the outcome of national elections or turn firms with few physical assets into trillion dollar companies. Association rules, matrix factorization and data driven clustering techniques can do what traditional sociological segmentation methods never could. They go beyond our preconceived ideas of what a typical target customer looks like and predict what a user is likely to want next.
So how do these systems work?
Take your protein pills and put your helmet on
During a Zoom conversation with fellow futurists and University of Houston alumni Lee Shupp and Jim Lee the subject of recommender systems came up. This is when a comparison to the expanding universe was made, and I thought this was such an interesting analogy that I decided to write this piece. Like celestial objects floating in space, we are the coordinates of our preferences moving around in a digital universe, each with unique temporary positions based on where we came from and where the algorithms think we should be going. Given the pull of our individual trajectories, our distance to other masses is growing and we’re finding ourselves increasingly disconnected from others. As we drift further into space we might be pulled into the gravitational field of black holes, making it difficult, if not impossible to return to our home galaxy.
Recommender systems are based on a similar logic. In order to predict your preferences you are placed in a vector space with coordinates that reflect your past behaviors or people the algorithms think look like you. These approaches are called collaborative filtering and content based filtering. Algorithms parse through a ton of unstructured data to predict your likely interests. It will then minimize the Euclidean distance between you and other users by giving you a similarity score. You now have a unique position in the preference universe based on your past behavior and proximity to others who are like you. So if you’re a middle aged guy in Wyoming who really digs Marvel movies, the algorithms might suggest similar movies and merch based on digital footprints left by you and by other people in the Marvel fanbase. Your interests may be paired with users in Georgia or Maine, but since you’re above the median age of similar users, it might skip promoting Disney or Hasbro products to you. Unless the algos figured out that you’re also a Brony, of course.
The effect of this dynamic is rising homogeneity between users who are being compared with one another and who can be placed in the same cluster, and less overlap between items or individuals in other clusters. Translated to normal speak it means that algorithms are currently optimized to look at a variety of our past user data, place us on a map, and pair us with other videos/ songs/ movies/ influencers/ people/ things/ information/ political parties/ religions/ cults/ whatever else that represents the shortest distance between where we are now and the objects we are clustered with. Since a cluster mean, or centroid, is defined only by the data points that make up each cluster, or filter bubble, there’s no universal gravity center or anchor outside of the individual cluster that will prevent it from drifting off into the outermost corner of the coordinate system.
So in data science and in astrophysics, while clusters will pull objects closer to their core, like the gravitational force of masses in a solar system, there’s nothing to prevent us from drifting light years apart. Or into a massive black hole. Perhaps this explains the rise of polarization and conspiracy theories in recent years?
A new kind of system?
Currently there are a few hacks you can use as a user to outsmart the algorithms, but the more important question is whether the platforms themselves will start to change how they work. In my next piece I will discuss why I think this is likely to happen and some optimization strategies that might be used to create better recommendation systems in the future.
See Anne’s Think Tank profile, and please reach out to speak to Nikolas Badminton about hiring Anne Boysen to speak or work with you – click here to contact.
Disclaimer: The views expressed here are Anne Boysen’s alone and do not necessarily reflect the views of Futurist.com, other Think Tank members, her employer, organizations affiliated with her employer, past nor current clients and customers I have worked with. All content shared is available to the public.
