Episode 1 - How AI Gets Made
Episode 1 - How AI Gets Made
0:00:00 - Kiran V
The last two recordings were over an hour, forty. So it's definitely yeah.
0:00:07 - Joe C
We got the overall length down. We can we can keep working on the individual length.
0:00:12 - Andy B
And the main thing that I think helped us for the length was picking something small and doable, not trying to explain, yeah too much
0:00:29 - Kiran V
and like even then, as we were going through, I was like dude, I can spend so much time talking about just this thing, yeah.
(Intro Music)
0:00:53 - Joe C
Hey everyone, today we're going to be talking about how AI gets made. We're going to be talking about the who, the why, the what and the how. We're going to be talking about different roles in the process and exposing what happens behind the scenes.
0:01:07 - Andy B
Yeah, and I'm going to get us started and talk about why companies, and it's usually is companies that are making AI, why they even do this. So I want to point out two different reasons why businesses invest in AI because it is expensive to make AI. You'll understand why in a second. How many people need to get involved. One is they're going to try and do something with AI to save the business money. That's one really common reason why companies try to use AI, but another, much more interesting for us to talk about reason is because they want to create a beautiful experience for their users. They want to solve one of their users problems or delight you and we're going to be talking about a company that has done exactly this for the rest of this episode and that's Spotify. Odds are you're even listening to this podcast on Spotify and are you aware of how much AI machine learning goes into making an app like Spotify? It's a ton.
People who pay for Spotify love to not just listen to music, but to discover new music, right, they want to be able to find new artists, share music that they like, and so there are bunch of Spotify smart features, as AI features are often called. Smart features are designed to keep you happy with the app right and give you a better experience. So Spotify earlier this year introduced a feature called smart shuffle. You might have seen this if you've ever seen a little sparkly icon next to the shuffle icon. That's what we're going to be talking about. They added this feature that uses AI to give you better shuffle experience so you can more easily find the next great song, and they just every couple of songs, they introduce something that wasn't in your original playlist just to see if you would like it, and that's an example of trying to improve a product or process for your users with AI.
0:03:09 - Kiran V
Yeah, I do also want to add another reason why AI gets made, and this is maybe a little more cynical, but companies that offer AI or claim to have some AI component have, in recent years, been getting significantly higher valuations just because they offer AI. So this is actually something that startups will often do is claim to be selling or creating some sort of AI product and go get high valuation, sometimes three to five X. Imagining what the same company would be without AI (quote, unquote) and I think in some of these cases we end up seeing AI applications or products that aren't actually useful.
0:03:59 - Andy B
Or they're not even AI. I don't remember if you, if you guys, remember this, but we had a customer at Figure Eight who made an app where you take a picture of a wine bottle and then it would automatically like read the wine label and organize your wine cellar and they would tell people it was AI. But it totally wasn't. It was just crowdsourcing. It was a live human being on the other side pretending to be an AI model so that these business could say that they do AI wine bottle reading.
0:04:30 - Joe C
Yeah, and some of the reasons why a company would build AI is really why they would build any product. They're really looking for that intersection of what gets returns and makes the company money and delights users or, as a sum, is something that will assist their users and grow their audience. And I think in between those two things is sort of the Goldilocks zones for things that actually get produced. So let's jump into a bit of information on the roles that go into producing AI. So, when you think of the folks at Spotify or any other company, what is the actual team made up of and what are all those different people doing? And we're going to start at the top.
A lot of companies are structured where the vision is going to be set by executives. Your CEO, your CTO, that C-level suite. They're also the ones who hold the purse strings, so they're going to be able to have a lot of sway on where the money's going. As we've talked about, AI projects can be very expensive and there can be a lot to them. It can be a big decision for a company to take one on and so you're going to want to go to the executive team and try and get that sponsorship or funding from them and sort of have a champion in that realm.
You'll see also a lot of roles really at any tech company and certainly at AI product companies that are in what we consider 'go to market'. So these are your sales folks, the marketing team, those who are putting out all the assets of the company, creating events, basically doing everything they can to sell the product or the service that the company provides, and bringing the money in and making those deals.
0:06:16 - Andy B
Yeah, in the Spotify case, imagine you're on the Spotify marketing team and you're like, hmm, we haven't gone viral, we haven't had any cool new features. Maybe you're asking like the product team, which we'll talk about in a second. You're asking them like hey, I need a cool new thing to show off, what's something we can do that no other platform can do, that we can put on our Instagram and we can put on ads about how awesome Spotify is. Can we do AI? Other companies are doing AI. They might be pinging and asking for some of these features to sell.
0:06:49 - Joe C
A lot of 'Go to Market' folks are very customer facing, so this would include folks who are helping with support, and they become very aware of the problems of the company's users or customers and the problems that the company can then go and try and solve. So they're often a great group to go and talk to about. What do we solve next and how do we begin to solve it? What are our customers asking for? And sometimes the answer to that is let's solve it with AI, let's solve it with an AI model. So that's 'Go to Market'.
Then the other side of things you could bucket as 'research and development', and a big part of that is going to be the product team, and often these folks hold a business degree. They're looking at the market and and really trying to understand how to meet users where they are, build something that will meet their expectations and something that can be sold to them, and that involves a lot of customer research as well understanding the market that's out there, understanding the competitive space. They're often going to work with researchers and designers so as they start to identify what problems to solve, they're going to work with a team to start imagining a solution, building out initial prototypes and that group is going to work very closely with engineering. And Kiran, you want to talk about engineers?
0:08:20 - Kiran V
Yeah, sure. So engineering is generally the team that's responsible for actually executing on the vision, creating what product and design has put together.
0:08:32 - Andy B
For context, I'm product, Joe is design, Kiran is our engineer. We would tell him go build this thing and he'd have to build it.
0:08:39 - Kiran V
Yeah, exactly so. Engineering, you know, takes requirements from product, will, do research and exploration in terms of the technologies required to make that come true, build it and then ultimately deliver that to the customer. And so engineering, if you think about AI, you generally think about data scientists or machine learning scientists, but there's a lot of other engineering disciplines that are involved in actually getting AI out into the market. That's going to be front end developers, back end developers, solutions architects, tech leads.
0:09:16 - Andy B
So Kiran, can you explain what a front-end versus a back-end developer is?
0:09:22 - Kiran V
Yeah, sure.
So with these different disciplines, right, the front-end team is going to be responsible for everything that a user touches and sees in the product, and again we're talking about software products, so these are going to be on your laptop, on your iPhone, the software, the thing that you're interfacing with.
That is what the front-end team is building. The back-end engineering team is generally going to be doing things like building API services so that the application has something to talk to, and run a lot of the heavy computation that's required to deliver the experience to the user. The data scientists are generally going to be involved in labeling data, in training models, in deploying and tuning those models, and then again you have a whole slew of other engineering roles, but generally you're going to fall into these three buckets. Something like a data engineer will generally fall between a data scientist and a back-end engineer, where they're going to be doing a lot of ETL or transferring large amounts of data, setting up frameworks so that the data scientists can run their experiments, and general engineering support, if you will, to make things effective and efficient for the other engineers to go and execute.
0:10:45 - Andy B
And to tie this back to Spotify, imagine that you have a team I'm guessing the smart shuffle team was like about 20 people. There was probably one product manager, maybe two product designers and a couple of people of each of the kinds that Kiran described, and they work together simultaneously to solve this problem. Like, okay, we want to build smart shuffle and it's a lot of like sharing information with each other and everybody has their little part to play to make the whole thing come together.
Hopefully, this is starting to make sense why some of these projects can be really expensive because it takes a lot of different people with a lot of different specialties in order to complete the entire vision, to deliver you the little sparkle next to your shuffle icon.
0:11:38 - Joe C
And this is one of the reasons why creating AI can be so expensive - is because you need to employ a whole lot of smart people, and this doesn't factor in the computing costs, which are another big factor.
0:11:50 - Kiran V
Yeah, and then two more groups of people. One very important role that has started to come along with AI is data security. So you might have a chief information security officer or a DPO (data protection officer). So this role is generally to make sure that we're adhering to privacy compliance in the EU, we're adhering to any customer requirements that we may have around data security or protection.
And we have a whole episode on training data. So go listen to that if you want to get more into. You know what data means in AI and all of that. But these people are really important and are going to be important in kind of the ethics and morality around data as AI becomes a lot bigger. And then the last group are not always part of the team, but subject matter experts, or SMEs as people will refer to them. These are folks that are experts in the particular task, if you will, that the AI model is trying to accomplish. So, for example, if you're trying to train an AI model to detect cancer cells in a body, you might have physicians or radiologists in the room with you actually assessing that model, actually providing training data to that model and providing feedback on the product to make sure that it's actually being built to solve the needs of that end user. So subject matter experts are generally going to be people who are similar to your end user or people who have a lot of experience in the real world. A human version of the thing that we're trying to do with machine learning.
0:13:35 - Andy B
Imagine again Spotify Shuffle. You know who's really good at picking what to play next? DJs. So probably Spotify has a couple of if I had to guess expert DJs that they keep on staff or can call up who can kind of help sanity, check what the model is doing and test the experience and give feedback before this is ever released To more people, to make sure that, like, oh yeah, like the random songs that's picking are actually the same vibe as what the user was going for. Because it wouldn't make any sense if the random shuffle, the smart shuffle, just threw you into a totally different energy. Right, you would hate that feature. It's very important that it behaves for you like it's your own magic AI DJ. So they talk to DJs.
0:14:25 - Joe C
So we touched on this. But, thinking about all these roles together, they're not always going to employ every role. Product design, front-end engineers they might not be involved in a project if there isn't a explicit customer facing experience, like something like Spotify. As we said, sometimes AI is used to solve a problem that's happening in the back end to save customers money or save the company money internally, and that might not use some of these people.
0:14:59 - Andy B
And all the interactions between these people can get really spicy. In the best of times when you're working with an incredible team like Joe, Kiran and I, we used to just like have a lot of fun working together. When we would disagree, we'd kind of tease each other and like figure out, and by working together we made some really incredible things. But it can also get like real messy. Because you've got like a data engineer who's like I'm sitting on a billion songs, you want me to index them how? And you've got a machine learning scientist saying like the data engineer won't do his job, I'm blocked, basically meaning I can't do my thing until they do their thing. So you've got one person's boss yelling at another person's boss about a third person's boss. It can get really messy really quickly.
0:15:44 - Joe C
Yeah, probably not unique to AI. If you've been to work you may know this concept.
0:15:50 - Kiran V
Yeah, and if you think about the types of personalities of the people that are typically in these roles, if we're stereotyping, you have, you know, nerds working with jocks working with you know, older people, younger people, college grads, different countries, different languages to deliver something that needs to be a cohesive, really enjoyable user experience for you, me, us, the end customers.
0:16:22 - Andy B
It's fun.
0:16:25 - Kiran V
All right. So should we talk about how we actually deliver an AI feature or product? If you think about, Google has an email product. They have a Word document product. These are all different products, right? When you have, like, a dedicated experience for a particular thing. A web browser is a product and it allows you to browse the web, so that's a product. Then we have features, which are going to be individual components of that product that do a specific thing, right? So I want to be able to save my Word document. That is a feature of that product.
0:17:08 - Andy B
And again, it's Spotify, the player for Spotify, your app. That's the product. The feature is called smart shuffle and it's the little button you push to shuffle that then gets a sparkle and introduces you to new music.
0:17:24 - Kiran V
And so there's a whole bunch of steps and so generally, when we're building a product, we break it down into a bunch of features. We break each of those features into a bunch of engineering or product requirements. So these are very, very small capabilities and if you aren't familiar with technology, it's a lot smaller than you're probably thinking like. Individual lines of code are written in order to deliver product, and you might have thousands or millions or billions of lines of code that are required to deliver a whole product.
0:18:01 - Andy B
Can you give us some examples of what some would be for Spotify's smart shuffle?
0:18:07 - Kiran V
So this could be a five hour episode.
No, no, this could be like a five hour episode. So, all right, so I'm a front end engineer, so by trade, I am trained in building experiences for users. Again, the things that you're touching and feeling and clicking and pressing and swiping as the end user. And an example of a piece of front end code right, this is, you know, the some component, if you will.
You see, if you see the Spotify application and you see the song listing in the scrolling view, that is an individual component that is rendering information about your song. So when we say rendering, we say this is how the computer is processing whatever was written in code and putting it onto a screen for a human to see. So we're rendering that component and that might be anywhere from 10 to 500 lines of code. You know, or more, depending on you know, how it's written in order to display that block, but then you might have 1000 songs in your playlist. There's only one of the components and that component will be repeated throughout the application. This is probably a topic for a totally separate podcast because we're doing back end and all these things. So, okay, so I'm going to go back off of my tangent.
0:19:41 - Andy B
Hold on. No, what I wanted to explain is if the feature is smart shuffle, the individual tasks might be like, 'add the shuffle button to the page'. Add the sparkle icon when user hits shuffle button to sparkle icon. Send a request to the model. Wait, every 10 songs predict a new song. Every 10 songs look at the last 10 songs that they skipped predict the next best song. So, like your very small pieces that you're getting working.
0:20:14 - Joe C
Yeah, and I don't even think a product team would be quite there yet. I mean, that's almost in prototyping and like design. But I do think very early on it's decided that AI or a model will be used and there's a lot of planning that goes into the model itself and like what exactly do you want the model to answer? Like, how do you define that problem of 'we want a better way to suggest music'? And you, I think, have to be a little bit more specific about how you're going to do that with AI. And you start to thinking about, think about what data is going to go into it. What are some of the costs, like the compute costs? Some of those things come up pretty early.
0:21:01 - Kiran V
Yeah, so we went on a tangent, but so what I want to talk about is how we deliver an AI product. There's a bunch of steps that the engineering and product and sometimes go to market sales teams will go through in order to deliver that feature or capability to the end customer. The first thing generally will be to start with the problem. Sometimes there are solutions that are looking for problems. We won't talk about those people, but you want to identify a problem and this is usually done through something called user research. So this is when you've identified a set of people. This might be, you know, teenagers that are interested in taking pictures and posting it of themselves, or old people that struggle to drive themselves around and need help.
0:21:58 - Joe C
In Spotify's case, a group of people who like to listen to music, which I'm going to imagine is everyone.
0:22:06 - Andy B
And this is what I do as a product manager. So, for example, if I was the product manager in Spotify, the problem I'm solving with Smart Shuffle is our users will get bored if they don't get introduced to new music. But they don't wanna only listen to new music. If we can find a way to give them a little bit of new music while they're listening to their favorite playlists, they're gonna really enjoy that experience.
0:22:32 - Kiran V
Yeah. So identifying the problem and that's talking to users, seeing how they're using maybe an existing version of what you've already built, and see them maybe continually doing something wrong and you're like, okay, well, if everyone is doing it wrong, clearly there's something wrong with the product. So identify your problem. The next step is usually gonna be some version of prototyping or building a MVP (minimal viable product). MVP generally will come later, but in this prototyping phase, what you wanna do is you wanna build versions of your solution so that these users you can take it back to these users that you talked to initially and say, hey, what do you think about this? Does this solve the problem that you had before?
0:23:14 - Joe C
I'll shout out product designers here. So I'm a product designer, and this is a point where a lot of product designers come into the process to start thinking about where things are placed. Buttons for example, what's the look and feel gonna be? It's certainly still far off from a final design, but a prototyping engineer and a designer can work together at this point.
0:23:36 - Kiran V
Yeah, so building prototypes, iterating with those users, you might have something like a sponsored user program where you have these customers or individuals that have opted in to provide you feedback through this prototyping process. So it's an explicit relationship that you can develop with these customers or end users to improve the quality of your features.
0:24:00 - Andy B
You can also think of this as beta users. If you've ever signed up for the beta program of anything that you're using, there's a product manager or somebody on the other side being like, I just want some people who are willing to tell me if I'm doing the right thing, if I'm building the right thing for them.
0:24:17 - Kiran V
And so the goal really of the prototyping step is to be able to validate that the problem you identified is 1) indeed a problem, and 2), that you can develop a solution that will solve that problem.
It's not always gonna be super high fidelity, and when I say high fidelity I mean - you can literally draw a bunch of screens on a paper and point to different things on the screens in a simulation environment with them where you might ask them some questions and have them point to what they think will do. That's a version of paper prototyping - All the way up to this is mostly a functioning product, and that's when you start to see things like beta releases where people are able to access that a little more broadly, once you get through the prototyping phase, you generally wanna get to something called the MVP, or a minimal, viable product. This is the minimum set of things that you can build in a product or in a feature that are gonna solve the end users problems. So if you all you ever did was develop that MVP, it's still gonna be something that's useful for these customers and they will be happy to use that and adopt it as part of their process.
0:25:40 - Joe C
It may not be very pretty, but hopefully it provides some value.
0:25:45 - Kiran V
This is also the phase where the model (when we're talking about AI) is gonna go through model tuning and development, model development and tuning.
So in model development, that's usually when you have to start with something that's off the shelf. Again, there are things like Hugging Face, where they have hundreds of AI models that you can use for free in a commercial application. They're gonna do things like collect training data. If it's not something off the shelf readily available, they might have to get a bunch of training data and train a new model, and then they're gonna go through iterations of improving the quality of that model to make sure that, again, we're ultimately solving the customer's problem so that the quality of the AI or the recommendations that Spotify is giving are gonna be songs that people are gonna be happy with when they get that recommendation and continue to use that feature. If you're someone that's very much into classical music and suddenly it's suggesting you R&B, hip hop and rap, you may or may not be a happy camper, and so this is where, in that prototyping and MVP phase, is where the model training is really gonna happen and improve.
Once you've gotten through your MVP, you're gonna eventually wanna get this thing into production, and that is generally where the engineering excellence, if you wanna call it, comes into play. So, as an engineer, this is the point when we're going from MVP to a full production environment is when we really need to make sure things are gonna scale well. So what happens if a billion people start asking you to shuffle their playlists? Are your machines gonna be able to handle it? Are things gonna crash? So scaling is a huge challenge in engineering. This is where you're going to want to make sure that the product is really buttoned up from a product and design perspective.
So, again, in MVP, if someone's not able to save this playlist, they might not care. Like, I know it's a beta, I can listen to it now. If I close my app, it won't be there. But in a production environment, someone that's listening to that playlist on Spotify if they like that, they're gonna wanna save that playlist, and if you haven't given them that ability, they're gonna be really frustrated because now they can't get back to something that they really enjoyed.
And so these are the types of things that are gonna get built after that MVP phase where you really need to make sure that every part of that user experience from the time they get in the application, when they're interacting with the AI, when they're accomplishing their goals. That whole process, or workflow as we call it, is really buttoned up to make sure that it's a great experience and they can do it over and over without frustration. So that's the medium version of the process that generally product and engineering or in other some places called research and development teams will go through in order to take an AI idea, an idea for AI product or feature, from research through development into deployment to ultimately getting that into the end customer's hands.
0:29:30 - Joe C
And I think the way that we described it sounds fairly linear, but it's not often the case. It is a lot of starting and stopping. Sometimes projects sort of get cut halfway through. Things definitely change.
0:29:42 - Andy B
A lot of these steps overlap and some people estimate that only one out of every five machine learning or AI features that get started ever makes it to the hands of a customer or a user. So there's probably a bunch of teams at Spotify. They have a lot of cool ideas. Only one out of five successfully makes something where, like, yep, we identified the good problem, we caught a good solution. It works really well, it scales really well, we can afford it, ship it.
0:30:15 - Kiran V
Wow, one out of five actually sounds really good. In my experience it's like one out of 10 or one out of 50 sometimes.
0:30:23 - Joe C
It's really important. The initial steps of this, I think, are really important because you know, if you sort of have a bad idea or a bad seed or like some poor planning in the beginning, it will affect things down the road. I'm sure we've all been part of projects that were maybe a bad idea to begin with or were ill-conceived, and then they just kept going and resources kept being put into them.
0:30:50 - Andy B
I'll take the blame for that. So if this sounds kind of crazy to you, imagine this is like an orchestra, right? Like the tuba needs to tuba, the cello needs the cello, the oboe needs to oboe, the violinist needs to violin everyone has their part to play, right? We didn't even go into all the different jobs that contribute to make this happen. And there is a director, kind of giving everybody a conductor, setting the pacing. Who that conductor is, is a different person depending on what kind of company you work in, but in a lot of more modern companies, it's the product manager. So that's what I do. =
The product manager is the person who, at the beginning, is like why are we doing this? Well, it's for our users, it's to make money. Why, why, why? And then, when it comes time to ask is this good enough? The product manager is the one that's saying nope, can't release this yet, no, can't release this yet, while their boss is yelling at them going you're late. We were supposed to put this feature out. Marketing asked for it two months ago. Where is it? Where is it? It's too expensive, give it to me. So that's how a lot of this gets made. Is like people like me, product managers, negotiating with people like Kiran and Joe. What can we do? How quickly can we do it? Can we make a compromise here? This person's not happy, and often it's my job as the product manager to make money for the business. It's Joe's job as the product designer to make the user really happy. Tell me if I'm wrong. and it's Kiran's job to make the whole thing actually work and be affordable. So that's kind of how we make the orchestra play music.
0:32:38 - Kiran V
Yeah, yeah, and we mentioned there's banter that happens between the teams earlier and that is some of that banter. And you know, for example, from the engineering perspective, you know we'll get a set of requirements from product and part of the process. There's a whole process we can actually maybe have an episode about it. Scrum and Agile which is the process that product and engineering teams that are doing software development will generally go through in order to successfully deliver a product and make sure that they're staying flexible along the way, as the user's needs might change throughout the duration of the product or the development process. And so product will come to engineering and ask hey, we need these requirements, when can you get them to me? Like great, that's going to take me, you know, two months to deliver everything.
0:33:37 - Andy B
No, I need it in a month. Give it to me in a month, please.
0:33:40 - Kiran V
Yeah, so already that estimate that engineering has given is too high. So it's like, okay, cut that down. And on top of that, the next week, the product manager will come back and say, oh, we just talked to three more customers and we actually need to add this, this, this and this feature and that other feature that we talked about. Yeah, we need to also expand the scope on that one too, and we need it in two weeks instead of a month. So this is there's a lot of conflict.
0:34:09 - Andy B
That happens a lot of times. People in our three roles - designer and engineer, and a product manager. They don't like each other and they argue with each other because they all have different goals and priorities and again, they have different bosses. That's why working with these two guys is really special, because we we work together really well and it's what makes it really fun. When things are working well and you can play that game, tease each other and find a good balance, you make incredible things and it's really fun.
0:34:40 - Kiran V
Yeah and when things aren't working well, we were able to say this really sucks, let's not do this thing and just walk away from it. And again, like going back to those different steps of the development process, you really want to fail as early as possible. So if you think you've identified a problem, if you can validate or invalidate that that's a problem immediately. That's going to save you weeks, months, years of development, right, if you can. If you prototype a whole bunch of things and none of the prototypes work, if you can say, hey, this is maybe not going to actually solve the problem for the customer, let's not do this thing. That's going to save you a lot of time and money down the road. And so you really want to fail early, which is why you hear stats like one in five or one in 10 projects actually goes through, because they're trying to fail as quickly as possible.
0:35:36 - Joe C
And this is particularly important in today's AI atmosphere or industry, because I think a lot of people hear the word AI or they know what it can do and so they sort of put the cart before the horse and say, oh, AI can solve that. It's kind of saying like let's just do it magically and it takes a whole team to really dig in and see if AI is actually the solution or an effective solution.
0:36:03 - Andy B
That brings us to sort of the last point of like well, you release the AI feature, what now? The product manager, my role I start the project off by saying we need to solve this problem and then my engineering team, my designers, grind, grind, grind. They give me the solution and then I go to the sales team and I'm like I have solved the problem. Marketing team go sell it and you really hope it works.
Because, even though you've done everything you can to prove it out, I can tell you that there was some product manager at Spotify who was responsible for smart shuffle, who they probably turned it on for like 1% of users every day and every single day when they turned it on for more people, she was probably freaking out, being like I really hope it works, I really hope it doesn't break, I really hope nobody runs into any horrible bugs or problems, and the thing is like.. you're measuring whether you solved that problem. So the whole hypothesis was that it's going to be a better experience if you have smart shuffle. So what I'd be looking for as a product manager is like are people staying in the app longer? Are they listening to more songs?
0:37:15 - Joe C
Are they hitting that button?
0:37:17 - Andy B
Yeah, are they hitting that button even, and hopefully I'll be able to say, you know, 90 days after we released this feature, like, hey, look at all this value we've added for our customers. That's awesome and I'm measuring the satisfaction of that. But on the other side, you may have gotten it wrong and sometimes you could have gotten it right and then it becomes wrong. So, especially in the world of AI, you have to do a lot of maintenance. Models drift and data drifts, and what we mean by that is language changes, music style changes.
Not a single person listening to this podcast knew what the word COVID meant in 2019. Now it's 2023 and we all know what corona virus is, what COVID means, what COVID-19 means. Like we've learned new words and just like language changes, concepts in the world change much faster than you would think. So if we release the perfect smart shuffle, probably a few months later, when some hot new music drops, that changes, some genre blows minds, maybe that model and you've got some user feedback that it's not doing very well for Whatever, like 1970s jazz, and you need to go back and fix the model to better predict what needs to get shuffled for people listening to 1970s jazz, and so you're going back through this process Kieran described periodically, to make sure that the feature Continues to perform like you want it, and this is especially important with AI, where it's not one and done because of how quickly the world of AI changes and and sometimes you might just want to do maintenance because something can be done cheaper or better or faster.
0:39:05 - Joe C
There's maintenance to keep things running, but then also, if a product's really successful, you build off of it and you add features off of it. So you see that as well. Maybe you introduce other models or you know like with the smart playlist or smart shuffle. Maybe in the future there's going to be different ways to smart shuffle, so that's a factor as well.
0:39:29 - Kiran V
Yeah, and this is again like a typical software development process, so it's not unique to products that use AI, but the products that do use AI are generally software products, and this is the process that you have to go through in order to develop that experience for the user, to make sure that, again, you're delighting them or solving their problems, depending on what the different use cases are. In the case of Spotify, again just giving them an enjoyable music listening experience.
0:40:03 - Andy B
Yeah, and to close this out, I'll just say, hopefully this gives everybody a taste of what it takes to bring something cool like smart shuffle to a product like Spotify how many people have to be involved and how long and kind of stressful and complicated that process can be for what is, to be quite honest, a very simple machine learning application. Pick the next song how hard could that be right? And so you think about people make self-driving cars. How much more complicated that is than picking the next song to play. Multiply the process we just described. This is why AI is expensive, and when you ask for things on the tools and applications that you use, just know that the people on the other end they want to help you solve those problems. It just takes a long time sometimes to figure these things out every decision can have a big cost. Yes, and it's expensive, and it's risky.
These types of people that get involved in this. Tech people make a lot of money. It's because they have to have a lot of skills and education. Its sort of like what's left of the American middle class, and so, if you imagine all the different kinds of people that I described like, you're paying salaries for those people. Even as they're trying to fail quickly. So, on the one hand, you're encouraging them to throw out ideas quickly, and it's because you're paying them a lot of money and you hope that at one point they hit an idea that's going to be worth a billion dollars and make a billion users happy.
0:41:48 - Kiran V
Talking through this whole process it's crazy that a lot of these things actually make it out from ideas and into the world, because as an engineer, I'm daily in the midst of this process. It feels like I'm constantly.. you know how, when you like trip and you're like trying to like catch yourself before you fall and you're just like running really fast. That's what it feels like most of the days when you're working in engineering. It's just like there's all this stuff happening everywhere around you and all these customers and demands and goals, and it's exciting and scary and frustrating. It's a whole bunch of emotions that happen and somehow, some way you're able to deliver something that makes people so happy. It's just kind of wild to think about.
0:42:44 - Joe C
Yeah, I'll second that. I think it's very wild. There's no end to things you can build. It's kind of all about making decisions to build the right thing. I like to think about how some of the things we talk about ladder up into bigger concepts and really product development, AI product development, it's the scientific method, it's trial and error and I think that's really neat and I'll also agree it's a lot of fun.
0:43:04 - Andy B
What it feels like for a product manager when you're shipping AI is like, you've talked a bunch of executives into help giving you the budget to use all these special talented engineers and designers to solve a problem and you make a promise to them saying you've given me a dollar, I'm gonna give you a dollar 50 back, and you really hope you guess right, because the company is relying on you to guess. You are the chief guesser. You get to decide what is going to help achieve the goals that the executives have and keep the users moving forward. So can be kind of stressful, can be really fun and rewarding. You can build some really cool stuff with that.
I think that's it for this episode of how AI gets made, and how it makes it to you, to your phone. We hope you enjoyed. If you have any feedback, you can email us at aifyipod@gmail.com. Our website is aifyipod.com. We're soon gonna be adding a little form for you guys to give us feedback on that. If you're listening to this on Spotify, please like, subscribe and rate. If you're listening to this on Apple or Google Podcasts, please do the same thing. We'd love to hear from you. Let us know if you have any questions about anything We'd love to hear from you. Let us know if you have any questions about anything we discussed and what you want us to talk about next.