Welcome to "AigoraCast", conversations with industry experts on how new technologies are transforming sensory and consumer science!
AigoraCast is available on Apple Podcasts, Stitcher, Google Podcasts, Spotify, PodCast Republic, Pandora, and Amazon Music. Remember to subscribe, and please leave a positive review if you like what you hear!
Dr. Michelle Niedziela is a behavioral neuroscientist experienced in both academia and industry. Michelle began her career as a post-doctoral fellow at Monell Chemical Senses Center working on research of functional foods. She continued her career at Johnson & Johnson where she specialized in innovation technologies for consumer products. At Mars Chocolate, Michelle worked on global sensory projects, and in her current role as VP of Research and Innovation at HCD Research, Michelle focuses on integrating applied consumer neuroscience tools with traditional sensory methods to measure consumer response with the goal of providing a comprehensive account of consumer decision making.
Kathryn Ambroze is a behavioral neuroscientist with experience in consumer research and methodological innovation. She currently works at HCD Research as the Manager of Behavioral and Marketing Sciences focusing on methodological development and innovation applying neuroscience and psychological tools to consumer and market research.
Michelle Niedziela on LinkedIn
HCD Research Twitter
Transcript (Semi-automated, forgive typos!)
John: Michelle and Kathryn, welcome to the show.
Michelle: Thank you so much, John. Glad to be here.
Kathryn: Thank you so much for having us.
John: Yes, it's great and I have to say, Michelle, you are the first repeat guest on AigoraCast. You were like guest number five or something and now here we are. Yes, it's been great. Thank you very much.
Michelle: Good to be back.
John: Okay, great. So let's kind of jump into this because something I'm increasingly interested in is the metaverse that I think we can all agree that we're going into a kind of augmented reality as a society. In fact, we're already there. We are in a shared virtual space right now today having this conversation. And actually our guests, in some sense will be joining us in this virtual space, even across time. Right? So, I mean, it's a different world that we live in now. And it's I think, increasingly important that we think about how sensory and consumer science is going to interact with this new world. So, the first thing I'd like to ask you about today is I know you all are developing some new technologies or some new capabilities in the area of augmented reality. So could you please tell our guests about this new space that you're working on?
Michelle: Yeah, it's been super exciting. It was a conversation that I had with the development team. Kathryn and I are on the development team together with Bill and Pat at HCD Research and we always just come up with random ideas and a big motivator for us to come up with new methodologies has been around removing frictions. Friction when it comes to just human behavior. It could be emotional frictions, anything that prevents you from really doing something and thinking about how consumers interact with surveys and other methodologies that we have to measure their behavior. What are some of the things that we can do to really tap into their natural environments, natural behaviors, and also just thinking about that Metaverse, right? Thinking that people are always on their phones. Phone has really become an appendage to people. It's an extension of who they are, how they experience the world around them and so in thinking about all of that and also my daughter likes playing Pokemon Go. When you think about augmented reality, I think that's really the first thing that comes into mind is Pokemon Go, where you can just suddenly see a Pokemon show up in your living room. So taking that into account and thinking, how can we access people in a way that is frictionless? It seamlessly integrates into how they interact with the world and how they interact with products. It seemed natural to find a way to move surveys into a mobile device that doesn't interfere with the world around them. And so we started playing around with the idea of augmented reality.
Kathryn: Absolutely. And I would complement what Michelle was saying with the idea of really meeting consumers where they are. And as Michelle had noted, iPhone is with everyone. People notice if they don't have their phone with them because it's become such an ingrained part of their daily lifestyles and daily routines. So it only makes sense that it's part of their shopping experience already. We may as well just really meet them again where they are. So that way we can get an idea of what it's like for them at the moment while they're actually viewing the product while they're in the environment and they can respond directly through that device and give us a really in-context understanding of what they're experiencing.
John: What are the actual tools or what's the technology you've developed? How does this actually look to conduct these surveys?
Michelle: So what we were trying to do is have them use their phone. So really the only technological device is their own phone. And for us, it's about inserting in the right measurement tool within the space that they can see through their phone. So then, for example, if they are in a store and they're looking at something on the shelf, they can still be looking at the shelf and look through the screen of their phone and the camera is looking at the shelf, and they can rate things in a way that's unobtrusive. So they can still see the product. They can still experience it in whatever way it is, or maybe it is going a step further and thinking about is there a way to put the product in their home? We know that AR is being used to say try out a couch, see what it's going to look like in your living room. But if you can sort of supplement that with a measurement that's also going to be unobtrusive, you can see the product in a natural space and rate it in a way that doesn't actually interfere with that experience. That was our goal.
Kathryn: Absolutely. And the other thing about having it in the home also allows us to get a little bit more of an intimate understanding of their experience, because if they are testing out a product that maybe they would be shy to actually evaluate in the store, we can now garner that understanding, garner the response that they're going to have in the space where they'll actually be using the product. So it was very important there again to have that in-context understanding.
John: And can you give some specific examples then of some use cases here of where this kind of technology?
Michelle: Yeah, we actually just tried this out and so it is fairly new, and it's something we're still working on. And to be totally honest, it's not perfect. There are a lot of things we're still working with. So we've been partnering with UPenn with the Decision Sciences program that they have there with students as a challenge to figure out how to use this. And so the use case that we developed with Bing Ling, who was a student who just recently graduated from UPenn with her Masters. She came up with a way to compare Walmart and Target and three products that are in both stores. So we want people to go to the stores, rate the items through the AR survey, and also what we were able to do is see the comparison among the type of items, but also between the stores. With some other measures that went along with it, like purchase intent, their feelings about the price, they're feeling about the store itself. So we had them rate the store at the entrance and at the exit when they left. And it was really interesting because we saw some certain drivers of behavior. I mean, there were certainly differences in the experience between emotional experiences, between Target and Walmart, but I think taking it even further and thinking, how does that affect your experience of the products, your purchase intent? We actually found that the emotional experience they had from the store had a direct impact on how they rated the different products even though they were the same products. So that was really kind of interesting.
John: Kathryn, did you want to answer that? I have some follow-up questions along those lines.
Kathryn: I mean, Michelle really explained it well, but I would mention as well that while we were designing this study, it is really important to keep in mind the type of questions that you're going to be asking, depending on what your research is going to be. So we decided for this study in particular since we were going to be looking into emotional profiles, we decided to use the self-assessment mannequin because that was a pictorial scale. So it avoided any issues with reading. And it was something quick that they could really quickly tap on and choose what their response was based on what they were looking at and how they were feeling. So it's a really quick, validated scale and that might not be the best approach for every single thing, but for what we were looking for, it worked really well. And as Michelle said, it garnered really interesting results.
John: Right. And so the way this works, then if someone has their phone, they open your app, presumably and so then through the camera on their phone, they're able to see the product. So now what's happening is, like you said, you're reducing the friction. They're able to observe the product through the phone, and they can provide the ratings in a kind of augmented overlay on the phone interface. Yeah. Well, that's great.
Michelle: Yeah.
John: Obviously, I think the next step is to put it into spectacles or some sort of glass, where are you and you're thinking on that? Is that the kind of long-term vision that it will be into frames and this kind of thing?
Michelle: There's a couple of things that I think you could take into. I think in one direction, just thinking about how people's behavior has changed so much in recent years that they aren't really so much even buying online, figuring out the experience that they have with Instore items, right? So thinking that people often use Best Buy as the showroom for Amazon, right? Having people go into a location and show really how they're feeling, what's driving their behaviors around is interesting absolutely. I think could it be done in something else, like glasses? Absolutely. Maybe that would even remove the friction of having to use your fingers to actually answer something. Could you do it with your eyes? I think there's a lot of evidence out there that perhaps using eye tracking with where they're focusing on could add further information to that so there are a million different possibilities that you could take it. But we also discovered a lot of hurdles, too, because when you're thinking about just recruiting in general for studies like this, it can be really challenging and very expensive because you're actually asking people to go to a store. So you have to think about the time that they're taking out to do that, the travel, all of those sort of things and so there are hurdles to it. But I think when we think about where research is going, I think such a huge part of it is kind of coming out of the CLT and going into personal environments that are more contextually driven, more naturally driven, and just also thinking about how people interact with even media. For that, it's a huge deal for us because HCD did not start in consumer products or sensory. We started in communications research, which is ad testing and so much of that is changing the way people experience ads. They're no longer watching television, network television. Right? They are watching things on their devices and so if there is more of a way that you can work with the devices, as Kathryn said, where people are and implement these sort of AR surveys that you can just overlay into the experience they're having very unobtrusively, really then I think you can get really a wealth of information there.
Kathryn: Yeah. The seamless shopping experience is really the goal of any future innovations in terms of shopping and buying and I think Michelle really mentioned beautifully how the buy online, pickup in the store is really what has it's accelerated because of COVID, but that it's not stopping there. There are going to be a lot more things like QR codes being implemented into and integrated into different ads so that way people can just have these shopping experiences where they see it. It's something they're interested in, and it's as frictionless as possible so that way it can get in their hands as quickly as possible.
Michelle: I think there's a lot of research that has been changing that way, too. I mean, you have places like Ibotta, right? Where you just upload your receipt and they're able to get a lot of information. There is a lot of data on people's shopping behaviors just from people's receipts and I think having something similar where you have people sort of opt-in into features like this, into research like this, where they can be members and panelists, maybe in something like an AR survey group or some sort of services like Ibotto or any of the other ones that are out there. But it's really interesting even with the QR codes, you could have more QR codes that are out there in the environment that people can interact with, and AR surveys can sort of being there and pop up right where they are.
John: Okay. Well, more follow-up questions. Kathryn, do you have anything else to say?
Kathryn: No. Please bring your questions to us.
John: Okay. I think when it comes to this, I mean, obviously, we should talk about the interaction with voice-activated technology and with smart speakers, right because that's important. I also want to make sure we get into how this kind of technology can help with chemical senses research when you've got taste and smell versus seems like it's very well suited to some of the more classic UX questions or maybe brand questions. But let's maybe start with the voice-activated technology. How do you see this AR research interacting with voice-activated technology?
Michelle: Well, I think it's a way to remove the hands.
John: Right.
Michelle: Right. I think if we're thinking about even in AR, it can be difficult if you were, say, holding something larger than your phone. If you're holding a full-on tablet right? Now, it's difficult to navigate. But if it could be done through voice, then that removes having to use your hands at all. It also becomes more accessible to more people. It can be challenging for older people to use touch screens or anybody who might have struggles with touch screens in any way. Being able to have more accessibility around this technology is definitely better.
Kathryn: Absolutely. And I think that there is a lot of potential with having the speakers not just be in the moment, but it also provides an opportunity for people. Again, like Michelle was saying, if they were struggling to use their phones because it's cold out or something like that, you can then have the smart speaker to really get that. Again, I will say it over and over again, but that in the context, at the moment answers, because that is really where people are deciding if they're evaluating it if they really like it and those really tough to get at questions.
John: Right. Okay. That's interesting because there are the Echo frames now. Right? And so people can Echo frames. Now, I have not seen it, maybe that some listener or maybe you all know the answer to this. I have not seen anything that is simultaneously voice-activated technology and also AR lenses. There's, of course, I think the spectacles, which are by who is it? I think it's like Instagram has the spectacles or something. That's I think the state of the art, like mainstream lenses and then you've got to do Facebook is supposedly going to come out with something as if they don't have enough data. And then you've got Amazon doing their voice-activated Echo frames but I haven't seen anybody put it all together and I think that's going to be really powerful. I assume that will eventually come.
Michelle: When we're thinking of an actual Metaverse that people are interacting with. It isn't just going to be one modality, it has to be multiple.
John: Right. Good.
Kathryn: I think that's the overall goal, too. When the Metaverse was originally introduced a few months ago or whatever it was, the goal was that it was all-encompassing. It was something that people can go there and have meetings with people in this 360 virtual environment. So keeping that in mind, a 360 virtual environment involves all five of your senses. So until we actually get to that place, there's going to be more work to be done.
John: Right. Yeah. When Facebook branded itself as Meta, this world that they're going into, actually, it's very interesting because there's a lot of research that's going on at the intersection now of UX and Sensory and you see tech companies like Apple is interviewing people for Sensory science roles. So let's talk a little bit about the chemical senses and some of this AR technology. So what are your thoughts then, on using the same way of collecting data when it comes to taste and smell experiences?
Michelle: Well, I think part of our thinking of even implementing this is that we get a lot of questions about how can we measure people at the moment at home. Right? And it is very challenging because then you're dealing with people who have different setups. Some people live in large houses, some people live in very small apartments, some people have children and dogs running all over the place, and some people live alone. And so it's always been challenging to do in-home research and get clean data from it, but accepting that is part of the game of doing this type of research. But we often have people that will say to us that the product that they test at home, tests quite differently than the CLT results that they get, and they kind of want to explore why that is. What are the drivers that are causing this difference in fragrance performance in a shampoo or in a cleaning product at home versus in the lab? And so I think being able to find better ways of measuring that experience that is unobtrusive is really the name of the game. And so if you're talking about a fragrance that is part of a cleaning product, having the person look at the clean surface and be able to rate it or be able to speak aloud while their hands are full of whatever it is that they're doing is important because that is the in the moment experience and that at the moment experience is busy. But it's also full of context. It is your kitchen, it is your mess, it is your situation, and it's your emotions that are driving your ultimate decisions. So it is really important to be able to assess that.
Kathryn: Yeah, absolutely. Expectations absolutely vary depending on wherever that environment is and where you're presenting that product or what it might be. So it makes sense that a CLT won't get the same results that something in the home will find out because you're really testing two different things. Because that person is coming in with this preconceived idea for what they're doing in a lab versus what they're going to be doing at their house.
Michelle: I really think the whole idea of metaverse research is going to have to be the way of the future because we understand better. Absolutely, yes, you're still going to have to do CLT type of research. You're going to have to have trained panels, all of those things. That's very important information. But we spent so much time focusing on the product and not really enough time focusing on the consumer. And the consumer is experiencing these things in the real world, getting that authentic experience and all the drivers of the decision-making that consumers are driven to. Right? Is going to involve understanding that context and it's going to involve understanding the overall experience, the emotions, the sensory, all of it. And really what's going on in a naturalistic environment.
Kathryn: Yeah. Especially because if you think about it, you put all this time, energy, and money into making the perfect product. What difference does it make if the consumer doesn't realize that you made all these amazing innovations? If they don't know it exists and they don't take advantage of it, it might as well not exist at all because perception, truly does become reality in this context.
John: Yeah. This is really fascinating. Okay, now speaking of the Metaverse, I don't know how much you all have talked about this, but I'd like to get your thoughts on NFTs because this is something that's become very interesting to me. Right? Let me just bring you up to speed on where we are in terms of our work in the Metaverse. Increasingly, clients are aware that there is this augmented reality coming and they're wanting to know, okay, what's our metaverse strategy? Just like three or four years ago, what's our AI strategy? Now it's what's the metaverse strategy and it seems to me that for a CPG company, there is kind of three paths, right? One is, how can you use these technologies to increase consumer engagement in various ways, whether it's through education, interaction, whatever, which could include data collection. Then you've got can you complement a fixed sensory experience with technology so that it becomes an enriched experience? Are there sounds that you could play that would make your food taste better, this kind of thing, right?
Michelle: I think we've known that.
John: Yeah. Well, definitely. Yeah. And then the flip side would be, this would be a third path. This is the hard path, is can you create this kind of digitized chemical senses experience where you've got 3D printed food aroma, this kind of thing that is kind of created on-demand in the Metaverse, where the three of us are going to meet at a bar in the Metaverse and have a drink. And Kathryn, you order for the three of us and now our drinks appear and we're going to drink them and it was your choice, what we got. So what are the areas of those three kinds of like engagement, the cross-modal work and then I think really hard science of the chemical senses which I think is more of your background, Michelle, although you have, of course, background in all three areas, which of those are most interesting? What do you see as most promising? What are the areas that you yourself are involved in?
Michelle: For me, it's the engagement. And I think it actually involves all the pillars because engagement is going to be really challenging. So when I think about future thinking and future-proofing consumer and market research, I think engagement is going to be the key but part of engagement is going to be that brand harmony. And that brand harmony is going to be able to bring, okay, the brand expectations versus the product experience and making sure that that metaverse representation, whether it is a sound, whether it is a chemical sense, whether it is an overall emotion, all of that has to match. So the brand has to match the experience and I think that ultimately brands are going to have to think about those ways, these additional sensory paths as ways to engage with consumers that are no longer watching their advertisements on TV. They're going to have to rely a lot more heavily on interacting with consumers in a deep and meaningful way within their own homes and their own environments, because it's not going to be we just had the Super Bowl, and that's very exciting. And people were tuning in, but how were they tuning in? I tuned in to Hulu. I don't know what other people did, but it wasn't watching traditionally on cable network television. Right? So how can you get engaged with people? How can you bring them in? These ads that people are spending billions on and this whole industry of ad research on connecting with consumers through these expensive ads, it's not going to work anymore. It's going to have to be through some deeper interaction, which is going to be very sensory-based, in my opinion, through a Metaverse environment.
Kathryn: Yeah, absolutely and the other thing that I find really interesting is if you compare something like the Super Bowl to the Olympics, the Olympics has been getting a lot fewer views in recent years, and it's because NBC has been taking down a lot of memes or influencer commentary that's been going on. And the truth of the matter is that's where a lot of people are engaging with that type of content. So because they've been removing it, people don't know it's really happening because that's the means by which they're watching things. So you have to embrace it as opposed to trying to just throw it away and try to silence it.
Michelle: That's what you said before, Kathryn. You have to meet the consumers where they are and right now we are in memeland. We are in memes. That's how we communicate with one another. And so you have to go where they are, and that's how you engage.
John: Yeah. I haven't watched 1 second of the Olympics. All I know is they have the ski jumping right next to some smokestacks, and it looks terrible.
Michelle: Yeah, that was a meme, right?
John: Yeah, I think it was real. I don't know why they put it.
Michelle: The question we should be dealing with is how do you make a meme out of, like, these other senses? Because these are all very visual memes. How do you make a flavor meme? That's cool. Like, how do you interact there?
John: Okay. That's a very good idea.
Michelle: Maybe the NFT is actually a meme, but a fragrance meme.
John: Yeah. That's interesting.
Kathryn: And to go back to what you were saying about brand harmony, it was just maybe a week or so ago that Gucci actually sold an NFT for $4,000. This purse and the physical purse itself only sold for $3,000 and I thought that was very interesting. But if you think about it a little bit further, there's only one of those NFTs. So they had a bit of a scarcity mindset involved in the way that they were selling it. While there is maybe you have four or five of those persons that you can go out and purchase at the physical store.
John: Right.
Kathryn: But again, it becomes a question of what is the consumer valuing? What does that Gucci consumer want? Do they want that NFT? Do they want that for the physical product itself?
John: Yeah. We should just mention NFT for people, it means nonfungible token. And so I actually did a podcast, Keir Finlow-Bates was on the show. And if people really want to know more about NFT, they should go back and listen to that. And then they'll understand Kathryn's sentence a little bit. I shouldn't have introduced the topic.
Michelle: We miss a lot internally as well when we think of brand harmony and NFT's, because if brands are really going to be stepping into that space, what does it mean when your brand is now in an NFT space? So if Gucci has an NFT, that almost makes sense because they're being highly valued. Right? But if Taco Bell steps into the NFT space,
Kathryn: which they have....
Michelle: Yes, which they have then what does that mean for the Gucci NFT? What becomes the value we see? What are the perceptions we have around these brands within this new sort of space of NFT, which is totally different than how you experience products normally?
John: Yeah. Okay. This has been great. So we actually are almost out of time and maybe we'll have you back in a few months for NFT discussion. That would be I think it's another area of shared interest, but it's definitely worth keeping an eye on. I think ownership, we were talking before the show has so much of life is stories. It's all made up anyway. And NFT allow us to carry those stories into the virtual world because a lot of life depends on things being unique. Somebody owns them. There's an idea of ownership, which is a story. Right? And things are not interchangeable. That if you go to a restaurant and you Valley Park your car and they bring back a very similar car, it's just not good enough. It has to be your car. Right? It could be the same make and model or whatever. It needs to be your car. The cars are not exchangeable. And that idea that something can be similar to but not the same as something else in the digital world, that's what NFT's brings. That's interesting, I didn't really talk about this, but allowing us to continue our stories in the virtual world about how we live our lives. Okay, so let's finish up with a kind of parting advice because you all are innovating, I think, at the intersection of consumer science and a lot of technology. What's some advice you have for our listeners about things, lessons you've learned along the way as you're innovating in this area?
Michelle: I think the biggest one for me is that you really have to be able to let go. So a lot of us hold so tightly to the methodologies that we know and trust so much. But in order to step forward and move forward, I feel like you do have to let go a little bit and try something new and be willing to accept that maybe the old way is not working out anymore.
John: Right. Like NBC is not letting go and it's getting themselves getting in trouble. Yeah, that's right. Kathryn?
Kathryn: Maybe a little bit more pessimistic look at things. But I would also argue that with all these really cool innovations that are coming out, it is important to also be critical and to really ask the questions. And if the questions and if you ask these questions and the answers don't make sense, push a little further, because if you don't... And you want to be able to explore. But a lot of exploring also means you don't always find gems. So you want to really be cautious while you're looking at these fun things and don't put your life savings into something you don't understand. You want to really be able to understand it and stand by it so that way you can be proud of the innovations that you have and the technological innovations that are coming up because you not only are supporting it, but you understand it and you're able to look at it with a critical eye.
John: So you're saying I should not have put my life savings into monkey JPEGs? That was a mistake?
Kathryn: As long as it wasn't, that with Taco Bell and Gucci and NBC.
John: That's funny. Okay, that all sounds good. Well, thank you both. It's been a lot of fun talking with you, as always and how can people get in touch with you after the show if they want to reach out?
Michelle: Always via the website, www.hcdi.net, and then we are very active on Twitter at @HCDNeuroscience, and you can find either of us on LinkedIn anytime, we respond really well and are open to anybody asking us questions.
Kathryn: We love to meet you in the Metaverse.
John: Or maybe even in real life. Who knows?
Kathryn: Yeah. What a concept.
Michelle: A lot of conferences are coming back online this year or offline, I guess, it's the way to describe it.
John: That's funny. Yeah, that's right. Okay. Thank you both.
Michelle & Kathryn: Thank you.
John Ennis is a leading expert in sensory science and consumer research, with extensive experience in statistical analysis and product development methodologies.
