Dr. Garmt Dijksterhuis - The Science of Validity

Welcome to "AigoraCast", conversations with industry experts on how new technologies are transforming sensory and consumer science!

AigoraCast is available on Apple Podcasts, Stitcher, Google Podcasts, Spotify, Podcast Republic, Pandora, and Amazon Music. Remember to subscribe, and please leave a positive review if you like what you hear!

Dr. G.B. (Garmt) Dijksterhuis studied experimental psychology and methodology at the University of Utrecht and wrote his Ph.D. dissertation at the department of Data Theory at the University of Leiden, in the Netherlands. He has written or co-authored some 150 publications in sensory and consumer science, statistics, and psychology. He is one of the founders of the Sensometric society and was its first president, and is a member of the editorial board of the journals Food Quality and Preference, the Journal of Future Foods and Frontiers in Psychology/Eating Behavior.

He has some 30-year experience in sensory science and experimental psychology and worked in academia, research institutes, and in the food industry. He has done research on a variety of topics including multisensory perception, consumer (food) choice, and sensometrics.

Garmt taught courses in sensory science and methodology and related topics and has been a guest scientist at several universities and research institutes. He is an invited speaker at many scientific conferences as well as at meetings for a more general audience.

From 2009-2014 he was Honorary Professor at the section Food Design and Consumer Science at the University of Copenhagen, Denmark.

Garmt is currently teaching at the experimental psychology department of the University of Utrecht.

Garmt Dijksterhuis on LinkedIn

Transcript (Semi-automated, forgive typos!)

Danielle: Welcome to the show, Garmt.

Garmt: Thank you. Happy to be here.

Tian: Yeah, it's very impressive. While I was listening to Danielle's introduction, I didn't know that you were the first president of Sensometric.

Garmt: Yeah, we set up the whole thing quite a while ago.

Tian: Our show here is to have conversations with an industry expert on how new technology transforming sensory and consumer science. And we always like to start by asking the question to our guests, how did you start your journey in the sensory field?

Garmt: Right, well, as Danielle said in the introduction, I started psychology at the University of Utrecht. And one of the professors there was Ed Kosta, a famous olfactory at Taste Scientist, and I was studying with him and then I said I want to do visual perception. And he said, well, in smell, there is so much more to discover. So I switched to food-related perception because he advised that and I stayed in the field of sensory and food-related perception ever since. But that was how I got into this field of food-related perception, and food behavior.

Tian: That's interesting. When was it?

Garmt: When was that? I think I graduated in 87 from the University of Utrecht or 89 or something. Yeah, it was quite a while ago.

Tian: Yeah, because when I was trying to get into neuroscience, I was told the same thing. The smell, you know, I wasn't in taste and smell was at the time it was just got his first Nobel Prize on the smell. That's how it got to around the year 2000. That's when I got to the field.

Garmt: Yeah, it's an amazing field because the sense of smell is related of course to food perception. But the smell is such an intriguing system compared to all the other sensory systems. I think we still have so much to learn there.

Tian: What's your experience after that?

Garmt: And then I was graduating in this extra mental psychology and also the methodology statistics group at Utrecht University. And as a student, I worked at the university in the sensory analysis group. And then I was switching to work with Peter Punter. He was also in the university back then and I switched to his company because he set up his company in sensory analysis and stayed with his company for a while doing data analysis and helping out with the sensory stuff. I had a lot of jobs in general. I think I switch jobs every six years. So I worked part-time at a university of Learning in the marketing department. Of course, I went to Copenhagen as an associate professor for a few years. I've been in Unilever research for seven years or so. I switched to the Wageningen research department. I'd been there twice and now I switched to Utrecht University to teach in the same group from where I graduated a long time ago.

Danielle: So you have experience both in industry, in academia, of course, but also in research agencies. Is that role of sensory, how is that different? Can you talk a little bit more about that? How that is different in those three different situations?

Garmt: Yeah, well, of course in the industry there was always discussion, do we still need the sensory department or don't we need it? When I started at Unilever, it was a big group in total and now it's a small group. I don't think they have their own map anymore. It's always a discussion. But I think everywhere the recognition that sensory research is important still stands. Whether they do it themselves, the industry or they outsource it to different places, I think they still realize it is important. Maybe not always to the same degree, but it is recognized. And imagining, of course, it was also the sensory department and the law of research around sensory, which was also basically on a commercial basis because we did the research for those who wanted research, but it was also subsidy-based research. And of course about having a university, which is not research but university they have the sensory department for education. Educating sensory scientists with a little bit different focus of the really applied sensory analysis.

Danielle: Yeah, and one recurring interest of you is also the validity of test methods. I think that's a really interesting topic for our field always to see how valid methods are representing reality. But maybe for our listeners, you can explain a little bit about what you did in that part, what you investigated.

Garmt: Yeah, certainly. I think that's due to AJ Kempster, my professor in Psychology and Utrecht University. He always stressed the relative value, maybe sometimes the low value of what consumers say and what consumers themselves mention as the motivations for the behavior because very often consumers cannot access their own hidden motivations. So when you have a research method that asks consumers things, I agree with AJ Kempster that the validity of that outcome is very low because consumers cannot access their own motivations very often. So if you do consumer science, you have to make sure that you have a method that gives you valid results. Preferably you have a method in which consumers do not know that they're studied or at least they do not know the research question or they give natural answers. They do not go to think about matters that normally they don't think about because in food perception, in the food purchase, a lot of behavior is automatic. People don't really think about it or ponder it deeply. So if you ask them, they start thinking about things that nobody can't think about. So that is a threat to the validity of consumer methods in food studies and also in general in marketing. But food is such a special thing of course. Food is a very special stimulus. If you do not buy food, you die. You have to have it. So from a psychological and biological point of view, it is very special.

Danielle: Yeah, and it becomes in sort of a way, part of you.

Garmt: You put it inside your body so it really is a special stimulus, special material. And that may also lead to the fact that you may need special methods if you want to investigate why people do eat or buy certain foods.

Tian: Right. Another word to explain because I actually don't know the word validity before I read the paper a few days ago. So I'm understanding it as you're trying to make sure the methods that we're creating as sensory consumer scientists and what the consumer thinks about it kind of overlap or we're looking at the same thing. So we're testing the real things that the consumer is perception.

Garmt: Exactly. Validity is the idea that you have a method and that the method measures what you think it measures. You want to measure the liking of food and you want to make sure that your measurement methods also actually reflect that. And if you ask the consumer if he likes the product, that's fine, everybody can answer that. But if you ask the consumer why did you buy or why do you like the product? That fairly often does not give a valid result because people do not know why they like it. They just like it. The reason that they think they like it, it's probably disconnected from the actual underlying reason. Customer always said the problem with asking a consumer a question is that they give an answer.

Tian: Right.

Garmt: Because you do not always know what the answer is worth.

Tian: Right. You know in sensory if you ask consumers liking what they actually say may not really reveal what they really like or what they want to buy. Then you ask about purchase intent.

Garmt: Exactly.

Tian: That may still not be the one that they will actually grab it. So you're trying to match like testing of the method is matching with these things that you put on the scale. Interesting.

Garmt: Exactly.

Tian: Right. So that actually leads to another question that we're very interested at is, these days we have a lot of new technologies. We're trying to invent new metrics to represent consumer liking, and purchase intent, or using other technology to help with that process. Some of those are like neurobiology basis of methods. Some of are like the multisensory immersion room VR type of technologies. So what do you think about those?

Garmt: Well, you mentioned purchase intent. Of course, intent is, I think, very often maybe not a valid measure because intent is something you express. I intend to buy that and that. But in the real world, when you are there in the shop or wherever you are to buy food, there is so much more distracting you from having that exact thing that you earlier said that you would buy. So the intention is not a good measurement. You mentioned neuroscience. Neuroscience can be a good measurement, but it is true that neuroscientific measurements are very often remote from a real situation. Of course, when you talk scanning, that's not real at all. That's so weird. Very weird environment, very noisy. And then of course, it's the idea that still if you have a connection between neuroscience and behavior then the consumer is really in another situation, you still don't know if he or she will do as you would predict from neuroscience. There may also be difficulty in getting the connection between actual behavior in real life and your measurement, which is neuroscientific. And that brings us to the other thing you mentioned, the immersive methods, like virtual reality or other immersive methods. I think that may be a very interesting way of actually getting a more valid response in the lab. Because if you are immersed in a situation, it may feel really like that situation, and then you may be able to display natural behavior concerning the food.

Tian: I see. So you're thinking those VRs or immersive rooms are part of creating a testing environment that create more validity to the actual..

Garmt: Yeah, because if you are in a VR situation, by the way, in the VR situation, it's still difficult to handle food. But that may be a different problem. But if you are in a sort of projective situation where you have a situation projected all around you, then you actually feel a little bit like you're really there and you may feel like it's a real situation and that may prompt you to display natural behavior concerning the food. So if you ask me to gamble on the validity of the method, I would rate the immersive type methods higher than the other methods.

Tian: I see.

Danielle: Yeah. I think in addition to that, I think those virtual realities should not be standing on its own, but I think people should also that is also what we saw in previous research that they should see at how can we get people also mentally into that state so that they, for example, by asking questions before the test to get them really into this situation where they normally would make a decision on buying things or choosing to eat something. And I think both together can be really powerful.

Garmt: You mentioned it because in theory, giving a consumer in the test situation or the sensory person in the test situation. A good story and good instructions is a way of immersing them in a situation. If you read a good story, you make up your own situation as a subject. You may even feel that you are there in a situation even from a good story. But that may already help really in getting natural behavior.

Tian: Right. Because this is a very interesting topic because I recently got a VR headset and I was doing a lot of things in virtual reality myself. First of all, I would say, like the games that I was playing and whatever I can do with a headset right now it is very real. But when we combine the sensory experience with that there's still like ways to go. I say it's not immediately available and of course, we can kind of create a new environment that kind of replicates the natural environment and we try to do sensory tensing it. But I wonder, is that VR or immense room experience really matches what you're doing in reality? I feel like we've seen research done on that and I think, in general, they don't find any difference if you're just doing in a lab and just doing testing in a lab, cube or you do it in the immense room. I was thinking the immense room or the VR actually gives an opportunity for consumers to really interact with that kind of enrich your eating or your sensory perception environment a lot, even within consumers so instead of them just doing it in your real reality but they do it in natural reality. That's my current reason for thinking about how this thing would go.

Garmt: I agree. Exactly. Because you immerse people in a situation by these emerging projected rules or by VR and as soon as you feel like being in a situation, you may indeed behave naturally. And that's the whole point of setting up these things, of course, because it's much if you have a lap like that than really going out and find a real situation or build a real situation for subjects. It gives you control and it also may give you some of the validity that we need.

Danielle: Yeah, because that is of course always the gap. If you do it in reality, in a normal environment, how can you control what is being done

Garmt: Exactly. You have a lot of noise going on and you're not in control. You can do home-use tests, but you never share what people do with the food that you hope to do with them. So you miss control in those situations and this immersive VR type method may give you that control indeed. But I'd like to stress that giving a good story may also work. I've some students now looking into that. We're trying to compare VR measurements, the same measurement, but only with a story read to people. Trying to set up a study to really compare what's the difference. Because of course, it's much cheaper to read a story than buying VR glasses so I'm curious to find out if there is a difference.

Danielle: Yeah, but we found also with this study on that storytelling or putting them into context, that it's really important to let people not just hear the story but really being engaged in it by asking them questions about what would you do in this situation because otherwise, you could just see it as somebody else's story. So really making them engaged into what they are going to do.

Garmt: Yeah, I think that's the keyword, engagement. Exactly. Engagement, I think is the key thing. If you feel engaged in a situation, it probably doesn't matter if you see it or just imagine it. As long as you're engaged, you're probably inclined to perform, to behave naturally. They have skills, I think, to measure after the fact how engaged or how immersive felt or otherwise find out how real they thought the situation was or how they felt during the test.

Tian: Yeah, but I think that's the key for these new technologies, a lot of them it's like a pretty picture right now, but the key underlying that is trying to engage with either the consumer with the environment, with the consumer with another brand or whatever. This engagement is actually the key of all this new technology, web 3 or VR that trying to help can bring us value as sensory consumer scientists. You also mentioned like using EGs or implicit methods to help these physiological measures. So what's your thought about it and I know there's a new term, it's not new anymore, but the neural marketing part of it. What are your thoughts about that?

Garmt: First, you mentioned EG and then you said implicit methods. There are much more implicit methods than neuroscientific methods of course. You can even have a survey with implicitly hidden questions. Very often people say implicit methods is neuroscience method, but that's not true. Implicitness is yet that the person doesn't know that he's being tested or doesn't know that he's being tested for something specific. Implicitness is not necessarily only neuroscience, but with respect to neuroscience methods, I'm as skeptical about neuromarketing. I think it's oversold a lot. I think people sell neuromarketing because it sells, it's a buzzword. But I am doubtful very often about the validity of those results because it's a weird situation you're in. I'm not quite sure if the results link to actual behavior. And you're talking and scaling, the FMI data is so complex and so difficult. It takes ages to analyze and a lot of steps in between. Sometimes subjective steps to analyze the neuroscience data or clean it, or modify it in order to get some good results. I find that difficult to see in a sort of neutral context. I'm always wondering, is the analysis of such complicated neuroscience data, is it really neutral? That doesn't something creeping of selection of subject, selection of good data, things like that.

Danielle: Well, what would be your response if people would say, well, I'm very interested in this area, so I just want to explore this. Because I know you have been involved in some of those.

Garmt: Yeah, we did it. But the thing is, you have to have a very specific question, I think, and study that in a neuroscience context. I don't think you could study a very general question like what product would you buy? Would you buy AOb in such a context? There are much cheaper ways to doing that. But as you have very specific questions about types of reactions that you know, have a correlation in the brain for specific situations or stimuli, I think you can do it. I mean, even emotions are very hard or impossible to point out in the brain. That doesn't work. So you have to have a measurement that you know is reflected in brain activity at some location. And if you're interested in that in response to some stimulation or some situation or some information to a subject, I think then it's perfectly okay to do FMRI. But translating that to a real market-relevant situation, I think it's beyond what we currently really can do in a valid way.

Tian: So what do you think about wearables? So those data may not be like EGs or MRI scans, that type of data, but the heartbeats.

Garmt: The heartbeats and things like that, and skin response, autonomic measures so they tell you something about interest and involvement and arousal concerning the surrounding situation, products that people are in. So they give you the information. So a wearable may work well, but of course, you do not know if people have a wearable they're worn for a week. You do not really know what situation they were in or what product they were involved with. If you find something interesting in the data. If you have wearables in a controlled environment, the testing environment, of course, you could find out.

Tian: Right. So, of course, it will be part of a study or of course, I was just thinking it will be really good if it combined with like a VR or immerse room and then you collect the physiological data that along with that. That might be a useful thing to look at, right?

Garmt: Yeah. But also here I would say you need a very good research question. So we expect an autonomic response because of that reason and then you can investigate it. I'm not sure if these responses can tell you preferences for subtly different products. And especially, of course, in food, there are no bad food products. I mean, some are maybe a little bit nicer than others, but the differences you study are very small. I remember Danielle, you may also remember in Unilever, we explored the area of positive psychology because bad foods, you don't know, you don't make bad food as a company, you make nice food. And the thing is, is the food nice or is it a little bit nicer? How can you test that? Those differences are so small? They are all nice.

Danielle: Indeed, Garmt, because one of the things that you studied also a lot is multisensory integration. And I think that's also very interesting because that also touches upon all those new tools and things like that. So maybe can you explain a little bit more about this topic?

Garmt: Yeah, actually that was also the field that I was starting with in Unilever. They asked if I could settle this. You will notice your integration for it. Well, of course, traditionally when we talk about food, we think of smell and taste, maybe texture. But in addition to that, you see your food, you hear your food. Food has a temperature. Food has haptic properties. You grab it. It has a weight, it has well, and there are many, many sensory systems that are active in food and in eating. And even after swallowing, you feel it in your stomach, I mean, things like that. And if you realize that there may be 1012 or 15 different types of sensory systems active when you eat a product, you want to know the interaction of all of those. When you make a new product. It's much more than smell and taste. And of course, if you think about the reduction of salt and sugar, you may be able to indeed reduce sugar, but compensate for that in another sensory system. Maybe you do not need a lot of sugar if it sounds crispier or something like that. So there are a lot of ways you can study the interaction and maybe compensate for making stuff more healthy. And in addition to the multisensory bottom-up integration also has top-down information. Top-down effect. That is information you already have in your system, in your memory, in your psychology, that also affects your perception of food. If you know something about your food, it'll affect how it tastes and what you find of it. So all this together paint a holistic very complex picture of the food you eat. But if you want to make new and appealing food, you may want to know all about these systems and how they interact. Also packaging pleasant role. The way you unwrap a product. I mean the sound of the pack has an effect on your flavor perception.

Tian: That's right. So that is more like a cross-modal association thing.

Garmt: It is quite a multisensory or cross-modal, it's all that. Exactly.

Tian: We're always thinking about in the VR or like the web3 metaverse world, how can we bring chemical senses into the metaverse. But then that's a good topic, you know, back of our head that we're thinking about.

Garmt: It's not going to work. I'm convinced it's not going to work for the next 150 years because if you talk about smell, you talk about a system with 300 plus different receptors. You will not be able to tune them. To address them. I mean the vision you have four receptors in your eyes. That's why VR works. You see it. The sound is fine, we understand that. But in smell, it's not going to work.

Tian: Because what we're thinking, probably in the metaverse world, we're not going to have the actual say smell or taste that transmits through your cable line. Put it that way. Although there are devices that they try to, there are headsets and there's the device that also put a little bit of the smell chamber that you can smell.

Garmt: A little bit, only a few compounds and they spray the nose and that may give some smell. I mean that we're able to capture the whole wide world of the trillions of smells you can perceive.

Tian: So that's the cross-modal association part, right? Because one idea is if we can kind of align that like your sensory smell, your chemical sensory smell along with the cross-modal association with the other stimulations like your vision, your auditory stimulation then along with the actual physical food that actually in front of you that you are eating and drinking that might be creating like a weak format. Because of course, John was always talking about the strong format and the weak format, the strong format, I agree with you. I actually totally agree with you because those receptors are so complicated even on a biological level, we don't really understand that to great content, how those smells are working, taste, and working. But then for our creation of this metaverse, we try to replicate it and we try to create an environment that all of us can be in that environment from different parts of the world but experience the same thing where the thing that really replicates or not replicates but resample the situation, the environment that you're in.

Garmt: Yeah, you mentioned a very weak version of it. I would say an extremely weak version of it in terms of smell. I mean, take the smell of coffee. 800 different chemicals do that for you. You're not going to have a good cup of coffee in your VR system. Never. It may smell of coffee a little bit, but it may smell of one type of coffee or maybe two if you're lucky. But never the difference subtle, difference between all coffees that you have. So I'm quite skeptical of that.

Tian: Right. I'm skeptical too. But I will not say never. I feel like there's a way that with AI and the new technology involved, we will be able to get closer to that. It's probably extremely hard to replicate the true experience.

Garmt: Have you ever seen the stand that a perfumer uses or flavorist uses to make a new flavor? You've got thousands and thousands of little vials with odors that could never go on a habitable mind.

Danielle: And what about, maybe a bit more scientific or science fiction-like, but what about stimulating directly in the brain? Because, for example, with COVID people get really weird smells from things that they normally knew. So there must be something direct.

Garmt: I know it happens in vision, of course, for the visually impaired, you can have maps and it turns on the occipital visual cortex. In smell, you still have those hundreds of receptors you want to and nobody who has a normal sense of smell will volunteer to have the sense of smell replaced by electrodes of his nose. I'm not going to do this. In theory, yes. But then we need to know the connection of the molecule at the receptor and we do not know that connection. That's a prerequisite, of course, before you can do that.

Tian: Yeah, that's a part, Danielle....

Danielle: And the part from the receptor to the brain, that part got to be something done there.

Tian: So the thing is that was actually very close to what my previous research was. First of all, your gas stationary cortex and olfactory part of the brain are very insightful. It's actually very close. It developed evolutionarily. It's one of the first senses that you develop. So in the brain, it's embedded in the very core office. So it's not on the visual cortex. Why those are easier to fix because that's our exposed, you can just do it on top of your head. The other part is because it's so hard to get into it, it's actually difficult to research. So on that part, how beyond that receptor level, how do we go to the central neural system? That research question is like the core question that today's researchers and scientists are asking. How are those higher levels of your brain functioning related to sense smell and taste? That's a key question that people are asking in the scientific world right now.

Garmt: And then if you understand the brain, you do not necessarily understand behavior.

Tian: Exactly. That's why we try to ask questions. We ask questions. We try to get it out of your mouth and we feel that's probably the easiest version of how you think. But that may not be true coordinates between each other.

Garmt: That's right. I must admit I believe in free will. I'm not a neuro-determinist. But that brings us into the philosophical study and philosophical discussion. There's still quite a distance between neuroscience and explaining behavior.

Tian: Yeah, it's definitely a way to go. But I still feel very hopeful, though, because I just feel a lot of things that we never thought about, say, 5-10-15 years ago, that we are actually experiencing right now. So we don't really know what happened in 5-10 years later, right?

Garmt: True.

Tian: Yeah.

Garmt: It's a great future ahead.

Tian: For sure, I used to be very skeptical when I joined Aigora, actually. But throughout my years here, I become a lot more optimistic about how new technology could be affecting and driving the world forward, in a sense.

Garmt: You can take the whole chat GPT discussion, for instance.

Tian: Exactly. I was just trying to.

Garmt: Who was thinking about that 10 years ago? Nobody

Tian: I know. It's more interesting than talking to a common person with, you know general knowledge. It's very good.

Garmt: I don't agree. Persons are always much more interesting. Real person.

Tian: It's been a great discussion today, Garmt. We talk about lots of things, and I feel time just flies. So our last question is always asking our guests to give some advice to the new sensory consumer scientists in today's field. What's your biggest advice, recommendation, or suggestion to them?

Garmt: Yeah. Thank you. Yes, I thought about that. You may have noticed that I am a little bit skeptical about lots of attention that neuroscience get. I think they getting enough attention. If you talk about understanding food-related perception and behavior, I think there is a need for more understanding of exactly that perception and behavior in food not necessarily mediated by neuroscience. Marketing science and psychology in general are built on stuff that is not food related. So my plea would be for psychology, which is food related, it's food based, because in food, a lot of psychological laws may not work in food because everything is different there. So we may need to build a new psychology based on principles from food. And I would say that if you are a young and aspiring sensory consumer scientist interested in food perception, and food behavior, I would say look at all the fields, also neuroscience. But if you really want to make a breakthrough, try to find a new psychology of food behavior and food perception and go into this area.

Tian: That's true. That's a new thing that I heard. Yeah, definitely. As I said earlier, this taste and smell is one of the original tastes that we developed. So, yeah, it tends to be more conservative to us.

Garmt: It's so deep in the system.

Tian: Exactly. It's not easy to.

Garmt: Yeah, so difficult.

Tian: Okay. It's a great conversation.

Garmt: Thank you.

Tian: Yeah, we're very happy. Talk to you next time.

Danielle: Thank you, Garmt

Garmt: Well, I was very happy, too. Thank you.

Okay, that's it. I hope you enjoyed this conversation. If you did, please help us grow our audience by telling your friend about AigoraCast and leaving us a positive review on iTunes. Thanks.