Why your surveys are fundamentally failing you

hand holding pen looking at a survey

In support of the research industry, I understand the importance of gathering accurate and honest data. Unfortunately, traditional explicit survey methods often fail to measure the truth. This is because they rely on conscious responses that are easily influenced by a variety of factors, such as social desirability bias (SDB) or lack of self-awareness. Therefore, I believe that implicit methods are a superior alternative for measuring true attitudes and behaviours.

'Explicit' survey methods involve asking questions and obtaining answers from participants. This type of research is based on the assumption that participants can accurately report their thoughts and feelings on a particular topic in an honest way. However, this type of measurement has significant limitations when it comes to assessing truthfulness.

Social Desirability Bias

First, explicit surveys rely heavily on self-report responses which can be easily influenced by social desirability bias. SDB refers to the tendency of people to provide answers that they think will make them look good in front of others or will be seen as socially acceptable (Gonzalez et al., 2009). People often do this in order to avoid negative evaluations or judgments from others, even if those responses don’t accurately reflect their true feelings or beliefs. As a result, explicit surveys can produce results that are not necessarily reflective of reality due to SDB.

Failing to capture what really drives us

Second, people may not always be aware of their own attitudes or behaviour when answering explicit survey questions (Schwarz et al., 1991). For example, people may say they like certain products but then never buy them because they aren’t aware of their underlying motivations for making those purchases. As such, relying solely on explicit surveys can lead to inaccurate conclusions about what people actually think and do in certain situations since it fails to capture unconscious influences that shape behaviour and beliefs without our awareness.

Self-report data is unreliable

Given these limitations with traditional explicit survey methods for measuring truthfulness, I believe implicit methods offer an effective alternative approach for gathering meaningful data about individuals’ thoughts and behaviour patterns. Implicit measures involve assessing reactions outside conscious awareness through unobtrusive tasks, such as reaction times or eye tracking (Perugini & Bagozzi 2001). The advantage here is that implicit measures are less likely to be impacted by SDB since participants cannot alter their responses in order to appear more socially desirable due to the nature of these types of tasks (Greenwald et al., 2009). Additionally, implicit measures capture unconscious information which may not be accessible through conscious reporting alone (Hofmann et al., 2005); this allows researchers to obtain more accurate insights into people's true attitudes and behaviours without relying solely on self-report data which may be unreliable due to lack of self-awareness or social desirability concerns.

To assess truthfulness, you need to go beyond the explicit to measure the implicit

Traditional explicit survey methods have limitations when it comes assessing truthfulness; I believe implicit measures provide researchers with an effective alternative approach for collecting accurate data about individuals' true thoughts and feelings on various topics without relying solely on self-reports which can be easily biased by social desirability concerns or lack of awareness regarding one's own behaviour patterns. By integrating both types measurement into research designs - conscious reports alongside unconscious reactions - researchers will obtain higher quality data which is more reflective of reality than what could be achieved using traditional explicit surveys alone.

How do you employ implicit methods?

We have a turn-key, cost-effective platform that makes it easy to use implicit and explicit methods side-by-side. Our Neurotech® platform is being used by many of the world's largest organisations to do just that, applying it to D&I, engagement, brand values, risk, learning & development, and social responsibility. So the answer is simple: contact us and we will tell you how.

References:

  1. Gonzalez R., Mehl M., Pennebaker J., 2007 “Social Desirability Bias in Personality Assessment: A Comparison Of Self Reports And Nonverbal Behavior” Journal Of Research In Personality 41(2) pp 494–500

  2. Greenwald A,. Nosek B,. Banaji M,. 2003 “Understanding And Using The Implicit Association Test: IV Applications To Social Issues” Journal Of Personality And Social Psychology 95(1), pp 805–819

  3. Hofmann W., Gawronski B,. Gschwendner T,, Le H,. Schmitt M,, 2006 “A Meta Analysis On The Correlation Between The Implicit Association Test And Explicit Self Report Measures” Psychological Bulletin 132(5), pp 794–811

  4. Perugini M., Bagozzi R,, 2001 “The Distinction Between Implicit And Explicit Attitude Measurement: A Theoretical And Empirical Review” British Journal Of Social Psychology 40(3), pp 347–371

  5. Schwarz N,, Strack F,, Kommer D,, Wagner D,. 1987 "Response Styles In Two Cultures: Repression In Japan And Uninhibited Communication In The United States" Journal Of Cross Cultural Psychology 18(1), pp 117–131

Previous
Previous

DEI Practitioners are being let down by their measurement tools. Here is the solution

Next
Next

Three Steps to Embed Financial Resilience in your Workforce