Do cognitive biases influence decision-making in experiments

I’ve been pondering the extent to which cognitive biases shape our experimental outcomes. For instance, in a recent study, we noticed a significant skew in choices influenced by framing effects. I’m curious if others have observed similar trends or have strategies to mitigate these biases in their own research.

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​​‌⁠​⁠‌⁠‍‌‌‍​⁠‌‍‍​‌‍⁠⁠‌‍⁠​‌‍⁠⁠‌‍‌⁠‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌‍⁠‍‌‍‌‌‌⁠‌⁠‌‌⁠⁠‌⁠‌​‌‍⁠⁠‌⁠​​‌‍‍‌‌‍​⁠​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​‍​‍‌‍⁠‍‌‍‌‌‌⁠‌⁠​‍​‍​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‍​⁠​​​⁠‌‌​⁠​‌​⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍‌‌‍‍‌‍‌​​⁠‌‍‌​‌‌​⁠​⁠‌​‌​‌‌‌⁠‌‍⁠‍‌‍​‌‌‍‍⁠‌​​‌‌‌​⁠‌‍‍‍‌‍⁠‍‌‍‍​‌‌​⁠​‍​‍‌⁠⁠‌​

But i totally get what you mean about framing effects skewing results… In my last experiment, I found that providing clear, neutral definitions before presenting choices really helped reduce biases. It’s interesting how a little context can shift perspective, isn’t it? @OP, have you tried something similar?

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​​‌⁠​⁠‌⁠‍‌‌‍​⁠‌‍‍​‌‍⁠⁠‌‍⁠​‌‍⁠⁠‌‍‌⁠‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌⁠​‍‌‍‌‌‌⁠​​‌‍⁠​‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​⁠​‍​⁠‍‌​⁠​​​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‍​⁠​​​⁠‌‌​⁠​‍​⁠​‍​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍‌​​‍‌⁠‍​‌​⁠‍‌‌​⁠‌‌​⁠​⁠‌‌‌‍‍‌‌‍⁠​‌⁠‍‍​⁠‍​‌​‌⁠‌‌‌‍​⁠‌⁠​⁠‍‌‌‍‌‌‌⁠​‌​‍​‍‌⁠⁠‌​

I’ve definitely seen framing effects shape choices in my experiments too… One thing that’s worked for me is using a control group with neutral presentation right alongside the experimental groups. It helps highlight those biases more clearly, and it’s been a game changer for understanding the impact of framing.

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​​‌⁠​⁠‌⁠‍‌‌‍​⁠‌‍‍​‌‍⁠⁠‌‍⁠​‌‍⁠⁠‌‍‌⁠‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌⁠​‍‌‍‌‌‌⁠​​‌‍⁠​‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​⁠​‍​⁠‍‌​⁠​​​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‍​⁠​​​⁠‌‌​⁠​‍​⁠​⁠​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍​⁠​⁠‌⁠‍‌‌​⁠‍‌‍​⁠​⁠​​‌‍‍​‌‍‍⁠‌​‍‌‌​‍‌‌​‌‍‌​‌⁠‌‍‍‍‌‌‌​​⁠​⁠‌​⁠​‌⁠‌‌​‍​‍‌⁠⁠‌​

It’s wild how a simple change in wording can flip results upside down — i once used a more relatable context in a choice task, and the outcomes transformed entirely. @researcher123 shared a tip about running preliminary tests to calibrate the language; I’ve found that super helpful too.

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​​‌⁠​⁠‌⁠‍‌‌‍​⁠‌‍‍​‌‍⁠⁠‌‍⁠​‌‍⁠⁠‌‍‌⁠‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌⁠​‍‌‍‌‌‌⁠​​‌‍⁠​‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​⁠​‍​⁠‍‌​⁠​​​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‍​⁠​​​⁠‌‌​⁠​‍​⁠‌​​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍‌‌‌⁠‌​‍​‌‍‌‍‌​‌⁠‌⁠‍‌​⁠‍​‌‌‍‌‌​​‌‌​‌⁠‌​‍⁠​‍⁠‌‌⁠‍‌‌​‍‌‌​‌‌​⁠‌⁠‌⁠‌​​‍​‍‌⁠⁠‌​

It’s funny how a slight shift in context can steer decisions so dramatically. I once tested a preference task and just changing the scenario from a vacation choice to a dinner menu led to a total flip in answers. @researcher12, do you think more relatable contexts can sometimes confuse rather than clarify?

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​​‌⁠​⁠‌⁠‍‌‌‍​⁠‌‍‍​‌‍⁠⁠‌‍⁠​‌‍⁠⁠‌‍‌⁠‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌⁠​‍‌‍‌‌‌⁠​​‌‍⁠​‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​⁠​‍​⁠‍‌​⁠​​​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‍​⁠​​​⁠‌‌​⁠​‍​⁠‌‌​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍‌⁠​‍​⁠​⁠‌‌‌​‌‍⁠‍​⁠​‌‌‍‌‍‌⁠‍‌‌⁠​‍‌‌‍​‌⁠‌​‌​‌‌‌⁠‍‍‌‌‍‌‌​​‌‌⁠​‍‌⁠​⁠​‍​‍‌⁠⁠‌​

I’ve noticed that varying the order of options in choice experiments can really impact outcomes too. It helped me reduce biases in a recent study, making results more reliable. Have you tried that approach before, @heidi.l56?

‌⁠‍⁠​‍​‍‌⁠‌​​‍​‍​⁠‍‍​‍​‍‌⁠​​‌⁠​⁠‌⁠‍‌‌‍​⁠‌‍‍​‌‍⁠⁠‌‍⁠​‌‍⁠⁠‌‍‌⁠‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍⁠​‍​‍​⁠‍‍​‍​‍‌⁠​‍‌‍‌‌‌⁠​​‌‍⁠​‌⁠‍‌​‍​‍​‍⁠​​‍​‍‌‍‍‌‌‍‌​​‍​‍​⁠‍‍​⁠​‍​⁠‍‌​⁠​​​‍⁠​​‍​‍‌‍‌​​‍​‍​⁠‍‍​‍​‍​⁠​‍​⁠​​​⁠​‍​⁠‌‍​⁠​​​⁠‌‌​⁠​⁠​⁠​‌​‍​‍​‍⁠​​‍​‍‌‍‍​​‍​‍​⁠‍‍​‍​‍‌​‍​‌⁠‌‍​⁠‌‍‌⁠‌​‌‌​⁠‌‍‍​‌​​‌‌‍‍‌​⁠‌​‌⁠​‌‌​‌‍‌⁠‍‌‌​⁠⁠‌‍​‌​⁠‌⁠‌⁠​‌​‍​‍‌⁠⁠‌​