Imagine surveys that aren’t just mundane questionnaires, but personalised, engaging conversations tailored to each participant. That’s the magic of AI survey chatbots. They don’t just stop at reading responses; they dive deeper, asking the right follow-up questions to unveil layers of valuable insights.
It almost sounds too good to be true, so we embarked on a journey to explore the impact of integrating an AI survey chatbot into surveys and whether claims of richer, higher quality responses are true!
We set out on a mission
Our mission was clear – to gauge the influence of the chatbot on open-ended response quality and participant satisfaction.
Using a ‘test and control’ approach to trial the chatbot with 1,500 UK respondents, we split all the respondents into 3 demographically matched groups, each experiencing a different type of probing.
- Typical probing (non-AI)
- AI probing without targeting
- AI probing with targeting – in this case, targets (words) of interest and trigger actions can be specified, e.g. “ask a specific probe when a target – say, a brand or product – is, or isn’t, mentioned“
With a robust scoring system, we were able to evaluate responses based on their length and depth of content. The richness of each response was assessed by a human researcher, using a specific set of scoring criteria.
The results are in
Drumroll, please… Our survey findings showcase very positive results towards AI and this new tool. It seems that integrating AI probing into our survey process has sparked a surge in both the length and depth of open-ended responses, cutting across all key demographics. Let’s delve into the specifics:
1 – Character length increase
Respondents exposed to AI follow-ups witnessed a staggering 79% increase in character length, while those probed without targeting experienced an even more impressive 86% surge. Talk about boosting engagement and depth of insight!
2 – Quality score boost
We saw a big lift in the quality of the AI-probed ends too, with the quality score almost doubling. When informed they had encountered an AI-generated open-ended question, 58% of respondents admitted to investing more thought into their responses when encountering the AI probing.
This shows this new tool is not only capturing their attention but also encouraging deeper reflection and engagement.
3 – Positive respondent experience
An overwhelming 85% of respondents sailed through the survey experience seamlessly, reporting a smooth journey from start to finish, with 75% of respondents saying the AI question that was asked felt natural. We also saw a slight positive uplift in terms of those who said they like the open-ended question “a lot” for those who were asked an AI probe.
Summary
We are excited to launch our new tool – an AI survey chatbot, specifically designed to help our clients gain deeper insights and make more informed decisions based on longer and better quality open-end questions from their surveys.
Hold onto your hats, because the future of surveys is here! From crafting questions to unravelling insights, AI is revolutionising every aspect of the survey game – and trust us, it’s just getting started!
