Note: Junmou is the XJTLU ChatGPT platform with an array of innovative characteristics developed by Learning Mall (LM) using the OpenAI model, and was officially launched on campus on 28 Jul 2023. The newest version of ChatGPT4.0 was also published on 27 Nov 2023. It is available to all our staff and students.

 

In our recent undergraduate year-4 teaching, I conducted instructional sessions on questionnaire design. Upon observing students' reactions and behavior in the classroom, I have some reflections to share particularly regarding the students' use of XIPU AI (Junmou) (https://xipuai.xjtlu.edu.cn/).

 

Overall, Junmou showed the ability to create structured questionnaires, swiftly generating questions and options, thus saving time and labor costs in market research. Traditionally, designing a comprehensive market survey questionnaire may take several days or even weeks, and it involves considerations such as appropriate question selection, option delineation, and the establishment of logical progression. However, with the assistance of AI technology, this process can be significantly expedited. The use of AI technologies like XIPU AI streamlines the questionnaire design procedure, enhancing efficiency and standardization in designing survey measurements. Nevertheless, potential drawbacks may include the potential diminishment of students' reflective thought processes and the attenuation of human designers’ capacity to incorporate professional knowledge and experience in defining questions, options, and questionnaire flow, thereby impacting the questionnaire's validity and effectiveness. The following example illustrates this point.

 

Drawing from our experience in conducting a seminar on questionnaire design: a group of students aimed to design a market research survey for their start-up, focusing on smoking cessation products to understand the potential consumers' demand and acceptance of the product. With the help of XIPU AI, the observed group harnessed Junmou's potential and automatically generated a questionnaire based on consumer preferences using existing smoking cessation market data and algorithms. Creating such a survey manually involves significant time investment and undergoes iterative editing and refinement. However, AI-assisted questionnaire generation allows for swift alignment with research objectives while mitigating human error and bias.

 

This example demonstrates the efficiency of AI in automatically creating questionnaire designs, notably reducing time and resource expenditure in market research endeavors with improved efficiency. However, AI-generated questionnaire designs still need to undergo manual review and testing to ensure accuracy and effectiveness. Consider our students' smoking cessation product design: When we asked XIPU AI with the keyword "smoke quitter service," one of the questions involved whether potential consumers had ever tried quitting smoking. The initially generated content primarily focused on products and methods in foreign markets, such as "Cold turkey," nicotine replacement therapy (such as patches, gum, and lozenges), and prescription medications (such as Chantix and Zyban). Yet, upon requesting AI to substitute these with locally relevant alternatives, the questionnaire content better reflected the local context. For example, "Acupuncture" emerged as an option for smoking cessation methods. This approach enhances user experiences as questionnaire alignment with respondents' daily lives enables considerations of their experiences and comprehension, facilitating the formulation of more suitable and easily answerable questions.

 

To provide another example, the reliability of questionnaire measurements is a crucial concern. Typically, statistical methods like Cronbach's alpha are utilized to assess the reliability of measurement tools, requiring analysis with actual data. In order to gauge the level of involvement in a product category, we provided students with questions based on five dimensions derived from the existing literature. They are: (1) I am interested in _______ in general. (2) _______ are important to me. (3) I get involved with what _______ I use. (4) _______ are relevant to my life. (5) I am going to purchase _______ in the next six months. According to the literature, the reliability on this scale ranges from 0.81 to 0.85 (Cho, 2001).

 

However, some students choose to skip this step and generate the scale with just one click using AI (with the emphasis that the questionnaire design adheres to academic standards). The scale generated by XIPU AI is as follows (also consisting of five dimensions): (1) I consider myself highly involved in the [product category]. (2) I actively seek information about new products within the [product category]. (3) The [product category] is important to my daily life. (4) I spend a significant amount of time thinking about the [product category]. (5) I am willing to spend more money on high-quality products within the [product category]. Although this approach generates the scale quickly, its limitation arises from potential reliance on extensive data, rendering the AI incapable of providing historical reliability values. Consequently, it is not possible to predict the reliability of the scale before scale deployment.

 

To sum up, questionnaire design, as one of the most common social research methods, is frequently employed to collect and analyze data related to people's behaviors, attitudes, opinions, and perspectives. Effective questionnaire design demands a profound comprehension of the research domain, inquiry focus, and the characteristics of the intended respondents. This necessitates a nuanced understanding of the societal context, participant contexts, and practical intricacies. Qualitative research methods such as in-depth interviews, observations, and literature reviews can provide a better understanding of the complexity and relevant factors within the research field. While AI can significantly assist questionnaire designers in improving work efficiency, researchers cannot completely detach themselves from social practice and experience. Depending solely on AI for generating questionnaires may present challenges, potential hazards in data analysis, and result interpretation. For instance, AI may struggle to accurately understand and interpret complex social concepts, human emotions, and implicit information. Additionally, AI-generated questionnaires may overlook specific cultural nuances and fail to comprehensively address crucial aspects such as logical question sequencing. Therefore, from the pedagogical perspective, combining AI technology with human expertise can indeed enhance the efficiency and standardization of questionnaire design. However, to ensure the effectiveness and quality of questionnaire design, researchers must possess an in-depth understanding of the research domain and participant characteristics. Engaging in qualitative comprehension and judgment based on real-life scenarios becomes imperative, ensuring that the generated questionnaires adeptly and effectively capture the requisite data.

 

 

 

Cho, C. H., Lee, J. G., & Tharp, M. (2001). Different forced-exposure levels to banner advertisements. Journal of advertising research, 41(4), 45-56.