ارزيابي كاربردپذيري و عملكرد جستجوي چت‌بات‌هاي جميناي و كوپايلت در مقايسه با موتورهاي جستجوي گوگل و بينگ

eva‎luating the usability and search performance of Gemini and Copilot chatbots compared to Google and Bing search engines

علم اطلاعات و دانش شناسي

Purpose: The purpose of this research is to compare the usability and search performance of Gemini and Copilot chatbots with the two search engines Google and Bing, and to identify a tool with the highest level of usability and search performance among these four search tools. Method: This research is applied in terms of purpose and descriptive-survey in terms of data collection. Due to the fact that this research is done at a specific point in time, the current research is considered a cross-sectional survey. The method used in this research is eva‎luative research. The statistical population consisted of three categories: general search engines, general chatbots, and graduate students. Targeted and accessible sampling methods were used for sampling. The sample size was 60 students (30 students from the Faculty of Biological Sciences and Technologies and Chemistry, and 30 students from the Faculty of Education and PsychologyEducational Sciences and Psychology). Data was collected using a checklist and questionnaire. To collect search performance data of search engines and chatbots in terms of accuracy and response speed, a checklist containing 60 questions from easy to difficult level and judgment criteria was set up. The SUS questionnaire was also used to collect usability data. To determine the validity of both instruments, three experts from the field of knowledge and information science were used, and Cronbachʹs alpha was used to determine the reliability of the SUS questionnaire. The data was also recorded in two ways by graduate students and researchers. Statistical methods (such as mean, percentage, frequency) and Excel software were used for data analysis. Findings: The findings of the research show that the Google search engine has better performance compared to other investigated tools, obtaining the highest score in the usability scale (62.54%). After Google, there are Gemini chatbot, Bing search engine, and Copilot chatbot. In eva‎luating search performance based on search speed, Gemini chatbot was recognized as the fastest tool with an average speed of 10.33 seconds. Bing search engine is in second place with 11.30 seconds, Google is in third place with 13.73 seconds, and chatbot CoPilot is in last place with 16.46 seconds. Also, in answering easy questions, Bing ranks first with 3.33 seconds, and in answering medium and difficult questions, Gemini ranks first with 11.28 and 14.50 seconds. In terms of response accuracy, Gemini has the highest accuracy rate among the investigated tools with 98% accuracy. After that, Copilot with 93%, Google with 91.46%, and Bing with 85% are in the next positions, respectively. The results show that Gemini was generally more accurate in answering the questions and made fewer mistakes. Conclusion: Research findings indicate that currently, traditional search engines such as Google are still superior to chatbots in terms of usability. However, due to the rapid advancements in the field of artificial intelligence, this superiority may change in the near future. Chatbots perform particularly well in answering medium and complex questions. The results of this research indicate that chatbots have the potential to replace traditional search engines in the future. However, to make this happen, chatbots need to improve in various areas such as accuracy, comprehensiveness, and response speed.

5

36430