AI for Biophysical Phenomena: A Comparative Study of ChatGPT and Gemini in Explaining Liquid-Liquid Phase Separation

Abstract

Featured Application Study explores the application of LLMs, specifically ChatGPT4 and Gemini for biophysical research, with a focus on liquid-liquid phase separation (LLPS). Our findings suggest that while both models show promise in facilitating detailed scientific discussions, further refinements are necessary to improve their accuracy and reliability.Abstract Recent advancements in artificial intelligence (AI), notably through generative pretrained transformers, such as ChatGPT and Google's Gemini, have broadened the scope of research across various domains. Particularly, the role of AI in understanding complex biophysical phenomena like liquid-liquid phase separation (LLPS) is promising yet underexplored. In this study, we focus on assessing the application of these AI chatbots in understating LLPS by conducting various interactive sessions. We evaluated their performance based on the accuracy, response time, response length, and cosine similarity index (CSI) of their responses. Our findings show that Gemini consistently delivered more accurate responses to LLPS-related questions than ChatGPT. However, neither model delivered correct answers to all questions posed. Detailed analysis showed that Gemini required longer response times, averaging 272 words per response compared to ChatGPT's 351. Additionally, the average CSI between the models was 0.62, highlighting moderate similarity. Despite both models showing potential to enhance scientific education in complex domains, our findings highlight a critical need for further refinement of these AI tools to improve their accuracy and reliability in specialized academic settings.