Approaching Human-Level Data Coding? A Systematic Comparison of OpenAI, DeepSeek, and Human Coders in Qualitative Analysis of Customer Reviews

Abstract

The rapid growth of online reviews presents both opportunities and challenges for qualitative research. Human coding ensures contextual accuracy but is difficult to scale. This study compares human coding with AI-assisted coding using OpenAI GPT-4o and DeepSeek r1 on customer reviews of a complex DIY product. Results show both platforms capture underlying relationships, with OpenAI aligning more closely with human coding and DeepSeek demonstrating stronger internal consistency. Systematic AI errors mainly take the form of conservative Type I errors. Findings suggest AI can complement, rather than replace, human coders to enhance scalability and efficiency.