Mining biometric data to predict programmer expertise and task difficulty

Abstract

Programming mistakes frequently waste software developers' time and may lead to the introduction of bugs into their software, causing serious risks for their customers. Using the correlation between various software process metrics and defects, earlier work has traditionally attempted to spot such bug risks. However, this study departs from previous works in examining a more direct method of using psycho-physiological sensors data to detect the difficulty of program comprehension tasks and programmer level of expertise. By conducting a study with 38 expert and novice programmers, we investigated how well an electroencephalography and an eye-tracker can be utilized in predicting programmer expertise (novice/expert) and task difficulty (easy/difficult). Using data from both sensors, we could predict task difficulty and programmer level of expertise with 64.9 and 97.7% precision and 68.6 and 96.4% recall, respectively. The result shows it is possible to predict the perceived difficulty of a task and expertise level for developers using psycho-physiological sensors data. In addition, we found that while using single biometric sensor shows good results, the composition of both sensors lead to the best overall performance.