데이터 주도 접근법을 활용한 소프트웨어 테스트 자동화 : 온라인 쇼핑몰 결제시스템 사례

Abstract

This study examines a data-driven approach for software test automation at an online shopping site. Online shopping sites typically change prices dynamically, offer various discounts or coupons, and provide diverse delivery and payment options such as electronic fund transfer, credit cards, mobile payments (KakaoPay, NaverPay, SyrupPay, ApplePay, SamsungPay, etc.) and so on. As a result, they have to test numerous combinations of possible customer choices continuously and repetitively. The total number of test cases is almost 584 billion. This requires somehow automation of tests in settling payments. However, the record playback approach has difficulties in maintaining automation scripts due to frequent changes and complicated component identification. In contrast, the data-driven approach minimizes changes in scripts and component identification. This study shows that the data-driven approach to test automation is more effective than the traditional record playback method. In 2014 before the test automation, the monthly average defects were 5.6 during the test and 12.5 during operation. In 2015 after the test automation, the monthly average defects were 9.4 during the test and 2.8 during operation. The comparison of live defects and detected errors during the test shows statistically significant differences before and after introducing the test automation using the data-driven approach.