Different Ways to Test Acceptability Judgments

Abstract

The testing of acceptability judgments has recently attracted some attention in the study of syntax. This is because many linguists have come to realize that the extensive measuring the correlations between native speakers' intuitions provides a reliable way to investigate the genuine nature of human language. The growing research interest notwithstanding, the methodology of acceptability judgment experiments has not been completely validated. Since experimental syntax is an empirical science, it is important to develop a strong research methodology for a large-scale and in-depth quantitative analysis. In this context, the research question this paper raises is whether or not different methodologies of acceptability judgment testing yield different conclusions. The present study conducts two experiments that share exactly the same test items but are controlled and analyzed in different ways. Looking into the convergence rate using different methods of experimental control and statistical analysis, the present study provides four findings which can lead to a better methodology for observing native speakers’ intuitions: First, randomization is a mandatory process in acceptability judgment testing. Second, the mean in itself does not provide good statistical power in acceptability judgment testing. Third, sociological variables can affect acceptability judgments. Fourth, the size of sample in acceptability judgment testing should be more than 40 in order to yield a trustworthy conclusion.