DiSCo: Distributed Scalable Compilation Tool for Heavy Compilation Workload

Abstract

The size and complexity of software in computer systems and even in consumer electronics is drastically and continuously increasing, thus increasing the compilation time. For example, the compilation time for building some of mobile phones' platform software takes several hours. In order to reduce the compilation time, this paper proposes a Distributed Scalable Compilation Tool, called DiSCo where full compilation passes such as preprocessing, compilation, and even linking are performed at remote machines, i.e. in parallel. To the best of our knowledge DiSCo is the first distributed compiler to support complete distributed processing in all the compilation passes. We use an extensive dependency analysis in parsing compilation commands for exploiting higher command-level parallelism, and we apply a file caching method and a network-drive protocol for reducing the remote compilation overhead and simplifying the implementation. Lastly, we minimize load imbalance and remote machine management overhead with our heuristic static scheduling method by predicting compilation time and considering the overheads invoked by the compilation process. Our evaluation using four large mobile applications and eight GNU applications shows that the performance of DiSCo is scalable and the performance is close to a profile scheduling.