Performance clarity as a first-class design principle.
Title | Performance clarity as a first-class design principle. |
Publication Type | Conference Paper |
Year of Publication | 2017 |
Authors | Ousterhout, K., Canel C., Wolffe M., Ratnasamy S., & Shenker S. J. |
Published in | Proceedings of the 16th Workshop on Hot Topics in Operating Systems HotOS'17 |
Abstract | Users often struggle to reason about the performance of today's systems. Without an understanding of what factors are most important to performance, users do not know how to tune their system's hardware and software configuration to improve performance. We argue that performance clarity -- making it easy to understand where bottlenecks lie and the performance implications of various system changes -- should be a first class design goal. To illustrate that this is possible, we propose an architecture for data analytics frameworks in which jobs are decomposed into schedulable units called monotasks that each consume a single resource. By untangling the use of different resources, using monotasks allows the system to trivially report time used on each resource and the resource bottleneck. Our prototype implementation of monotasks for Apache Spark is API-compatible and achieves performance parity with Spark, and yields a simple performance model that can predict the effects of future hardware and software changes. |
Acknowledgment | We thank Radhika Mittal, John Ousterhout, Aurojit Panda, and Patrick Wendell for helpful comments on earlier drafts of this paper. We are appreciative of Shivaram Venkataraman for discussions during the tiny tasks project [8] that led to the idea of breaking jobs into small, single-resource units of work. This research was supported in part by a Hertz Foundation Fellowship, a Google PhD Fellowship, and Intel and other sponsors of UC Berkeley’s NetSys Lab. |
URL | https://dl.acm.org/ft_gateway.cfm?id=3102981&ftid=1895463&dwn=1&CFID=166401991&CFTOKEN=27df600ecf0aaf52-62C0ACA5-E29B-257F-B9726ADC4B43E082 |
ICSI Research Group | Networking and Security |