Advanced Parallel Programming

Uitgebreide vaknaam	Advanced Parallel Programming
Leerdoelen	After completing this course, you will become proficient in: o programming GPU and Reconfigurable Accelerators to achieve magnitude faster application level performance; o reasoning about and understanding the spatial complexity of algorithms, arithmetic optimisations and costs of data movement; o designing precise performance models driven by the targeted algorithm, dataset as projected on the system architecture; o implementing and debugging high-performance application specific parallel implementations; o devising transformations in order to reach the maximal application performance on a given system instance; o applying computational and bandwidth trade-offs to improve system level performance; o achieving maximal performance within constrained space and power datacentre environments. will understand: o the necessary details of the underlying computational substrates, their architectures and microarchitectures; o the concepts of data parallel and spatial execution models and how to apply those for performance scalability; o parallel memory systems and how to manage locality, reduce bandwidth consumption and facilitate parallel algorithm patterns; o the advantages and shortcomings of emerging accelerator technologies and product lines in order to make the right choices; o latency, throughput and cost dimensions of computing; o the conceptual differences between traditional processor-based computing and heterogeneous application specific acceleration. will know how to address design implications and optimizations: o at application and algorithmic levels; o to systematically resolve system-level bottlenecks; o by customizing data placement and movements; o by overlapping computation with communication; o by applying arithmetic level optimizations.
Omschrijving	The aim of this new course is to introduce you to the fundamentals of programming massively parallel heterogeneous accelerators and in particular to advanced techniques towards system design and programming of emerging computing systems, e.g., GPUs and FPGAs. The course will consist of lectures with periods of discussion as well as practical programming exercises and coursework to further understanding of the material. The course will cover advanced material and start with practical programming examples early on. Introductory material will be minimized. The lab exercises will let the students gain experience with programming massively parallel accelerators available from Nvidia (GPUs), Xilinx (Alveo), Intel (Xeon CPU+FPGA), Microsoft’s Azure and Amazon EC2 cloud instances with FPGAs, and other emerging industry platforms.
Uren per week
Onderwijsvorm	Hoorcollege (LC), Practisch werk (PRC)
Toetsvorm	Practisch werk (PR), Schriftelijk tentamen (WE) (The written examination (open book) contributes 85% to the overall mark while the individual project contributes 15%. Unassessed practice test for formative assessment.)
Vaksoort	master
Coördinator	prof. dr. ir. G. Gaydadjiev
Docent(en)	prof. dr. ir. G. Gaydadjiev
Verplichte literatuur	Titel Auteur ISBN Prijs “Validity of the single processor approach to achieving large scale computing capabilities”, AFIPS spring joint computer conference, IBM Sunnyvale, California, 1967 (not mandatory) Gene M Amdahl “Some Computer Organizations and Their Effectiveness”, IEEE Transactions on Computers, C-21(9): pp 948-960, Sept 1972 (not mandatory) Michael J Flynn “Patterns for Parallel Programming,” Addison Wesley, 2005 (not mandatory) T. Mattson, et al “Decoupled access/execute computer architectures”, Computer Systems, ACM Transactions on; Volume 2, Issue 4, pp 289-308, November 1984 (not mandatory) J. E. Smith “Computer Architecture: Pipelined and Parallel Processor Design”, (Chapters 1-7), May 1995 (not mandatory) Michael J Flynn “Programming Massively Parallel Processors”, 2nd Edition, O’Reilly, 2012 (not mandatory) Wen-mei W. Hwu, David B. Kirk NVIDIA, NVidia CUDA Programming Guide, NVidia, 2007 (not mandatory) “Readings in Computer Architecture”, Morgan Kaufmann, 1999 (not mandatory) Mark D. Hill, Norman P. Jouppi, Gurindar S. Sohi
Entreevoorwaarden	C, Java, Computer Architecture (WBCS14002)
Opmerkingen	This course was registered last year with course code WMCS19001
Opgenomen in	Opleiding Jaar Periode Type MSc Computing Science: Data Science and Systems Complexity  (Guided choice course units) - semester II b keuze MSc Computing Science: Intelligent Systems and Visual Computing  (Guided choice course units) - semester II b keuze MSc Computing Science: Software Engineering and Distributed Systems  (Guided choice course units) - semester II b keuze MSc Courses for Exchange Students: AI - Computing Science - Mathematics - semester II b