CMU 15-418
Parallel Computer Architecture and Programming
Tues/Thurs 12:00 - 1:20pm (Spring 2012)
GHC 4307
Kayvon Fatahalian
[ Click Here for the Most Recent Offering of the Course ]
Spring 2012 Student Final Project List

From smart phones, to multi-core CPUs and GPUs, to the world's largest supercomputers, parallel processing is ubiquitous in modern computing. The goal of this course is to provide a deep understanding of the fundamental principles and engineering trade-offs involved in designing modern parallel computers as well as the programming techniques to effectively utilize these machines. Because writing good parallel programs requires understanding of key machine performance characteristics, this course will cover both parallel hardware and software design. Course programming assignments will be implemented in a number of modern environments including ISPC, OpenCL, OpenMP, and MPI.

1Jan 17 Why Parallelism? (the 2012 answer)  
2Jan 19 A Modern Multi-Core Processor: Forms of Parallelism + Latency & Bandwidth Issues  
3Jan 24
Reading: Textbook Section 1.2
Assignment 1 out (Tue 1/24)
4Jan 26
Reading: Textbook Chapter 2
5Jan 31
Reading: Textbook Chapter 2, 3.1
Assignment 1 due (Tue 1/31)
6Feb 2  
7Feb 7
Reading: Textbook Sections 1.3, 3.1.2, 3.2-3.3
Assignment 2 out (Tue 2/7)
8Feb 9
Reading: Textbook 3.5
9Feb 14
Reading: Textbook Chapter 4
10Feb 16
Reading: Textbook Sections 5.1 and 5.3
11Feb 21
Reading: Textbook Section 5.4, 6.3
Assignment 2 due (Tue 2/21)
12Feb 23
Reading: Textbook Sections 8.1, 8.2
13Feb 28
Directory-Based Cache Coherence II (+ Begin Memory Consistency)
Reading: Textbook Sections 8.2, 5.2, 9.1
14Mar 1
Reading: Textbook 9.1
 Mar 6Exam 1 (material up to and including Lecture 14)
15Mar 8
MPI Introduction + Class Fireside Chat: Do Grades Matter?
 Mar 12-16Spring Break 
16Mar 20
Reading: Textbook 6.1, 6.2
Assignment 3 due (Sun 3/18)
17Mar 22  
18Mar 27
Interconnection Network Design (by Michael Papamichael)
Reading: Textbook 10.1 - 10.6
Assignment 4 due (Tues 3/27)
19Mar 29
Basic Synchronization (locks and barriers)
Reading: Textbook 5.5, 7.9, 8.8
Project Proposal Due (Mon 4/2)
20Apr 3  
21Apr 5  
22Apr 10Heterogeneous Parallelism and Hardware Specialization 
23Apr 12Domain-Specific Parallel Programming 
24Apr 17Implementing Scan and Segmented Scan (+ for fun: the NT method) 
 Apr 19No class (Carnival: CMU holiday)Project Checkpoint (Mon 4/23)
 Apr 24Exam II Review Session 
 Apr 26Exam II (covering lectures 14-24) 
25May 1Parallel Real-Time Rendering 
26May 3Course Recap and Project Presentation Tips 
 May 10Parallelism Competition 

Assignment 1: Analyzing Program Performance on a Quad-Core CPU

Assignment 2: A Parallel Renderer in CUDA

Assignment 3: OpenMP Programming on PSC's Blacklight Supercomputer

Assignment 4: Wandering Salesman Problem in MPI

Final Project: self-selected project on the parallel platform of your choosing.

Assignment 1 must be performed individually. Assignments 2-4, as well as the final project, may be completed and handed in in pairs. Assignments 1 and 2 require use the of the machines located in Gates 5205 (for remote access, use:, where 51 <= XX <= 81). Assignments 3 and 4 will be conducted via remote access to Blacklight at the Pittsburgh Supercomputing Center.

Late hand-in policy: Each team is allowed a total of three late days for the semester. Late days can be used only on assignments 2-4.


Course Staff


Kayvon Fatahalian
Gates-Hillman Center 7005
Office Hours: drop by anytime, or by appointment
[ kayvonf at ]


Michael Papamichael
Hamerschlag Hall A-312
Office Hours: Wed 12:00-1:00pm
[ papamix at ]

Mike Mu
West Wing Cluster
Office Hours: Mon 4:00-5:00pm
[ mikemu at ]

We're using Piazza for our class discussion board and Q/A system.


David E. Culler and Jaswinder Pal Singh, with Anoop Gupta.
Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann, 1998.


15-213 (Intro to Computer Systems) is a strict prerequisite for this course. We will build directly upon the material presented in 15-213, including memory hierarchies, memory management, basic networking, etc. While 18-447 (Intro to Computer Architecture) would be helpful for understanding the material in this course, it is not a prerequisite. Students are expected to be strong C/C++ programmers as there will be exposure to a variety of "C-like" parallel programming languages in this course.


Special thanks to NVIDIA Corporation for their generous equipment donation to support GPU-computing assignments and projects. Intel has also provided generous financial support. Thanks to Matt Pharr and Warren Hunt at Intel for technical assistance with ISPC. Solomon Boulos is now getting the credit he deserves.