go to the list of abstracts

go to the list of previous talks
previous talks

go to the list of other SCS seminars
scs seminars

go to the SCS home page

go to the CMU home page


The Next Talk Fa'15 Talks General Info Speaking Req't

STRADS: Parallelizing ML over Model Parameters

Monday, November 30th, 2015 from 12-1 pm in GHC 8102.

Jin Kyu Kim, CSD

Machine learning (ML) methods are used to analyze data which are collected from various sources. As the problem size grows, cluster computing technology has been widely adopted for solving ML problems. There are two driving factors behind this trend: Big-data: computation and storage capacity of a single machine becomes not-enough to process data; Big-model: the number of ML parameters to learn becomes large to the extent that a single machine can not finish learning in a reasonable amount of time. In this talk, we focus on big model problem. A natural solution is to turn to parallelizing parameter updates in a cluster. However, naive parallelization of ML algorithms often hurts the effectiveness of parameter updates due to the dependency structure among model parameters and a subset of model parameters are often bottlenecks to the completion of ML algorithms due to the uneven convergence rate.

In this talk, we will present Scheduled Model Parallel approach for addressing these challenges of parallelizing big model problems efficiently, and a distributed framework called STRADS that facilitates development and deployment of scheduled model-parallel ML applications in distributed systems. I will first talk about SMP scheduling schemes: 1) model parameter dependency checking to avoid updating conflicting parameters concurrently; 2) parameter prioritization to give more update chances to the parameters far from its convergence point. To realize SMP, we implement a prototype SMP framework called STRADS. STRADS improves updates executed per second by pipelining iterations and overlapping update computation with network communication for parameter synchronization. With SMP and STRADS, we improve convergence per update and improved updates per second. As a result, we substantially improves convergence per second and achieve faster ML execution time. For benchmark, we implement various ML algorithms such as MF, LDA, Lasso, Logistic Regression in the form of SMP on top of STRADS.

In partial fulfillment of the CSD Speaking Skills Requirement.

Fall 2015 Schedule
Mon, Aug 31 GHC 6501 Expired
Fri, Sep 4 GHC 6501 Expired
Mon, Sep 7 GHC 6501 Labor Day UNAVAILABLE
Fri, Sep 11 GHC 6501 Yan Gu Sorting with Asymmetric Read and Write Costs
Mon, Sep 14 GHC 6501 Expired
Fri, Sep 18 GHC 4303 Ankush Das Termination of Initialized Linear Loop Programs
Mon, Sep 21 GHC 6501 Jesse Dunietz Annotating and Automatically Tagging Constructions of Causation
Fri, Sep 25 GHC 6501 Expired
Mon, Sep 28 GHC 6501 Expired
Fri, Oct 2 GHC 6501 Samantha Gottlieb Scheduling Black-box Mutational Fuzzing
Mon, Oct 5 GHC 6501 Waleed Ammar Towards a language-universal syntactic parser
Fri, Oct 9 GHC 6501 Thomas Tauber-Marshall ThomasDB: Automatic Incremental Computation for Big Data
Mon, Oct 12 GHC 6501 Expired
Fri, Oct 16 GHC 6501 Vittorio Perera Simple And Complex Natural Language Commands For A Mobile Service Robot
Mon, Oct 19 GHC 7101 Expired
Fri, Oct 23 GHC 6501 Expired
Mon, Oct 26 GHC 6501 Expired
Fri, Oct 30 GHC 6501 Expired
Mon, Nov 2 GHC 6501 Expired
Fri, Nov 6 GHC 6501 Expired
Mon, Nov 9 GHC 6501 Expired
Fri, Nov 13 GHC 6501 Expired
Mon, Nov 16 GHC 6501 Expired
Fri, Nov 20 GHC 6501 Ashiqur Rahman Khudabukhsh Building Effective Query Classifiers: A Case Study in Self-harm Intent Detection
Mon, Nov 23 GHC 6501 Expired
Fri, Nov 27 GHC 6501 Thanksgiving UNAVAILABLE
Mon, Nov 30 GHC 8102 Jin Kyu Kim STRADS: Parallelizing ML over Model Parameters
Fri, Dec 4 GHC 4303 Shiva Kaul New classifiers for noisy data
Mon, Dec 7 GHC 6501 Zhaohan (Daniel) Guo Concurrent PAC RL
Fri, Dec 11 GHC 6501 Georg Schoenherr TBD
Mon, Dec 14 GHC 6501 AVAILABLE

General Info

The Student Seminar Series is an informal research seminar by and for SCS graduate students from noon to 1 pm on Mondays and Fridays. Lunch is provided by the Computer Science Department (personal thanks to Sharon Burks and Debbie Cavlovich!). At each meeting, a different student speaker will give an informal, 40-minute talk about his/her research, followed by questions/suggestions/brainstorming. We try to attract people with a diverse set of interests, and encourage speakers to present at a very general, accessible level.

So why are we doing this and why take part? In the best case scenario, this will lead to some interesting cross-disciplinary work among people in different fields and people may get some new ideas about their research. In the worst case scenario, a few people will practice their public speaking and the rest get together for a free lunch.

Guideline & Speaking Requirement Need-to-Know

Note: Step #1 below are applicable to all SSS speakers. You can schedule AT MOST THREE talks per semester.

SSS is an ideal forum for SCS students to give presentations that count toward fulfilling their speaking requirements. The specifics, though, vary with each department. For instance, students in CSD will need to be familiar with the notes in Section 8 of the Ph.D. document and follow the instructions outlined on the Speakers Club homepage. Roughly speaking, these are the steps:

  1. Schedule a talk with SSS by sending your talk title, abstract, additional info (like "Joint work with..." or "In Partial Fulfillment of the Speaking Requirement"), and a picture of yourself (preferably jpeg) to sss@cs at least TWO WEEKS before your scheduled talk.
  2. After you are confirmed with your SSS slot, go to the Speakers Club Calendar and schedule your talk at least THREE WEEKS in advance of the talk date.
  3. On the day of your talk, make sure you print Speakers Club evaluation forms for your evaluators to use.
Students outside of CSD will need to check with their respective departments regarding the procedure. As another example, ISRI students fulfill their speaking requirements by attending a semesterly Software Research Seminar and giving X number of presentations per school year. If you have experience with your department that might help others in your department, please feel free to contribute your knowledge by emailing us. Thank you!

SSS Coordinators

Armaghan Naik, Computational Biology
Lin Xiao, CSD


Web contact: sss+www@cs