\documentclass{article}

\usepackage{amsfonts, amsmath, amsthm, bm, amssymb}
\usepackage{listings}
\usepackage{graphicx}
\usepackage{geometry}
\geometry{
 a4paper,
 total={210mm,297mm},
 left=20mm,
 right=20mm,
 top=20mm,
 bottom=20mm,
 }
 
\title{Instructions on How to Run Your Job on AWS EMR}
\author{Chenran Li}

\newcommand{\bs}{\boldsymbol}
 
\begin{document}
\pagenumbering{gobble}
\maketitle

\section{Have Your Account Ready}
Once you have finished the registration, you should first redeem the promo code. To redeem your promo code, go to the \textbf{AWS Management Console}, click your account name at the upper right corner of the screen, and then click \textbf{Billing \& Cost Management}. At the new page, click \textbf{Credits} at the left side of the page, then you can input your promo code. \$50 is more than enough for you to complete this assignment.

\section{Create Your S3 Bucket}
Go back to the \textbf{AWS Management Console}, and click \textbf{S3} to go to \textbf{S3 Management Console}. Click \textbf{Create Bucket}, input whatever you like as a bucket name, and select \textbf{US Standard} as the region. For example, I chose \textbf{605demo} as the bucket name.

\begin{figure}[h]
	\center{\includegraphics[width=110ex]{pic/pic1.png}}
	\caption{Create S3 Bucket}
\end{figure}

Then you can create a folder in the \textbf{605demo} bucket. For example, I created a folder named \textbf{hw1b}, which is empty.

\begin{figure}[h]
	\center{\includegraphics[width=70ex]{pic/pic2.png}}
	\caption{Create Folder}
\end{figure}

\newpage
\section{Upload Your JAR File}
If you have debugged your code on OpenCloud cluster, you might have created the JAR file. You can just upload the JAR file you created on OpenCloud cluster. For example, I uploaded the JAR file \textbf{NB\_train.jar} to the folder I just created.

\begin{figure}[h]
	\center{\includegraphics[width=70ex]{pic/pic3.png}}
	\caption{Upload the JAR File}
\end{figure}

\section{Launch a EMR Cluster}
Go back to the \textbf{AWS Management Console}, and click \textbf{EMR} to go to \textbf{AWS EMR Management Console}. Click \textbf{Create cluster} and then click \textbf{Go to advanced options} on the top of the screen.

\subsection{Step 1: Software and Steps}
For this step, choose \textbf{Amazon} as the \textbf{Vendor}. For the release version, you can choose any version you like. Here I chose \textbf{emr-5.0.0}. Since we only need Hadoop for this assignment, we can only check \textbf{Hadoop 2.7.2} and uncheck all other boxes. Leave the other sections in this page as default, and click \textbf{Next}.

\begin{figure}[h]
	\center{\includegraphics[width=100ex]{pic/pic4.png}}
	\caption{Step 1: Software and Steps}
\end{figure}

\newpage
\subsection{Step 2: Hardware}

For this step, you can use the default \textbf{Network}. For \textbf{EC2 instance type}, you can choose whatever you like, but the recommended choice is \textbf{m3.xlarge}. For the \textbf{Instance count}, the recommended choice is 2 Core instances. If you don't want to use spot instances, you can choose a \textbf{EC2 Subnet} at random. If you do want to use spot instances, you can check the \textbf{Request spot} checkbox and input the \textbf{Bid price}. Here my \textbf{Bid price} was 0.5 (Refer to \verb|https://aws.amazon.com/ec2/spot/| to know more about spot instances). Click \textbf{Next}.

\begin{figure}[h]
	\center{\includegraphics[width=120ex]{pic/pic5.png}}
	\caption{Step 2: Hardware}
\end{figure}

\subsection{Step 3: General Cluster Settings}

For this step, first input whatever you like as the \textbf{Cluster name}, and choose whatever \textbf{S3 folder} you like as the logging folder. For example, I chose \textbf{Hadoop NB} as the \textbf{Cluster name} and \textbf{s3://605demo/} as the folder. Leave other sections as they are and click \textbf{Next}.

\begin{figure}[h]
	\center{\includegraphics[width=100ex]{pic/pic6.png}}
	\caption{Step 3: General Cluster Settings}
\end{figure}

\subsection{Step 4: Security}

For this step, you can leave all sections as they are and click \textbf{Create cluster}. Now you have to wait for a few minutes until the cluster is successfully launched.

\section{Run Your Job}
When the cluster is ready, click \textbf{Add step} on the top of the screen. Choose \textbf{Custom JAR} as the \textbf{Step type}, and input a name as you like. For \textbf{JAR location}, choose the JAR file you just uploaded.\\

\noindent
For the \textbf{Arguments} section, remember that the shell command of running the Hadoop job is

\verb|hadoop jar NB_train.jar input/path output/path num_reducers|\\
\noindent
Therefore, we need to input \verb|input/path output/path num_reducers| as the \textbf{Arguments} here. The three arguments are separated by a single space character. Note that the output path should not exist, otherwise the job will fail.

\begin{figure}[h]
	\center{\includegraphics[width=100ex]{pic/pic7.png}}
	\caption{Add Step}
\end{figure}

\section{View Your Steps}
After your step is added, you can go to the \textbf{Steps} section to view your step. Click \textbf{View logs} to download the log files. Usually it will take you less than 10 minutes to finish the job in this assignment.

\begin{figure}[h]
	\center{\includegraphics[width=120ex]{pic/pic8.png}}
	\caption{View Your Steps}
\end{figure}

\section{Terminate Your Cluster}
Don't forget to terminate your cluster after your work is done.









\end{document}