Unit 3

____ Takes a piece of data, processes it, and finds a partial result
____ Combines results together, sends to the reducer
____ Splits up data, sends to mappers, gets results back
____ Takes a set of partial results and combines them together
____ Gets the final result, outputs it
   

Solution:

Mapper -> Takes a piece of data, processes it, and finds a partial result
Manager -> Combines results together, sends to the reducer
Manager -> Splits up data, sends to mappers, get results back
Reducer -> Takes a set of partial results and combines them together
Manager -> Gets the final result, outputs it

                            

    A. To analyze the destination of incoming data and choose the best available path to forward it.
    B. To manage user accounts and control who is allowed to access the internet.
    C. To break data into smaller pieces so it can travel more efficiently across the internet.
    D. To translate data into a format that different devices can understand.
    E. To convert domain names into IP addresses.    

Solution: A

    A. DNS receives packets from the browser, assigns them a URL, and forwards them to the closest router for display.
    B. A browser takes a URL, uses DNS to find the website’s IP address, breaks the request into packets, and sends them across the internet to the server.
    C. An IP address asks the browser to create packets, which DNS then sends back as a URL for the website.
    D. A browser sends a URL directly to the website, where DNS converts the packets into an IP address before displaying the page as multiple packets.    
  

Solution: B

    Terms

    Browser
    Protocol
    Distributed computing
    IP address
    Router
    DNS server
    Cloud computing
    Packet
    ISP

    Descriptions

    A. A set of rules that devices follow to communicate over a network
    B. A company that provides users with access to the internet
    C. A small piece of data sent across a network
    D. A device that forwards data toward its destination
    E. Application that receives data from the internet and organizes it into a readable webpage
    F. A system that translates domain names into numerical addresses
    G. Computing that uses many computers working together
    H. A series of numbers that uniquely identifies the computer hosting a website, can be static or dynamic
    I. Computing resources provided over the internet on demand
    

  

Solution:

• Browser → E
• Protocol → A
• Distributed computing → G
• IP address → H
• Router → D
• DNS server → F
• Cloud computing → I
• Packet → C
• ISP → B

    a. Running all programs on your personal computer
    b. Storing data using computers accessed over the internet
    c. Sending packets directly between routers
    d. Translating URLs into IP addresses
    e. Set of computer that are connected to the internet and can scale at need
    f. Managing local hardware devices, like printers and hard drives
    g. Controlling network traffic by selecting the appropriate path
    
  

Solution:
b
e

    A. T or F: A router’s main job is to display web pages to users.
    B. T or F: Packets may take different paths across the internet to reach the same destination.
    C. T or F: An IP address works like a mailing address for a device on the internet.
    D. T or F: Protocols describe what data is sent, not how it is communicated.
    E. T or F: A router’s main job is to control the content of websites that users visit.
    
    
  

Solution:
A. F
B. T
C. T
D. F
E. F

The browser takes a URL and sends it to a DNS server to translate it into an IP address. The browser then uses the IP addess to request the website.

On the other hand, an ISP (Internet Service Provider) connects a user's computer to the core of the internet. Without an ISP, the browser cannot send the URL to the DNS server or the request to an IP address.

Data sent over the internet is divided into small pieces called packets so it can travel efficiently across networks.

Possible Solutions (answers may vary):

1. User types the URL into the browser.
2. The browser sends a request to the DNS server to get the IP address for the given URL. The rquest travels through the ISP and various routers to get to the DNS server.
3. The DNS server translates the URL to an IP address and sends it back to the browser. The IP address travels throug routers, the ISP and reaches the browser.
4. Having received the IP address, the browser sends a request to the specific page. The request is sent via packets.
5. The packet travels via the routers to reach the destination.
6. When the website receives the request, it sends back a response, i.e., a webpage, solit into multiple packets. Packets again travel through the routers to reach your browser.
7. When all the packets get to your computer, the computer assembles them to produce the website.

        A. It speeds up your internet connection
        B. It prevents lag by preloading early content while the rest loads
        C. It compresses packets to make them smaller
        D. It stores your passwords for faster login
    

Solution: C

One Possible Solution:

Mapper Input	One slice of pizza
Mapper Output	Total count of pepperoni pieces on that slice
Reducer Input	A list with the counts of pepperoni pieces
Reducer Output	A final count of pepperoni pieces
Manager Actions	Step 1: Splits the pizza to slices and sends one slice to each mapper Step 2: Gets the pepperoni counts from each mapper Step 3: Compiles the pepperoni counts in a list of counts and sends to reducer Step 4: Gets final pepperoni count from the reducer & displays it

Note: This is one possible answer for the MapReduce Algorithm. Ex: You could have also chosen to give each mapper two slices instead of one (or half a slice). As long as your answer aligns with mapper, reducer, and manager definitions and each answer is feasible (in the both data type proposed and conceptual backing) then it should be correct.

Correct Answers:

• Mapper Input → a (One file)

a) One file

Correct Option: This option correctly gives a split of data feasible from the original data given as an input that the mapper can operate on.

Note: Recall the mapper takes partitions of the original data to do an analysis or computation in. All of these partitions of the data are sent to multiple mappers, which makes the process of answering the problem statement more efficient than if one mapper processes all of the files. Furthermore, this is not the only answer to a mapper input, as long as your answer fulfills these criteria it would also work.


b) Text

Incorrect Option: “Text” is too vague of a description that a) doesn’t explain a split of the initial collection of files b) doesn’t give a data type the input will be in

Note: Avoid giving vague descriptions or blanket terms as mapper inputs, you want to focus on giving a specific partition of the data based on the problem statement that is intuitive and feasible.


c) A list containing the words of all the text files

Incorrect Option: The option is incorrect as it states that the source is all text files. Thus, this answer choice is not describing a split of the original data given in the problem which is the main purpose of using mappers in MapReduce algorithm.

Note: Mapper inputs should be appropriate partition of the data based on a splitting criterion (if it be words, files, etc) instead of passing the whole data input.


d) Something that has the words of one individual file

Incorrect Option: The option is incorrect as it gives a vague description of the data type of its input, with the word “Something”. The data type that the words of the file will be stored in needs to be specified.

Note: When designing mapper inputs, make sure that you are specifying the data type of your input (file, list, string) and that the data type is feasible and aligned to the input you are proposing.



---





• Mapper Output → d (An int count of all the instances of “Python” in the file )

a) Output 1 for every “Python” word seen in the input

Incorrect Option: The option is incorrect as instead of defining a result as per mapper output definition, this option is specifying a rule that the mapper should follow, which makes it difficult to later align with future reducer input (that takes in complete results of mappers rather than independent events).

Note: In specifying a mapper result, avoid breaking down the mapper input further. You want to specify a result from the complete mapper input rather than parts of it. For example, if the mapper input was a file, you want to specify a result for that file rather than the individual words in the file.


b) A list of all the words in the file

Incorrect Option: This option is incorrect as it doesn’t give a result that is relevant to the problem. We want to count the number of instances that word “Python” appears in the collection of files, therefore, by simply returning a list of all words, we cannot get a partial count or any output that is of use to the problem at hand.

Note: Recall that the mapper is doing the main task in the problem and giving the answer to partitions of data, while the reducer combines partial results from these mappers to a final output. Therefore, make sure the mapper output is reflective of actually solving the problem for a part of the data.


c) “Python”

Incorrect Option: This option is incorrect because it simply returns “Python” which is our word of interest but not what the problem statement is asking for - we want to return the number of times “Python” occurs in the collection of files.

Note: Make sure to align your mapper and reducer outputs to the problem statement’s end goal.


d) An int count of all the instances of “Python” in the file

Correct Option: This option is correct. It specifies an mapper output that outlines a a concrete result for a partition of data (which in this case a file), an appropriate data type to match the output description (int count), and an aligned answer to our original problem, which is to count the number of “Python” instances in all the files.

Note: This may not be the only correct mapper output, but make sure to have these main components when designing your mapper outputs.



---





• Reducer Input → c (List of counts of the word “Python” from each mapper input )

a) Number of words that equal “Python”

Incorrect Option: This option is incorrect as it does not specify a collection of partial results. Is the number of words a result from one mapper, multiple mappers, etc?

Note: Recall that the reducer input is the collection of partial results outputted by mappers. Therefore instead of describing an individual result as the reducer input, you should be specifying a data type that stores many individual results received from the manager and by extension each mapper.


b) The Mapper Results

Incorrect Option: This option is incorrect as it simply states “Mapper Results” instead of specifying a) what these results are in relation to the problem b) describing a data type to store these mapper results.

Note: Avoid using extremely broad or definitional terms when describing the inputs or outputs in the MapReduce algorithm. You want to give a specific reducer input that aligns with the definition and more importantly the problem at hand.


c) List of counts of the word “Python” from each mapper input

Correct Option: This option is correct as it describes a) a collection of results: the individual counts of “Python”, b) a valid data type to contain them: the list, and c) alignment with the problem: which is counting the instances of “Python”.

Note: This may not be the only correct reducer input, but make sure to have these main components when designing your reducer inputs.


d) Count of “Python” overall in the files

Incorrect Option: This option is incorrect as it gives a vague description of the input - where is this count originating from and does it represent the collection of multiple partitioned inputs? Due to this, it will be difficult to specify a reducer output later on.

Note: Make sure to avoid giving vague or incomplete answers that can be misinterpreted when specifying the reducer input, instead try to apply all aspects of reducer input definition to the problem specifics.



---





• Reducer Output → a (The total count of “Python” words )

a) The total count of “Python” words

Correct Option: This option is correct as it defines a concrete combined/merged result whose data type and content aligns with the problem statement that asks for a count of “Python” words.

Note: Make sure to align your reducer output’s data type and content to both the problem statement and your reducer input. Furthermore, be specific in your answer.


b) A list of counts of the word “Python” in files

Incorrect Option: This option is incorrect as it does not describe a combined or merged result that the reducer requires, but instead a collection (or list) of mini results (which is more indicative of the reducer input instead of output).

Note: Recall that the main task of a reducer is to take a collection of partial results and combine them together to form a final result to output in relation to the problem statement. Make sure that your output format follows this aspect of the definition and describes a concrete merged result.


c) A final result from partial inputs

Incorrect Option: This option is incorrect as it simply states the definition of the reducer output rather than applying the definition to the problem statement. For example, what are the “final results” described in this option?

Note: Avoid using extremely broad or definitional terms when describing the inputs or outputs in the MapReduce algorithm. You want to give a specific reducer output that aligns with the definition and more importantly the problem at hand.


d) A count

Incorrect Option: This option is incorrect as it doesn’t specify what the “count” refers to and thus is a vague answer in the algorithm design.

Note: Make sure to avoid giving vague or incomplete answers in specifying the reducer output, and instead apply the problem at hand and your answers to previous steps (mapper input/output, reducer input) to guide your formation of the reducer output content.



---





• Manager Actions → c (Splits up files to send to each mapper, receives “Python” count results from mappers, formats the counts to a list of counts to send to the reducer, receives final count from reducer, and then displays it )

a) Splits up files to send to each mapper, receives results from mappers, formats the results to a list to send to the reducer

Incorrect Option: This option is incorrect for two main reasons. Firstly, it misses a step in the manager action - what happens after we send the input to the reducer? Secondly, it is not specific in later steps, for example what are the “results” that it receives from the mapper and sends to the reducer? Importantly, the correct manager steps and specificity are two aspects of the manager that are required in the description.

Note: When designing the manager actions, reflect on the task sequence between the manager, reducer, and mapper. Recall that the manager facilitates communication between the reducer and mapper and also displays the final result to the user. In tracing this pipeline, if it feels like one of these relationships are not clearly explained or it doesn’t make sense (like how did the reducer receive the output of the mapper?), revisit the manager's actions you’ve specified to see if one is missing.


b) Splits up files to send to each mapper, receives count results from mappers, sends result to the reducer and prints out reducer output

Incorrect Option: This option is incorrect as it misses describing how the numerous mapper inputs are combined to send to the reducer. It solely states, “sends results to the reducer” without this key clarification detail. Is it through a list? A dictionary?

Note: Recall that the manager is the main communicator between the reducer and the mapper, formatting and manipulating the input and output so that both of those components can do their jobs. For example, for this option, we know each mapper outputs a single result for its partition of data but the reducer requires a collection of partial results. Thus, the manager is needed to make these individual results into a collection to send to the reducer, a fact that needs to be specified in the manager answer. Make sure to account for these details in your manager description.


c) Splits up files to send to each mapper, receives “Python” count results from mappers, formats the counts to a list of counts to send to the reducer, receives final count from reducer, and then displays it

Correct Option: This option is correct as it correctly lists all of the managers main functions, gives good specification about what the manager sent/received from the mapper and reducer, and applies the problem of Python count to the description.

Note: In order to best design the manager action efficiently, take the core functions of the manager (listed in lecture) and adapt it both to the answers you put for mapper & reducer inputs and outputs and to the problem.


d) Splits up data to send to each mapper, receives results from mappers, compiles & sends the partial results to the reducer, receives the final result from the reducer and displays it

Incorrect Option: This option is incorrect because it simply states the definition of the core manager functions without applying either the problem statement or relevant mapper and reducer inputs & outputs.

Note: Avoid using extremely broad or definitional terms when describing the actions in the MapReduce algorithm. You want to give a specific manager's actions that aligns with this definition and more importantly the problem at hand.