Project 2 Grading

Your handin directory should now contain two files: grade.txt and report.txt. The former file contains your grade for project 2, the second file lists how your code performed for each of the tests we performed on it. The tests are described in more detail below. You need to execute 'cklog cs.cmu.edu' to access your handin directory.

Your grade for project 2 is computed based on four components, each of them having a weight of 25%. The four components are

Correct handin,
IP layer,
NAT/firewalling,
DHCP.

In case you would like a component to be re-graded, you must take a closer look at the tests run for the evaluation of this component first. You should re-run the tests. If you notice that your code passes tests that have been reported to fail, please send email to uhengart+@cs.cmu.edu. In case your tests confirm the reported failures and you still think that you deserve a better grade, you should investigate the failures in more detail. We expect you to have a very clear picture of why your code fails, what needs to be fixed, and why you think you deserve a better grade. Only if you can answer all of these questions, send email to uhengart+@cs.cmu.edu. As a rule of thumb, each line of code that needs to be modified in your code to make it pass a test results in a penalty of 10 points from the overall grade.

Requests for re-grading must be received by Nov, 13.

We now describe the evaluation of the four components in more detail.

1. Correct Handin

This component consists of two parts (A1 and A2), each of them having a weight of 50%.

A1 - Write-up

Does your write-up include all the sections listed in the project handout? Each missing section results in a penalty of 20%.

A2 - Handin

Did you hand in your project in the way required by the project handout and by the project Web page. Each mistake results in a penalty of 50%.

2. IP Layer

This component consists of three parts (functionality, robustness, and code quality), the first one having a weight of 60% and the second and third one each having a weight of 20%.

The files used for testing this component (and all the other ones) are in /afs/cs/academic/class/15441-f01/projects/project2/grading. In the rest of this document, we use $grading_dir when referring to this directory. In all experiments, node 1 is running your kernel. All the other nodes in the network are running the sample kernel provided in $grading_dir. This configuration also holds for the NAT and firewalling tests, but not for the DHCP tests, where all nodes are running your kernel.

Functionality

This component consists of four parts (B1a, B1b, B1c, and B1d), each having a weight of 25%.

B1a - Send

A UDP packet is sent from your kernel to our kernel. Each error results in a penalty of 25%. Some possible errors are

Computing the checksum over the entire packet or only parts of the IP header.
Setting the IP header length field to a wrong value (i.e., other than 5).

Your code is evaluated by running an experiment and visually inspecting it. The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel.pl $grading_dir/network2.cfg 
rttest -n 1 add 0.0.0.0 1
rttest -n 2 add 0.0.0.0 1
udprcv -n 2 50 &
udpsend -n 1 1.1.2.1 50

The source code for the udprcv and udpsend applications is available in $grading_dir.

B1b - Receive

A UDP packet is sent from our kernel to your kernel. Each error results in a penalty of 25%. Some possible errors are

no checksum check,
no ICMP message for "port unreachable" errors,
ICMP messages for errors other than "port unreachable".

Your code is evaluated by running an experiment and visually inspecting it. The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel.pl $grading_dir/network2.cfg 
rttest -n 1 add 0.0.0.0 1
rttest -n 2 add 0.0.0.0 1
udprcv -n 1 50 &
udpsend -n 2 1.1.1.1 50

B1c - Forwarding

A UDP packet is sent/received by our kernel and forwarded by your kernel. Each error results in a penalty of 25%. Some possible errors are

TTL not decremented,
no ICMP message upon TTL == 0,
checksum not recomputed,
accepting packet not for it.

Your code is evaluated by running an experiment and visually inspecting it. The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel.pl $grading_dir/network3.cfg 
rttest -n 1 add 1.1.1.2 1
rttest -n 1 add 1.1.2.2 2
rttest -n 2 add 0.0.0.0 1
rttest -n 3 add 0.0.0.0 1
udprcv -n 2 50 &
udpsend -n 3 1.1.1.2 50

B1d - Broadcasting

Your kernel is tested for its broadcasting behavior. Each error results in a penalty of 25%. Some possible errors are

Bcast packet not sent over all interfaces.
Bcast packet not copied when sending over multiple interfaces.
Bcast packet not accepted.
Bcast packet gets forwarded.
Bcast packet results in ICMP message upon "port unreachable".

Your code is evaluated by running an experiment and visually inspecting it. The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel.pl $grading_dir/network3.cfg 
udprcv -n 2 50 &
udprcv -n 3 50 
udpsend -n 1 255.255.255.255 50

B2 - Robustness

Your source code is visually inspected for robustness issues. Each undealt issue results in a penalty of 25%.

No version, ihl, length, and/or protocol check at receiver.
No host/network order byte conversions for packet length.
No maximum packet length check for outgoing interface (12% if maximum length is hardcoded).
No error handling for failing routing lookups.

Code Quality

This component consists of two parts (B3a and B3b), each having a weight of 50%.

B3a - Comments

Is your code commented?

B3b - Readability

You get zero points here if you access header fields in ptr+number fashion instead of using an appropriate data type or (at least) replacing the number by a #define'd constant.

3. NAT/Firewalling

This component consists of three parts (Functionality, parsing, and code quality), the first one having a weight of 50%, the second one of 30%, and the third one of 20%.

Functionality

This component consists of two parts (C1a and C1b), each having a weight of 50%.

C1a - NAT

Your code has to do NAT for a forwarded outgoing, forwarded incoming, and locally generated outgoing packet. Each correct case contributes 33% (with an additional 1% if all three cases are correct).

The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel.pl $grading_dir/network3.cfg 
setnat -n 1 $grading_dir/nat_rulesA.txt
rttest -n 1 add 1.1.1.2 1
rttest -n 1 add 1.1.2.2 2
rttest -n 2 add 0.0.0.0 1
rttest -n 3 add 0.0.0.0 1
# outgoing packet
udprcv -n 3 -d 50 &
udpsend -n 2 1.1.2.2 50
# incoming packet
udprcv -n 2 50 &
udpsend -n 3 5.6.7.8 50

$grading_dir/start_kernel.pl $grading_dir/network2.cfg 
setnat -n 1 $grading_dir/nat_rulesB.txt
rttest -n 1 add 0.0.0.0 1
rttest -n 2 add 0.0.0.0 1
# locally generated packet
udprcv -n 2 -d 50
udpsend -n 1 1.1.2.1 50

C1b - Firewalling

First, three tests examine whether your kernel is able to handle simple rules and whether input, forward, and output filtering are placed correctly in your code. Second, your kernel is evaluated with two more complicated, forwarding rules. Your code passes a test when a packet to be filtered gets filtered and a packet not to be filtered gets delivered.

The first experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel.pl $grading_dir/network3.cfg 
setfilter -n 1 $grading_dir/fw_rulesPlacement.txt
rttest -n 1 add 1.1.1.2 1
rttest -n 1 add 1.1.2.2 2
rttest -n 2 add 0.0.0.0 1
rttest -n 3 add 0.0.0.0 1

# input filter
#  to be dropped
udprcv -n 1 50 &
udpsend -n 2 1.1.1.1 50
#  not to be dropped 
udprcv -n 3 50 &
udpsend -n 2 1.1.2.2 50

# forward filter
#  to be dropped
udprcv -n 3 51 &
udpsend -n 2 1.1.2.2 51
#  not to be dropped
udprcv -n 1 51 &"
udpsend -n 2 1.1.1.1 51

# output filter
#  to be dropped
udprcv -n 3 52 &
udpsend -n 1 1.1.2.2 52
#  not to be dropped
udpsend -n 2 1.1.2.2 52

The second experiment is run in the following way (from the directory containing your kernel):

# first complicated rule
$grading_dir/start_kernel.pl $grading_dir/network3.cfg 
setfilter -n 1 $proj_dir/grading/fw_rulesComplicated.txt
rttest -n 1 add 1.1.1.2 1
rttest -n 1 add 1.1.2.2 2
rttest -n 2 add 0.0.0.0 1
rttest -n 3 add 0.0.0.0 1
# to be dropped
udprcv -n 3 50 &
udpsend -n 2 -t1 1.1.2.2 50
# not to be dropped
udpsend -n 2 -t2 1.1.2.2 50

#second complicated rule
$grading_dir/start_kernel.pl $grading_dir/network3.cfg 
rttest -n 1 add 1.1.1.2 1
rttest -n 1 add 1.1.2.2 2
rttest -n 2 add 0.0.0.0 1
rttest -n 3 add 0.0.0.0 1
# to be dropped (IP id will be zero)
udprcv -n 3 51 &
udpsend -n 2 1.1.2.2 51
# not to be dropped (IP id will be one)
udpsend -n 2 1.1.2.2 51

Note that for the first complicated rule, we use a hack in the ip_output() routine of our kernel that "transforms" a UDP packet into a TCP packet. Also note that the kernel will crash for the "TCP" packet that should not be dropped, which is "fine". Our hack looks as follows:

{ 
   /* 
    * make udp packet appear as tcp packet and set some flags
    * depending on payload 
    */

   struct udphdr* udp_h = ((struct udphdr *) (pkt->p_data + sizeof(struct ip)));
   struct tcphdr* tcp_h = (struct tcphdr*) udp_h;
   char* payload = ((char*)udp_h) + sizeof(struct udphdr);
   if (!strncmp("tcp1string", payload, strlen("tcp1string"))) {
       printf("ip_input: found tcp1\n");
       tcp_h->th_flags = TH_SYN|TH_FIN;
       hdr->ip_p = IPPROTO_TCP;
   }
   else if (!strncmp("tcp2string", payload, strlen("tcp2string"))) {
       printf("ip_input: found tcp2\n");
       tcp_h->th_flags = TH_ACK;
       hdr->ip_p = IPPROTO_TCP;
   }
}

Parsing

This component consists of two parts (C2a and C2b), the first one having a weight of 30% and the second one of 70%.

C2a - NAT parsing

Your NAT parsing code is given an incorrect and a correct rule to parse. Failure to detect an incorrect/correct rule is given a penalty of 50%. We consider parsing of a rule as success when Setsockopt()gets called (i.e., we completely ignore the output and any return codes of setnat). The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel.pl $grading_dir/network2.cfg 
setnat -n 2 $proj_dir/grading/nat_rules1.txt
setnat -n 2 $proj_dir/grading/nat_rules2.txt

C2b - Firewall parsing

Your firewall parsing code is given five rules, where the first two rules are correct and the last three rules are wrong. Failure to detect an incorrect/correct rule is given a penalty of 20%. We consider parsing of a rule as success when Setsockopt()gets called (i.e., we completely ignore the output and any return codes of setfilter).

The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel.pl $grading_dir/network2.cfg 
setfilter -n 2 $proj_dir/grading/fw_rules1.txt
setfilter -n 2 $proj_dir/grading/fw_rules2.txt
setfilter -n 2 $proj_dir/grading/fw_rules3.txt
setfilter -n 2 $proj_dir/grading/fw_rules4.txt
setfilter -n 2 $proj_dir/grading/fw_rules5.txt

Code quality

This component consists of two parts (C3a and C3b), each having a weight of 50%.

C3a - Organisation/Modularity

If you place all your NAT code directly in the IP functions, without introducing proper function calls, or if you keep all the firewalling/NAT functions in ip_io.c, without putting them into separate files, you will get a penalty of 50%. If you even place the firewalling code directly in the IP functions, you will get 0% here.

C3b - Comments

Is your code commented?

4. DHCP

This component consists of two parts (DHCP server and DHCP client), the first one having a weight of 40% and the second one of 60%.

Your DHCP server and client are tested separately by letting them interact with the sample DHP client and server, respectively. We did not require you to have your DHCP implementation work with the sample implementation. However, your implementation has to follow the requirements in the RFC. Since the sample implementation was implemented according to these requirements, your solution should be able to interact with it. If you think that your solution complies with the RFC, though it is not able to interact with the sample implementation, write up the observed problem and why you think the behavior of your implementation is legal and send email to uhengart+@cs.cmu.edu.

Our DHCP implementation used for testing is a functionally slightly extended version of the provided sample implementation. It examines compliance of received DHCP messages with the RFC.

All the nodes in the network are running your kernel.

DHCP server

This component consists of two parts (D1a and D1b), the first one having a weight of 60% and the second one of 40%.

D1a - single client

Our DHCP client requests an address from your server. Each error results in a penalty of 20%. Some possible errors are

Handed out address not or incorrectly added to the routing table.
Handed out address not removed from the routing table.
Answers not broadcasted.
Wrong address offered.
No server Id in offer messages.

The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel2.pl $grading_dir/network2.cfg 
$your_dhcp_server -n 2 --topology=$grading_dir/network2.cfg --config=$grading_dir/dhcp_network2.txt --default_lease=60 --max_lease=120 &
$our_dhcp_client -n 1 --lease_length=60 --release_time=10

D1b - multiple clients

Two clients request an address from your DHCP server. You get a 50% penalty for handing out the same address twice.

The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel2.pl $grading_dir/network3.cfg 
$your_dhcp_server -n 1 --topology=$grading_dir/network3.cfg --config=$grading_dir/dhcp_network3.txt --default_lease=60 --max_lease=120 &
$our_dhcp_client -n 2 --lease_length=60 --release_time=20
$our_dhcp_client -n 3 --lease_length=60 --release_time=20

DHCP client

This component consists of four parts (D2a, D2b, D2c, and D2d), the first one having a weight of 40% and the other ones of 20%.

D2a - Request

Your DHCP client requests an address from our DHCP server. Each error results in a penalty of 20%. Some possible errors are

Gateway not or incorrectly added to the routing table.
Source address of DHCP messags not zero.
Request DHCP message not broadcasted.
ciaddr in Discover message.

The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel2.pl $grading_dir/network2.cfg 
$our_dhcp_server -n 2 --topology=$grading_dir/network2.cfg --config==$grading_dir/dhcp_network2.txt --default_lease=60 --max_lease=120 &
$your_dhcp_client -n 1 --lease_length=60 --release_time=10

D2a - Release

Your DHCP client releases an assigned address. Each error results in a penalty of 20%. Some possible errors are

Client renews unnecessarily
Gateway not removed from routing table
Release message broadcasted.

We use the same experiment as for D2a.

D2c - Renewal

Your client needs to perform a renewal. Each error results in a penalty of 20%. Some possible errors are

Renewal message broadcasted.
No ciaddr in Renewal message.

The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel2.pl $grading_dir/network2.cfg 
$our_dhcp_server -n 2 --topology=$grading_dir/network2.cfg --config==$grading_dir/dhcp_network2.txt --default_lease=60 --max_lease=120 &
$your_dhcp_client -n 1 --lease_length=20 --release_time=25

D2d - Retransmissions

The first Discover message gets dropped so that the client has to retransmit it.

The experiment is run in the following way (from the directory containing your kernel):

$grading_dir/start_kernel2.pl $grading_dir/network2.cfg 
$our_dhcp_server -n 2 --topology=$grading_dir/network2.cfg --config==$grading_dir/dhcp_network2.txt --default_lease=60 --max_lease=120 --loose_first_discover &
$your_dhcp_client -n 1 --lease_length=60 --release_time=10

We use a specially prepared version of our DHCP server and instruct it to drop the first Discover message using the --loose_first_discover option.

Late Penalties

L1 stands for one day late (10% off) and L2 for two days late (20% off).