## Assignment 4: Segmentation and PreprocessingMethods In Medical Image Analysis (BioE 2630 : 18-791 : 42-735) - Spring 2018

 ITK Segmentation and Preprocessing Assignment by Vikas Revanna Shivaprabhu and John Galeotti, © 2012-2017 Carnegie Mellon University, is licensed under a Creative Commons Attribution 3.0 Unported License. Permissions beyond the scope of this license may be available by sending email to itk ATgaleotti.net. This assignment was made possible in part by NIH NLM contract# HHSN276201000580P.

70 points total

Due Date: Your final submission must be committed to svn by 5 pm EST on Wednesday March 28. Big problems may not show up until the end, so finish early!

E-mail your TA or instructor with questions or problems.

This is your first real programming assignment, and your first use of SVN. Upon completing this assignment, you should feel comfortable using ITK and working with differnt pixel types and dimensionality. This assignment requires that you have ITK or SimpleITK installed and working. This assignment includes:

For further guidance you should consult the ITK software guide and the lecture slides, especially those from the background lectures on programming and ITK basic usage, and the segmentation lectures on Overview and Thresholding, Active Contours, and Level Sets.

### 0. Using SVN & Preparing for this Assignment (5 points, apx. 30-45 minutes)

Usage of SVN was discussed in ITK Background Lecture, starting on slide 13. Note, if installing a command-line svn program such as TortiseSVN, then be sure to have it install the command-line svn tools as well. You will be required to submit your code for all remaining assignments using SVN. You should have received your SVN username and password either in class or via email. All of the instructions given here will be for the command line version of SVN. You may want to use a GUI instead (see the above-mentioned lecture slides), but it is up to you to learn the interface of the GUI of your choice.

Begin by confirming access to your SVN module, by listing the current contents of your module (the following {} denote something for you to replace, and the {} characters should not be typed):

svn list -R --verbose --username {Your_SVN_User_Name} https://biglab.ri.cmu.edu/svn/mimia18/{Your_SVN_User_Name}

Your password should be accepted, and you should not have any connection errors.

(If you are prompted about the server certificate beloning to vialab.sp.ri.cmu.edu insterad of biglab.ri.cmu.edu, then please just permanantly accept the certificate. Alternately, you could just always use the svn server's primary name of vialab.sp.ri.cmu.edu instead of the biglab.ri.cmu.edu shortcut, as follows https://vialab.sp.ri.cmu.edu/svn/mimia18/{Your_SVN_User_Name}.)

Change to the directory you previously created (in HW2) to contain ITK, VTK, etc. (e.g. cd c:\MIMIA), and then checkout your module:

{SVN will probably remember your password for you, but if not then provide it again when requested.}

You should now have a new directory named C:\MIMIA\{Your_SVN_User_Name}. This directory should contain all of your code for the remainder of this class (but none of your binaries unless specifically instructed otherwise).

Now add a directory to your SVN module to contain this assignment. You must first create the directory locally, then tell SVN to add it to your module, and then commit your changes to the SVN server:

cd c:\MIMIA\{Your_SVN_User_Name}
mkdir hwSeg
svn ci hwSeg -m "Setting up module for hwSeg"

This is the general template you must follow to add any new file or directory to your SVN repository. When you change a file that already exists, you only need to run the last line above (the commit line) to update the server's copy of your file. Notice the -m option, followed by a quoted string. SVN requires that every committed change be given a brief description, and this is how to do it. (Warning: If you get strange errors when trying to commit, then make sure you are using the double-quote characters around your commit description.)

Now, to further test and exemplify things, you are going to create a directory for this part of the assignment, add a file containing the one word "Hello", commit the directory + file, change the file by adding the word "World" to the end, and then commit the change:

cd c:\MIMIA\{Your_SVN_User_Name}\hwSeg
mkdir Part0
echo "Hello"> Part0\file.txt
svn commit Part0 -m "Initializing Part 0"
echo "World">> Part0\file.txt
svn commit Part0 -m "Fixed file.txt"

In the above, if you prefer you could also use a text editor, such as Windows Notepad, instead of the echo commands. Also, as you may have noticed, running an svn command on a directory runs the command on all of the files and directories it contains. This can be nice...or not. Use with caution.

#### NEVER build C++ source code inside your svn module. For C++ always do an out-of-source build!

Now, create a new svn directory structure for this assignment, following the pattern below exactly. Assuming you are currently in the directory c:\MIMIA\{Your_SVN_User_Name}\hwSeg, this would look like:

mkdir Part1
mkdir Part2
mkdir Part3
svn ci Part1 Part2 Part3 -m "Setting up module for segmentation hw"

Inside the resulting SegmentationAssignment_Images directory, you will find several images and a text file named ReadMe.txt, which explains the files and also give seed points for initializing/starting segmentations. Be sure you understand everything this file says.

Run the ITK-SNAP program (which you installed in HW2, Part 1). From the initial dialog, choose "Open Image ..." and then choose "Browse...". Browse to the SegmentationAssignment_Images directory, and choose the file ChestCT_VascularContrast.mha. Choose "Next >" and then choose "Finish" (don't worry about the "Loss of Precision" warning). Now on the top menu bar go to "Segmentation" -> "Open Segmentation ...". As you just did, browse to the SegmentationAssignment_Images directory again, but this time choose the file ChestCT_VascularSeg.mha. Once the segmentation is loaded, click the "update" button at the bottom of the main window. A screen-shot of what this should look like after you update SNAP's mesh is included in the file SegHW-SNAP.png. Spend a little time using SNAP to look through the volumetric data, to get a feel for it. (There is also a similar SNAP screenshot for the cropped ascending aorta volume, named SegHW-crop-SNAP.png.)

### 1. Region-Growing Threshold Segmentation (10 points)

Begin by copying c:\MIMIA\SimpleITK\Examples\Python\ConnectedThresholdImageFilter.py from your SimpleITK source directory to Part1\threshold.py. Note that this program is made to be directly executable from the normal command line, by entering something like ipython Part1\threshold.py. When you run the program without any command line arguments, it will tell you what arguments you should use.

 SimpleITK Python vs. SimpleITK C++ vs. "Full" ITK C++ From this point on, we will assume that you want to use SimpleITK in Python. However, if you prefer to use SimpleITK in C++ instead, then rather than grabbing the .py example files, look in the same (or similar) directory for the corresponding .cxx example files and use those instead. If you use C++, you will of course also need to submit an appropriate CMakeLists.txt file as well. Finally, if you're really brave, then feel free to try this in "full" ITK instead of SimpleITK, but keep in mind that you may need content from future lectures (try looking at last year's course website). You should be able to find equivalent "full ITK" example files in ITK's Examples directory.

Reminder: Remember that matched indentation is very important in Python, and you should always uses spaces (not tabs, since they are treated differently).

Test the code using 2-dimensional image AscendingAorta_2D.png with the provided seed point (from ReadMe.txt) and compare your result with AscendingAorta_2DSeg.png.

• You will have to experiment with different Lower and Upper threshold values.
• You can visualize the image and examine the intensity values by loading the image into either ImageJ or ITK-SNAP and reading the Intensity at the cross-hair position. (ITK-Snap should use the .nii images instead of the .png images, and the intensity under the crosshairs will be shown on the left-hand side of the winow.)
• Set the output filename as Result_Part1_2D.png
• As a sanity check, after following the above instructions, you should now be running your program using a command line that looks something like this:
• ipython Part1\threshold.py {your\path\to}\AscendingAorta_2D.png {your\path\to}\Result_Part1_2D.png {your_low_threshold_number} {your_high_threshold_number} 35 34.

Make the following changes to threshold.py, being sure to commit your changes to svn after you complete each step:

• Make it operate on 3-dimensional images. Be sure to make it read in and use the Z-coordinate of the seed point, and change as necessary all references to argv (which should now contain one last mandatory element for the z-coordinate on lines 28-29, and should contain optional arguments in multiples of 3 instead of 2 on lines 56-59). I strongly recommend testing to make sure this works before continuing.
• Test your code using AscendingAortaCrop.mha, with the provided seed point, and compare your results with AscendingAortaCropSeg.mha.
• Once again, experiment with different values for the upper and lower threshold.
• Set the output filename as Result_Part1_3D.mha
• Include a comment at the top of threshold.py indicating your best choice of upper and lower thresholds, as well as how you derived your answers. In the same comment, rate (on a scale of 1=bad to 10=perfect) how well you think this segmentation worked. (Note: Randomly picking a higher number here will not give you a higher grade.)

Big Hint: Be aware that 3D CT images have 12-bit data stored in 16-bit pixels, and so their intensities can be negative or exceed 1000 (unlike standard jpegs or pngs that have a maximum intensity of 255).

Reminder: When working on any part of any assignment for this class, be sure to add any new source files to svn as soon as you create them, and be sure to frequently commit you code as you make changes to it. Above all, don't forget to committ your final code to svn!

### 2. Fast Marching LevelSet (15 points)

Implement a fast marching solution to a level set evolution problem by building the following sequence of filters:
reader -> edge preserving smoothing -> gradientMagnitude -> sigmoid ->  fastMarching -> thresholder -> writer

• You can look at c:\MIMIA\SITKBin\ITK\Examples\Segmentation\FastMarchingFilter.cxx for detailed explanation on how the algorithm works and how to set up the above pipeline.
• Begin by creating a new source file Part2\fastMarchingSeg.py. Do not forget to save your changes to SVN. Use the following filters for different stages of the pipeline
• Sigmoid: Use SigmoidImageFilter
• Documentation can be at http://www.itk.org/SimpleITKDoxygen/html/classitk_1_1simple_1_1SigmoidImageFilter.html
• Important note: Fast Marching requires a speed image that has large values inside the object and (near-) zero values around the boundaries. This is the opposite of what gradient magnitude gives us. You have a couple of easy choices, either (1) use an additional (normalized) recriprical function to invert the output of the sigmoid filter, or else (2) use a negative parameter for the sigmoid to invert the gradient magnitude. If you choose to use a negavive parameter from the command line, then you have to be careful how you type it on the command line, or else python will mistake the negative sign for a different kind of argument; I suggest passing negative arguments inside quote marks with a leading space like this, python fastMarchingSeg.py ... " -0.5" ...
• writer: Set up the writer similar to part1 of the assignment.
• Test the code using the 2-dimensional image with the provided seed point and compare your result with the reference segmentation.
• Set the output filename as Result_Part2_2D.png
• Now, make it operate in 3-dimensions (itis okay to hard-code it for just 3D for this part). Test your code using the cropped volume and compare your results with the reference segmentation.
• Set the output filename as Result_Part2_3D.mha
• Once again, experiment with different parameter values. Include a comment at the top of fastMarchingSeg.py indicating your best choice of parameters, as well as how you derived your answers. In the same comment, rate (on a scale of 1=bad to 10=perfect) how well you think this segmentation worked (don't expect it to be perfect).

### 3. Creative Experimentation on Big Volume (40 points)

You have previously tried some of the SimpleITK iPython tutorial notebooks, and now you should read through (and I suggest experiment a little with) the Level Set tutorial notebooks.

Now, find your best combination of preprocessing and segmentation algorithms for the full-size image volume. In addition to the seedpoint provided, you may optionally choose up to 4 additional seedpoints for the large volume (5 seedpoints total). Carefully distributing your seed points can substantially speed up your segmentation, and potentially increase its accuracy as well. (You are free to use segmentation algorithms other than level sets if you want.)

To find appropriate algorithms for vasculature segmentation, as well as example code to use, I highly recommend using the ITK Software Guide, Google, and/or the ITK mailing list archives (please do not ask the mailing list how to do this--search the list archives instead; if you still have questions, then it is appropriate to ask the list). I also highly recommend starting with the small (64x64x64) volume for initial experimentation, and then only trying things that seem promising on the larger data set.

Scoring for this part is as follows:

• Large volume basics: If you successfully implent and apply a different segmentation method, and your code produces a segmentation of the large volume that looks reasonable, then you qualify for up to 30 points, depending on how good your resulting segmentation is.
• Large volume competition: The remaining 10 points will be subjectively awarded based on 3 things: (1) How well each of you does compared to your peers (this is a competition, after all), (2) How advanced / difficult-to-implement your segmentation method is, and (3) How well you seem to have optimized your parameter choices for your segmentation. It is possible for most students to simultaneously get at least 4 or 5 of these 10 points one way or the other, if sufficient effort is put in.

Since this part is a competition to see who can get the best segmentation, you are on your own. Neither the TA nor myself will help you choose or implement your segmentation algorithm (but feel free to ask your TA for help with "non competitive" issues such as compiler problems, etc.). Feel free to use all other sources of help at your disposal, including internet mailing lists (but please be polite and only ask specific questions on the itk mailing list). Place your well-commented code in Part3\best_segmentation.py (or for C++ use an .cxx extension and include an appropriate CMakeLists.txt file).

Be sure to comment in your code about any areas where you believe your segmentation does a better job than the expert segmentation. (If you're not sure about a piece of vasculature that you believe should be segmented, but was not in the expert segmentation, then feel free to ask your TA or instructor for clarificaiton.)

*** Also, be sure to include a comment at the top of best_segmentation.py indicating how long it takes your code to run on your machine, so that the TA has some idea how long he should wait for your code to finish executing.

Please note that I strongly discourage trying to use really complex segmention methods for this part. On such a large, branching data set they almost certainly will not work. If, contrary to my advice and in the spirit of competition, you feel compelled to try anyway, then I would suggest (1) lots of cafine, and (2) a 2-stage segmentation, where you only use an advanced segmentation method to refine the output of some other simpler and more robust method.