|
Shape
from Shading under Near Point Lighting and Partial views
for Orthopaedic Endoscopy
|
|
|
|
Abstract
Bone reconstruction using endoscopy is important for
computer aided
minimally invasive orthopedic surgery. During surgery an endoscope
consisting of a camera and one or more light sources is
inserted through a small incision into the body and the acquired
images are analyzed. Since bone surface is featureless,
shading is the primary cue for shape perception. However,
due to the small field of view of the endoscope, only a
small part of the bone and its occluding contour are visible
in any single image. Therefore even human perception of bone
shape from such images can be hard.
We present a
novel technique to reconstruct the surface of the bone by applying
shape-from-shading to a sequence of endoscopic
images, with partial boundary in each image. We first perform geometric and the
photometric calibration for the endoscope. We then extend
the classical shape-fromshading
algorithm to
include a near point light source that is not optically co-located with
the camera. By tracking the endoscope we are able to align
partial shapes obtained from different images in the
global (world) coordinates. An ICP algorithm is then used to
improve the matching, resulting in a complete occluding boundary
of the bone. Finally, a complete and consistent shape is
obtained by simultaneously re-growing surface normals and depths in all views.
We demonstrate
the accuracy of our technique using simulations
and experiments
with artificial bones.
|
Publications
“A
Multi-image Shape-from-Shading Framework for Near-Lighting Perspective
Endoscopes”
Chenyu Wu,
Srinivasa G. Narasimhan,
Branislav Jaramaz,
Accepted for publication to International
Journal of Computer Vision (IJCV), Dec 2008
“Shape-from-Shading
under Near Point Lighting and Partial views for
Orthopedic Endoscopy”
Chenyu Wu, Srinivasa
G. Narasimhan, Branislav Jaramaz,
Workshop on Photometric Analysis For Computer Vision (PACV’07), in conjunction with
ICCV’07
[PDF]
[Adobe Best Paper, PACV 07]
“Endoscope
Calibration and Derivation for Shape from Shading”
Chenyu Wu, Srinivasa G. Narasimhan, Branislav
Jaramaz,
Tech. Report CMU-RI-TR-07, Robotics Institute, Carnegie Mellon
University, Dec, 2007
[PDF]
Videos
|
|
|
Geometric Calibration: (wmv).
|
|
|
|
Photometric Calibration: (wmv).
|
|
|
|
Capture Images: (wmv).
|
|
|
|
Endoscopic Images: (wmv).
|
|
|
|
Remove
Illumination Effect: (wmv).
|
|
|
|
Shape from shading from Partial Images: (wmv).
|
|
|
|
Global Shape from shading from Multiple
Partial Images: (wmv).
|
|
Pictures (click on thumbnails to
enlarge images)
|
|
|
Endoscope:
An oblique endoscope consists of a scope cylinder with a lens and two point light
sources at the tip (the tip has a tilt from the scope cylinder) and a camera head
that captures video. The scope cylinder is connected flexibly to the camera head via a coupler.
The
scope cylinder or the camera head can be rotated together or
separately.
|
|
|
|
Geometric model:
(a) The geometric model of endoscope based on
a tracking
system.
(b) Two optical markers are attached to the scope
cylinder
and camera head and tracked separately.
|
|
|
|
Photometric
model:
Perspective projection model for endoscope
imaging system
with two near point light sources: O is the
camera projection
center. s1 and s2 are two light sources. We assume the plane
consisting of
O, s1 and s2 is parallel to the image plane. The
coordinate system is centered at O and Z is
parallel to the optical
axis,
and pointing toward the image plane. X and Y are parallel to
the
image plane. F is the focal length. a and b are two parameters
related
to the position of the light sources. Given a scene point
P, the corresponding image pixel is p.
Assuming a Lambertian
surface, the surface illumination therefore
depends on the surface
albedo,
light source intensity and fall off, and the angle between
the
normal and light rays.
|
|
|
|
Images used
for calibration:
Images used for geometric calibration (1) and
photometric
calibration (2-3).
Row 2-3 show different light intensities and
Column a-c show different albedos
on a color-checker.
|
|
|
|
Results
of shape from shading from single image:
(a) input
image. (b) shape from shading. I.
synthesized sphere. II.
cylinder.
III. a part of spine. This algorithm performs well for the
synthesized image and endoscopic images of
simple objects with complete boundaries. For the result of the artificial
spine, however, a part of the shape is not accurate due to the partial
boundary. Furthermore, due the small field of view, the reconstruction
from the single image cannot provide enough information about the
complete shape of the bone.
|
|
|
|
Simulation
results of shape from shading from multiple
views: (a)-(d)
Synthesized images of different parts of a sphere.
(e)-(f) Different
views of reconstructed sphere.
|
|
|
|
Reconstruction for the cylinder (37mm x
12mm x 12mm)
from
6 images.:
(a) Six images
are captured by moving the endoscope along the cylinder axis. Four of them
are shown as an illustration. (b) After removing the distortion and
illumination effects, the boundary in each image is labeled by hand, and
then the initial normals are computed
automatically. (c) Shape from each single image are
reconstructed using the method described in Section 3. (d) Unaligned
partial shapes in the world coordinates. (e)
Aligned shapes (gray) and contours (color) using ICP in the world
coordinates. (f) Updated local contours based on the global boundaries.
(g) Final shape are reconstructed using the
method described in Section 4. Note low resolution of global constraints
can cause aliasing on the surface, which can be alleviated by increasing
the resolution (see better result in next figure).
|
|
|
|
|
|
|
|
Reconstruction for the artificial spine
(147mm x 60mm x 60mm) from 18 images:
(a) 18 images
are captured by moving the endoscope horizontally (only translation). (b)
Unaligned partial partial shapes in the world
coordinates. (c) Aligned shapes in the world coordinates. (d) Aligned
contours (boundaries) in the world coordinates.(e)
Final shape are reconstructed using the method described in Section 4.
|
|
|
|
Comparison between the reconstructed
spine with the ground truth obtained using a laser range scanner:
(a)
Reconstructed shape (yellow) and ground truth surface(
red). (b) Error map
(vertex distance from the reconstructed
shape to the ground
truth surface) in HSV color space. (c) Error
distribution. For
comparison, we choose only the points that are
on the surface of
the spine. Then, the maximum, minimum,
mean and RMS
errors are 3.1mm, 0.0mm, 1.16mm
and 1.5mm
respectively. This level of accuracy is practical
for surgery.
|
|
|
|
|
