PCA Based Analysis of External Respiratory Motion Using an RGB-D Camera

Introduction

Human respiration induces considerable external and internal motion in the thoracic and abdominal regions. Tracking and modeling of this motion is an important task for accurate treatment planning and dose calculation during external beam radiotherapy. Inaccurate motion tracking can cause severe issues such as errors in target/normal tissue delineation and increment in the volume of healthy tissues exposed to high doses. Different methods have been introduced to model the respiratory motion, but most of them use wearable markers or surgical node implanting techniques, which are inconvenient to patients.

In this research, we experiment the feasibility of using a commercial RGB-D camera along with Principal Component Analysis (PCA) to track and model the subject-specific external respiratory motion. Marker-based depth frame registration technique is also introduced to limit the measuring area into an anatomically consistent region during the treatment.

Overview of the proposed method

Process of respiratory motion modelling

System setup for data acquisition

An Asus Xtion PRO depth camera is used to acquire the depth data in real time covering the whole thoracic and abdominal region of the patient. However, the coverage of the depth sensor on the chest wall is variable due to the surface motion of the chest wall and also contain a lot of unnecessary regions. In radiotherapy, intended measuring area should be anatomically consistent during the whole treatment time for delivering the radiation dose accurately. Therefore, we attached four dot markers to the chest wall of the patient to define a consistent measuring area.

System setup for data aquisition

 Process of rectangular ROI generation

ROI Generation using Marker Detection

(a) Captured visual image. (b) After binarization using Otsu’s method. (c) Defining the measuring area after finding the center coordinates of the four markers. (d) Identified measuring area projected onto the aligned depth image. (e) Generated rectangular ROI using perspective transformation.

Principal Component Analysis

After acquiring the depth data, PCA is applied to capture the respiratory motion signal which is integrated into the major principal components. According to the observation on several input data, the first eigenvalue dominated the rest of the eigenvalues and accounted for over 95% of data variation. Therefore, only the first principal component can be used to represents the respiratory motion, significantly reducing the dimensionality of input depth data.

PCA

Respiratory motion modeling

We  reconstruct the depth data, which is in the metric domain, from the projection
coefficients and use it to model the respiratory motion.

Reference

  • U. Wijenayake and S. Park, “PCA Based Analysis of External Respiratory Motion Using an RGB-D Camera,” in 2016 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Benevento, Italy, 2016.
    [Bibtex]
    @INPROCEEDINGS{wijenayake2016pca,
    AUTHOR="Udaya Wijenayake and Soon-Yong Park",
    TITLE="{PCA} Based Analysis of External Respiratory Motion Using an {RGB-D} Camera",
    BOOKTITLE="2016 IEEE International Symposium on Medical Measurements and Applications
    (MeMeA)",
    ADDRESS="Benevento, Italy",
    DAYS=14,
    MONTH=may,
    YEAR=2016,
    KEYWORDS="Respiratory motion, PCA, RGB-D camera",
    ABSTRACT="Human respiration induces considerable external and internal motion in the
    thoracic and abdominal regions. Tracking and modeling of this motion is an
    important task for accurate treatment planning and dose calculation during
    external beam radiotherapy. Inaccurate motion tracking can cause severe
    issues such as errors in target/normal tissue delineation and increment in
    the volume of healthy tissues exposed to high doses. Different methods have
    been introduced to model the respiratory motion, but most of them use
    wearable markers or surgical node implanting techniques, which are
    inconvenient to patients. In this paper, we experiment the feasibility of
    using an RGB-D camera along with Principal Component Analysis (PCA) to
    track and model the subject-specific external respiratory motion.
    Marker-based depth frame registration technique is also introduced to limit
    the measuring area into an anatomically consistent region during the
    treatment. We evaluate the accuracy of the proposed method using a
    Spirometer and a laser line scanner."
    }

MuGSM (Multi-Directional Greedy Stereo Matching)

MuGSM(Multi-directional Greedy Stereo Matching)은 두 대의 카메라에서 획득한 영상으로

Fig.1 Flow of the MuGSM

> Objects
To obtain depth image from stereo images, we use stereo matching algorithm.
For a long time, many stereo matching algorithms have been developed.
And the performance are improving. But algorithms have weak points.
Those are long computation time, memory consuming, and scan line problem.
To solve these problems is the objects and motives.
We proposed an algorithm, whose accuracy is as good as SGM.
But, the computation time is shorter, and memory usage is smaller.
> Multi-directional Greedy

Make disparity maps using Greedy matching along 8 or 16 directions.

Fig 2. Matching directions Fig 3. Range of disparity
Number of matching direction is 8 or 16. The relation between current disparity and next disparity.
> Consistency Check
Find reliable area from greedy results using consisteny check.
Fig 4. Results of each greedy matching through 8 matching directions
Red point and blue point at each result mean same position of the original image.
Fig 5. Consistency check
Red points : The rate of inlier is higher than Consistency_ratio.
Blue points : The rate of inlier is smaller than Consistency_ratio.
> Iterative Expansion
1) After first ‘Consistency check’ get the reliable area
2) Restart Multi-directional Greedy Matching using reliable area.
3) Consistency check for the result of process 2.
4) Only in the holes(unreliable area), Iterate processes(2~3) are executed.
Fig 6. results of 1st iteration, 2nd iteration, 3th iteration, 4th iteration (from left-top, clockwise)
> Result
Computation time is 2~3 times faster than SGM’s. Memory usage is about 33% comparing to SGM’s. Accuracy almost equal.
Table 1. Compare with other stereo algorithms
Table 2. Memory usage
Table 3. Computation time