A CRF Approach to Fitting a Generalized Hand Skeleton Model



Similar documents
Optical Illusion. Sara Bolouki, Roger Grosse, Honglak Lee, Andrew Ng

A Spam Message Filtering Method: focus on run time

Assessing the Discriminatory Power of Credit Scores

DISTRIBUTED DATA PARALLEL TECHNIQUES FOR CONTENT-MATCHING INTRUSION DETECTION SYSTEMS

Report b Measurement report. Sylomer - field test

DISTRIBUTED DATA PARALLEL TECHNIQUES FOR CONTENT-MATCHING INTRUSION DETECTION SYSTEMS. G. Chapman J. Cleese E. Idle

Profitability of Loyalty Programs in the Presence of Uncertainty in Customers Valuations

Two Dimensional FEM Simulation of Ultrasonic Wave Propagation in Isotropic Solid Media using COMSOL

A technical guide to 2014 key stage 2 to key stage 4 value added measures

1 Introduction. Reza Shokri* Privacy Games: Optimal User-Centric Data Obfuscation

Support Vector Machine Based Electricity Price Forecasting For Electricity Markets utilising Projected Assessment of System Adequacy Data.

TRADING rules are widely used in financial market as

Morningstar Fixed Income Style Box TM Methodology

CASE STUDY BRIDGE.

Growing Self-Organizing Maps for Surface Reconstruction from Unstructured Point Clouds

AN OVERVIEW ON CLUSTERING METHODS

A note on profit maximization and monotonicity for inbound call centers

Queueing systems with scheduled arrivals, i.e., appointment systems, are typical for frontal service systems,

Performance of a Browser-Based JavaScript Bandwidth Test

Redesigning Ratings: Assessing the Discriminatory Power of Credit Scores under Censoring

1) Assume that the sample is an SRS. The problem state that the subjects were randomly selected.

Improving the Performance of Web Service Recommenders Using Semantic Similarity

A Resolution Approach to a Hierarchical Multiobjective Routing Model for MPLS Networks

REDUCTION OF TOTAL SUPPLY CHAIN CYCLE TIME IN INTERNAL BUSINESS PROCESS OF REAMER USING DOE AND TAGUCHI METHODOLOGY. Abstract. 1.

Exposure Metering Relating Subject Lighting to Film Exposure

A New Optimum Jitter Protection for Conversational VoIP

Acceleration-Displacement Crash Pulse Optimisation A New Methodology to Optimise Vehicle Response for Multiple Impact Speeds

TIME SERIES ANALYSIS AND TRENDS BY USING SPSS PROGRAMME


Apigee Edge: Apigee Cloud vs. Private Cloud. Evaluating deployment models for API management

TOWARDS AUTOMATED LIDAR BORESIGHT SELF-CALIBRATION

BUILT-IN DUAL FREQUENCY ANTENNA WITH AN EMBEDDED CAMERA AND A VERTICAL GROUND PLANE

CHARACTERISTICS OF WAITING LINE MODELS THE INDICATORS OF THE CUSTOMER FLOW MANAGEMENT SYSTEMS EFFICIENCY

Unit 11 Using Linear Regression to Describe Relationships

Progress 8 measure in 2016, 2017, and Guide for maintained secondary schools, academies and free schools

Bi-Objective Optimization for the Clinical Trial Supply Chain Management

Risk Management for a Global Supply Chain Planning under Uncertainty: Models and Algorithms

Utility-Based Flow Control for Sequential Imagery over Wireless Networks

SENSING IMAGES. School of Remote Sensing and Information Engineering, Wuhan University, 129# Luoyu Road, Wuhan,

Performance of Multiple TFRC in Heterogeneous Wireless Networks

Mixed Method of Model Reduction for Uncertain Systems

Return on Investment and Effort Expenditure in the Software Development Environment

CASE STUDY ALLOCATE SOFTWARE

Scheduling of Jobs and Maintenance Activities on Parallel Machines

Independent Samples T- test

Brand Equity Net Promoter Scores Versus Mean Scores. Which Presents a Clearer Picture For Action? A Non-Elite Branded University Example.

EXPERIMENT 11 CONSOLIDATION TEST

Project Management Basics

Linear energy-preserving integrators for Poisson systems

QUANTIFYING THE BULLWHIP EFFECT IN THE SUPPLY CHAIN OF SMALL-SIZED COMPANIES

A Note on Profit Maximization and Monotonicity for Inbound Call Centers

Analysis of Mesostructure Unit Cells Comprised of Octet-truss Structures

Sector Concentration in Loan Portfolios and Economic Capital. Abstract

Simulation of Sensorless Speed Control of Induction Motor Using APFO Technique

Control of Wireless Networks with Flow Level Dynamics under Constant Time Scheduling

Research Article An (s, S) Production Inventory Controlled Self-Service Queuing System

CLUSTBIGFIM-FREQUENT ITEMSET MINING OF BIG DATA USING PRE-PROCESSING BASED ON MAPREDUCE FRAMEWORK

Mobile Network Configuration for Large-scale Multimedia Delivery on a Single WLAN

EVALUATING SERVICE QUALITY OF MOBILE APPLICATION STORES: A COMPARISON OF THREE TELECOMMUNICATION COMPANIES IN TAIWAN

International Journal of Heat and Mass Transfer

A Review On Software Testing In SDlC And Testing Tools

FEDERATION OF ARAB SCIENTIFIC RESEARCH COUNCILS

HUMAN CAPITAL AND THE FUTURE OF TRANSITION ECONOMIES * Michael Spagat Royal Holloway, University of London, CEPR and Davidson Institute.

Heat transfer to or from a fluid flowing through a tube

Bio-Plex Analysis Software

Availability of WDM Multi Ring Networks

Chapter 10 Stocks and Their Valuation ANSWERS TO END-OF-CHAPTER QUESTIONS

Design of Compound Hyperchaotic System with Application in Secure Data Transmission Systems

Introduction to the article Degrees of Freedom.

License & SW Asset Management at CES Design Services

G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences

Mathematical Modeling of Molten Slag Granulation Using a Spinning Disk Atomizer (SDA)

Growth and Sustainability of Managed Security Services Networks: An Economic Perspective

Optimizing a Semantic Comparator using CUDA-enabled Graphics Hardware

6. Friction, Experiment and Theory

A COMPARATIVE STUDY OF THREE-PHASE AND SINGLE-PHASE PLL ALGORITHMS FOR GRID-CONNECTED SYSTEMS

THE IMPACT OF MULTIFACTORIAL GENETIC DISORDERS ON CRITICAL ILLNESS INSURANCE: A SIMULATION STUDY BASED ON UK BIOBANK ABSTRACT KEYWORDS

Cluster-Aware Cache for Network Attached Storage *

Digital Communication Systems

Software Engineering Management: strategic choices in a new decade

Transcription:

A CRF Approach to Fitting a Generalized Hand Skeleton Model Radu Paul Mihail Univ. of Kentucky Dept. of Computer Science r.p.mihail@uky.edu Gutav Blomquit Univ. of Kentucky Dept. of Radiology gbl224@email.uky.edu Nathan Jacob Univ. of Kentucky Dept. of Computer Science jacob@c.uky.edu Abtract We preent a new point ditribution model capable of modeling joint ubluxation (hifting) in rheumatoid arthriti (RA) patient and an approach to fitting thi model to poteroanterior view hand radiograph. We formulate thi hape fitting problem a inference in a conditional random field. Thi model combine potential function that focu on pecific anatomical tructure and a learned hape prior. We evaluate our approach on two dataet: one containing relatively healthy hand and one containing hand of rheumatoid arthriti patient. We provide an empirical analyi of the relative value of different potential function. We alo how how to ue the fitted hand keleton to initialize a proce for automatically etimating bone contour, which i a challenging, but important, problem in RA dieae progreion aement. 1. Introduction Imaging of the hand i routinely done to diagnoe and ae the everity of dieae that alter the normal appearance of the muculokeletal ytem. One uch dieae i rheumatoid arthriti (RA), a chronic ytemic inflammatory autoimmune dieae that primarily affect joint. The ymptom are pain, welling and the lo of the joint function due to inflammatory procee. The underlying caue of RA i multifactorial [16, 8] including genetic uceptibilitie, nutrition, lack of exercie and environmental factor. Joint inflammation caued by RA lead to over-vacularization, proliferation and ynovial car formation. The ynovial proliferation i mot marked at the margin of the joint, where the ticht pace lead to bone eroion [3]. The inflammatory procee do not pare the ligament, tendon and mucle, which lead to weakne, laxity and deformity. We propoe a novel method to automatically fit a keleton model to a hand radiograph. Our approach build on a previou model by Fernändez et al. [17], who propoed a point ditribution model with landmark located at joint center. To upport the type of deformation common in RA, we relax thi model by adding additional landmark point. Intead of a ingle point per joint, we have one located on each of the cortical articular urface of adjacent bone in the joint. Thi modification allow u to model ubluxation (dilocation) of bone and upport our longterm goal of automatically meauring inter-joint pacing. We provide a probabilitic formulation of our approach a a Conditional Random Field (CRF) and how to perform learning and inference with the model. We ue a collection of potential function, each tuned to a particular anatomical feature, uch a upper and lower joint urface, or bone orientation. Each of thee feature make unique contribution to our fitting proce. We evaluate thee feature, and our CRF model, on real data from hand with and without deformation due to RA. Baed on an analyi of the relative value of different potential function, we find that the term that etimate the orientation of the joint make ignificant contribution to rough alignment but other term, uch a the upper and lower joint potential, make ignificant contribution by enabling more precie poitioning of landmark point. The main contribution of thi work are: 1) introducing a new point-ditribution model uitable for deformed hand, 2) a CRF framework for fitting thi model to hand radiograph, 3) the definition of a et of potential function that focu on pecific anatomical tructure in the hand, and 4) the evaluation of the accuracy of the method and the relative value of variou potential function on two dataet of hand radiograph. 2. Related Work In thi ection, we decribe previou work on viionbaed method for proceing and analyzing hand radiograph. Hand Radiograph Model Fitting Regitering a parametric model to a hand radiograph i a key problem in thi domain. Fernändez et al. [17] ued a landmark-baed wire model (which we extend in thi paper) to develop a regitration algorithm that outperform thin-plate pline (TPS). 1

They initialize the wire model through a cacade of image proceing routine baed on bone axe. Van de Gieen et al. [20] developed a method to regiter CT can of writ by enforcing ditance between bone urface to remain the ame after regitration. Bellerini et al. [2] ue nake optimized a initially propoed by Ka et al. [11] uing genetic algorithm. They encode the parametric nake a polar coordinate centered at the origin which can be placed arbitrarily on the image. Xu et al. [21] introduced gradient vector flow a external force, which eliminate the need to know a priori whether the nake will hrink or grow. Our work extend thi line of reearch by generalizing the contrained hape model of Fernändez et al. [17]. Thi modification lead to the need for improved image feature extraction. Hand Radiograph Pixel Labeling Numerou approache have been propoed for pixel-level labeling of hand radiograph, we preent everal recent example. Yukel et al. [22] ue a combination of feature claification and morphological operation to egment bone tiue from hand radiograph. Chai et al. [4] ue the gray level co-occurrence matrix to egment texture and egment bone tiue from oft tiue. Thee approache are imilar to our feature extraction approach, but our feature were developed to directly aid in model fitting, wherea thee were developed for other purpoe. Hand Radiograph Analyi Hand radiograph are ued frequently in medical diagnoi becaue the hand are where the pathology i mot evident. We introduce everal common medical ue for hand radiograph, each of which could benefit from the improved keletal model fitting method we propoe. Lang et al. [13] preented a combination of active hape model and active contour model to egment bone and detect eroion on RA patient hand radiograph. Their approach relie on an initialization baed on a local linear mapping net. We point out that hand radiograph of late tage RA patient are much more challenging due to evere ubluxation, which reult in overlapping bone, and joint pace narrowing which lead to weak edge information, thu decreaing performance of purely edge-baed method. In pediatric radiology, keletal age i an important indicator of a healthy development proce. Not only the bone location and contour are of interet; bone denity meaurement aid in the diagnoi of keletal development. One of the firt complete decription of a ytem for hand radiograph analyi for keletal age aement i preented by Michael et al. [18]. Hue et al. [10] propoed an algorithm to egment hand bone on hand radiograph of children. Their approach relie on an overegmentation uing the waterhed algorithm and region of interet extraction and merging algorithm to egment oft tiue and background from noie. Sotoca et al. [19] propoed a emiautomatic approach where a uer place the template at or near the center of a bone and the contour i approximated uing active hape model (ASM). Radiologit rely on expertie to aign a bone maturity core relative to age and gender. The mot commonly ued method to perform thi evaluation i the atla matching method by Greulich and Pyle (GP method) [2]. Thi i a time conuming proce and correct aement i highly dependent on the radiologit experience and expertie, thu automated method have been propoed. Giordano et al.[9] developed a method to predict bone age uing a combination of filtering and Gradient Vector Flow Snake with accuracy of 90%. Bayeian network have been ued by Mahmoodi et al. [14, 15]. Fuzzy ytem have been ued for keletal age aement by Aja-Fernändez et al. [1]. State-of-the-art bone egmentation and joint pace width meaurement approache rely on landmark detection algorithm, uually baed on a cacade of image proceing technique. Thi firt tep of landmark detection lead to mot failure in exiting algorithm. Our work fill that gap by accurately computing key anatomical point. roughly centered, currently done manually. Recent work by Davi et al. [7] provide encouraging reult on automating thi proce. 3. Problem Definition Given a roughly centered hand radiograph, our goal i to etimate landmark point location on the edge of cortical articular urface along the main axi of long bone. In thi ection, we formally define our hape model and identify key challenge in olving thi problem. 3.1. Shape Model We repreent a hape,, by a et of n landmark point location = (x 1, x 2, x 3,..., x n, y 1, y 2, y 3,..., y n ) T. The choice of landmark depend on the object of interet and the application, but for hand radiograph they are uually choen a joint center and fingertip. A recent example i the work of Fernändez et al. [17] where a hape model i ued a an initialization tep to an image regitration algorithm. We choe to generalize their repreentation by having two landmark per joint, one on each ide of the joint on the cortical articular urface of the bone, collinear with the bone main axi, (i.e., the tip of long bone). Figure 1 how a viualization of thi model. Thi relaxation allow u to model the ubluxation deformitie that are common in moderate to late tage rheumatoid arthriti patient. 3.2. Key Challenge Automated method for radiograph analyi rely on conitent alignment and appearance, which rarely happen in

B4 4d 4p 3d B3 3p 2d B 2 2p 12d B 8d 12 B 12p 8 11d 8p B 7d 11 B 7 11p 10d 7p 6d B 10 B 6 10p 6p 9d 5d 16d B16 16p 15d B 15 15p 14d B 14 14p 13d B19 19p 19d the ubpace panned by a Point Ditribution Model (PDM) and the appearance i defined a a collection of likelihood term that depend on local feature detector. We ue a local optimization trategy to jointly maximize feature repone at landmark location by minimizing an energy function. Minimizing energy in thi context i equivalent to maximizing the poterior ditribution over the correct location of landmark given an image. Our propoed CRF ha the following form: 1d 18d B 5 B 9 B 13 18p B 18 P ( I, θ) = 1 Z exp{ i { j Ψ j ( ip, id, θ)}+ (1) B 1 17d 5p 1p 9p 13p B 17 + k φ k ( i, θ) + ζ(, θ)} 17p Figure 1. Our propoed hape model: each egment correpond to a bone (B 1... 19). Individual point are indexed a proximal {1...19}p and dital {1...19}d. practice. In thi work, we focu on olving the alignment problem by fitting an initial hand model to the radiograph. We decribe everal important challenge in olving thi fitting problem. Depite attempt to control hand poition uing clinical protocol, hand radiograph of healthy patient how ignificant variation in poe. In addition to RA deformitie, uch a joint fuion, cleroi and ubluxation, other dieae including oteoarthriti may be preent further increaing variability in poe. For example, RA damage and pain can prevent patient from flattening their hand on the imaging urface. A olution to the hand model fitting problem mut be able to cope with ignificant change in poe and joint deformitie. The appearance of bone alo varie ignificantly from patient to patient. Thi i epecially true in RA patient becaue the dieae affect the denity and hape of individual bone. Many previou approache to the hand model fitting problem focu on pecific anatomical feature for alignment, but thi lead to brittle olution. Therefore, an approach that combine image information from multiple anatomical feature i needed. Our work addree both of thee concern in a conitent, and adaptable probabilitic formulation. 4. Approach We propoe a CRF-baed model that combine a hape prior with appearance term that identify variou anatomical tructure. The hape prior i baed on the ditance from where i i an index over model egment, j and k index our pairwie and unary appearance term, Z i the partition function, Ψ and φ are pairwie and unary appearance term, ζ i a hape model prior and θ i a weight vector we ue to balance the variou term. 4.1. Potential Function The data term in our model are baed on a collection of dicriminative feature that we combine into a et of potential function. Dicriminative Feature The ucce of the hape fitting proce i heavily dependent on a et of feature that are highly dicriminative. Recently, Coote et al. [5] howed how regreion voting uing random foret in the contrained local model (CLM) framework outperform exiting method on hape fitting. Our approach extend Contrained Local Model (CLM) [6], by formulating the hape fitting problem in a general CRF framework. We claify each pixel in an image independently uing a randomized deciion foret (RDF) claifier and ue dene SIFT feature a input. We train 4 RDF claifier, one for joint center (the midpoint between adjacent bone connected by joint), two for cortical articular urface point, one for proximal and one for dital, and a bone tiue claifier. For a new image, we compute DSIFT decriptor and run them through our RDF claifier. The output i a likelihood that repreent cla memberhip of each pixel. We note that thi proce implie independence in pixel memberhip (e.g., we could potentially have a pixel be labeled a both bone and cortical urface). Let the claifier repone for joint be f j ( i ) where i i a landmark point with a correponding image location. Similarly, we define f pc ( i ) and f dc ( i ) for proximal and dital cortical articular urface pixel. Finally, let f b ( i ) be the claifier repone for bone tiue. Example of RDF claifier output can be een in Figure 3.

We then apply a threholding operation on fj and fb to compute binary region of high probability bfj and bfb. Uing the threholded repone, we apply ditance tranformation, and combine them with value inide the region to compute dfj (i ) and dfb (i ). The ditance tranformation return 1 on the region border and 0 inide. The location inide the region are filled with 1 fj and 1 fb repectively. Thi approach increae alignment preciion by providing extra information about the mot probable location of anatomical interet point. Example can be een in Figure 4. In the next ection we how how we convert thee lowlevel image feature into potential function in our CRF model. Figure 2. Left: in red, egment pan the major axi of connected component in a binary image (bfb ) ued for initialization. Right: in green, the top 3 model from the training et with the lowet ICP regitration error. Pairwie potential The firt pairwie potential encode the compatibility between a egment in our hape model and evidence of bone tiue: t 1X dfb (pxn, pyn ) Ψ1 (ip, id ) = θ1 n n=1 (2) where the ummation i over point p ampled along a egment. Thi function take a input the ditance tranformation dfb and i low when a egment i placed over a bone. The bone evidence from the image can be further exploited by conidering egment orientation extracted from the threholded bone tiue claifier connected component. We define the following potential: Ψ2 (ip, id ) = θ2 (tan 1 (ip id ) fo (i ))2 (3) In the above equation, fo i a function that return an angle at an image location computed via a weighted averaging of angle of the connected component with repect to the horizontal image axi. Intuitively, if a egment i i placed perpendicular to the major axi of a connected component, the potential will be at it maximum. In Figure 4 we how fo for an image. We now define a pairwie potential that encode a prior over adjacent cortical articular urface in a joint: Ψ3 (id, i+1p ) = θ3 ( log N (0, Σ)) Figure 3. Left: joint center pixel probabilitie. Middle: color coded probabilitie for dital (red channel) and proximal (blue channel) cortical urface pixel. Right: bone pixel probabilitie. (4) where N i a Gauian with full rank covariance Σ computed from the training et. Thi term contrain joint pace to be at reaonable ditance in order to avoid local minima during optimization. Unary potential We define three term that encourage the landmark point to be near appropriate anatomical feature of the joint. The motivation for our firt unary potential i that all point on the model hould be in area of high Figure 4. Left and middle: Ditance tranformation dfj (i ) and dfb (i ) of threholded claifier repone. Right: Orientation term fo. probability indicated by our joint feature, fj. To further penalize point from being far from a joint, and to improve optimization performance, we ue dfj, which i an augmented verion of fj. Thi unary potential i defined a follow: φ3 (i ) = θ6 dfj (i ) (5) The econd two unary potential function encourage landmark point to align to the joint contour. Thee potential are defined a follow: φ1 (ip ) = θ4 fpc (ip ) (6) φ2 (id ) = θ5 fdc (id ). (7) and

For thi domain uing claifier, f dc and f pc, intead of a generic edge detector i critical becaue it allow the model to ditinguih between the true bone contour ued for diagnoi and analyi and apparent edge caued by the radiographic projection of other bone tructure. We find that in practice thee term complement each other. The firt i very important for rough initial alignment, while the econd two are critical for the precie alignment. See the evaluation ection for detail. 4.2. Shape Model Prior The hape prior term ζ(, θ) i ued to penalize unlikely hape. Uing Probabilitic Principal Component Analyi (PPCA), we eek to relate hape to a k-dimenional vector x that i normally ditributed with zero mean and covariance I(k): T = W x T + + ɛ (8) where W i the matrix of principal component and i the average hape. ɛ i the model noie component, aumed to be normally ditributed ɛ N (0, σ 2 I). Under thi model, i normally ditributed: P () = N (, W W T + σ 2 I(k)) (9) o our hape prior i a weighted negative log likelihood of P (): ζ(, θ) = θ 7 ( log P ()) (10) W and σ 2 are etimated uing an Expectation- Maximization algorithm from a training et of hand hape extracted from hand radiograph. 4.3. Shape Inference We now decribe our trategy for etimating the optimal hape model for a given hand radiograph. We firt compute a et of initial model, then ue local optimization for each and elect the bet. We ue threholded bone tiue claifier, bf b, for initialization by computing the connected component tatitic in the binary image. We model each component a an ellipe and compute it centroid, major axi, orientation and length. The line egment panning the major axi of the connected component (ee Figure 2) form the bai of our initialization cheme. We ue a variation of the Iterative Cloet Point algorithm to regiter ample from our training et to the egment extracted from the binary bone tiue claifier. The ICP regitration error i then ued to elect the 3 bet hape that are ued a tarting point for our local optimization. Our local hape objective function i the poterior probability of a hape,, given image data, or equivalently the log likelihood of our CRF model (1), which i defined a follow: E(ŝ, θ, I) = arg min + k { i j φ k ( i ) + ζ(, θ). Ψ j ( ip, id )} (11) To minimize (11) we ue coordinate decent with tep ize choen by an independent local earch in each dimenion. 4.4. Etimating CRF Weight We ue a upervied learning approach to etimate the model parameter θ. The partition function Z i NP-hard to compute [12]. We overcome the difficulty of computing Z in learning the model parameter by uing Peudo- Likelihood learning, where a uniform prior over model parameter i aumed by etting τ = in P (θ τ) = N (θ, 0, τ 2 I) where I i the identity matrix. We find the optimal parameter θ by minimizing the difference between our etimated hape and ground truth hape over a et of training image: ˆθ ML = arg min θ arg min i E(, θ, I i ) GT i 2. (12) The above minimization i non-convex ince we allow to vary during optimization. We ue the implex method with random retart to compute model parameter, θ. In the following ection we evaluate the model parameter and how reult from inference. 5. Evaluation Dataet We evaluate our method on two dataet with poterior-anterior view hand radiograph: the Digital Hand Atla Databae 1 and a et of 43 radiograph of RA patient from the Univerity of Kentucky Department of Radiology. The econd dataet i of a hand in a range of dieae tage, from minimal to extreme deformation. For evaluation purpoe, we manually determined landmark location for each image in both dataet. Since radiograph contrat varie due to calibration parameter and noie, we truncate the upper 20% of the intenity hitogram. Quantitative Evaluation We ued a et of 20 image from both dataet to train our dicriminative claifier and etimate our remaining model parameter. The remaining image were ued for teting and validation. We evaluate the model by computing the um of abolute difference between ground truth hape and reult from inference. The model error for both dataet can be een in Figure 7 and 6. For the Hand Atla et, the average per point 1 http://www.ipilab.org/baaweb/ 2

Figure 5. Reult on a repreentative ubet of tet image. Top row: healthy radiograph from the Hand Atla Databae. Bottom row: rheumatoid arthriti et. error wa 2.72 pixel, while for the RA dataet it wa 2.85 pixel. A comparion to the tate of the art i difficult due to our model landmark election and RA deformity everity. We divide the tet et into early (16 image), moderate (11 image) late tage radiograph (11 image), the average per point error (meaured in ditance from ground truth, in pixel) are a follow: 2.30, 2.24, and 4.56. To help undertand the failure mode of our approach, we further invetigate two radiograph with poor hape etimate. Thee correpond to image 23 and 27 in Figure 7. We find that by inpecting the optimal fit for both image, hown in Figure 8, that they are both from patient with latetage RA and have evere deformitie and ubluxation. In uch cae, aitance from a radiologit will be required. Term Contribution To provide more inight into the model, we etimate the amount each potential function contribute to reducing error in the RA dataet. We plit the dataet into two group, a training et of ize 20 and a teting et of ize 12. We ue the training dataet to etimate the optimal potential function weight, θ, by minimizing (12), a decribed above, for the full model. Then, for each of the even potential function, we olve for the optimal et of weight for the model without that potential function, leaving one out. Thi reult in a et of eight different model. For each model, we infer the hape in each image in the teting dataet and um the abolute pixel error with repect to the ground-truth hape to obtain an error meaure. Table 1 Table 1. Percent increae of error for each term when omitted from the model. Term Ψ1 Ψ2 Ψ3 φ 1 φ 2 φ3 ζ % Error increae 39.14 1.42 1.87 2.19 0.09 10.56 0.27 how the ratio of the error of a model without the potential function to the error for the full model. Intuitively, an important term will reult in a model with ignificantly higher error if it i removed. We find that there are two dominant term: Ψ1 and φ3 that correpond to bone and joint evidence from the feature et. Thee term are clearly the mot important in gro alignment, however the other term each make a contribution to reducing error in the full model. Example Application: Initialization for Etimating Bone Contour The weakne of mot tate-of-the-art approache for identifying bone contour i the initialization tep. A an example application, we propoe to fit an active hape model (ASM) to the cortical articular urface for each finger joint. Thi i challenging becaue ASM model mut be initialized very cloe to the optimal location or they will fall into non-optimal local minima. We ue our propoed approach to etimate a model keleton and ue the landmark point and bone egment to align the initial ASM model for each joint. We optimize the ASM hape parameter, uing an off-the-helf oftware library, and ob-

hape 10 20 30 40 50 60 70 70 60 50 40 30 20 10 hape 10 20 30 40 50 60 70 70 60 50 40 30 20 10 10 20 30 40 50 60 image Figure 6. Hand Atla Dataet model error computed a um of abolute difference from ground truth. 5 10 15 20 25 30 35 image Figure 7. Rheumatoid Arthriti Dataet model error computed a um of abolute difference from ground truth. tain the reult een in Figure 9. Thi demontrate that our etimated keleton model are ufficiently accurate to provide initial condition for ASM model of joint contour. In combination, uch an approach could be ued to automatically etimate the joint pace width, an important metric for RA progreion. 6. Concluion We introduced a new method for fitting wireframe-hand model to radiograph. A key innovation in our approach i in fitting a relaxed hape model, with four degree of freedom at each joint, that i capable of repreenting the dramatic ubluxation preent in patient with rheumatoid arthriti (RA). Fitting thi model effectively i more challenging than tandard model, which only have two degree of freedom at each joint. We how that our method, which combine low-level dicriminative feature in a conditional random field framework, i capable of fitting thi relaxed model on healthy hand a well a thoe deformed by RA. We provide quantitative reult that highlight which feature are mot important and how an application of our method to fitting bone contour, which i critical in aeing RA damage. Acknowledgement We thank Dr. Kritine Lohr and Judy Goldmith for helping u obtain the Rheumatoid Arthriti Dataet. We alo gratefully acknowledge DARPA grant D11AP00255 which partially upported thi work. Figure 8. Failure that correpond to high error from Figure 6 and 7. Reference [1] S. Aja-Fernández, R. de Lui-Garcıa, M. A. Martın- Fernández, and C. Alberola-López. A computational tw3 claifier for keletal maturity aement. a computing with word approach. Journal of Biomedical Informatic, 37(2):99 107, 2004. 2

Figure 9. Joint contour (red) etimated by initializing an active hape model baed on our initial hand keleton (green). [2] L. Ballerini and L. Bocchi. Multiple genetic nake for bone egmentation. Application of Evolutionary Computing, page 346 356, 2003. 2 [3] M. Buch and P. Emery. The aetiology and pathogenei of rheumatoid arthriti. HOSPITAL PHARMACIST-LONDON-, 9(1):5 10, 2002. 1 [4] H. Chai, L. Wee, T. Swee, and S. Huain. Glcm baed adaptive croed recontructed (acr) k-mean clutering hand bone egmentation. Recent Reearche in Communication, Automation, Signal Proceing, Nanotechnology, Atronomy and Nuclear Phyic, page 192 197, 2011. 2 [5] T. F. Coote, M. C. Ionita, C. Lindner, and P. Sauer. Robut and accurate hape model fitting uing random foret regreion voting. In Computer Viion ECCV 2012, page 278 291. Springer, 2012. 3 [6] D. Critinacce and T. F. Coote. Feature detection and tracking with contrained local model. In BMVC, volume 17, page 929 938, 2006. 3 [7] L. Davi, B.-J. Theobald, A. Tom, and A. Bagnall. On the extraction and claification of hand outline. In Intelligent Data Engineering and Automated Learning-IDEAL 2011, page 92 99. Springer, 2011. 2 [8] K. D. Deane, J. M. Norri, and V. M. Holer. Pre-clinical rheumatoid arthriti: Identification, evaluation and future direction for invetigation. Rheumatic dieae clinic of North America, 36(2):213, 2010. 1 [9] D. Giordano, C. Spampinato, G. Scarciofalo, and R. Leonardi. An automatic ytem for keletal bone age meaurement by robut proceing of carpal and epiphyial/metaphyial bone. Intrumentation and Meaurement, IEEE Tranaction on, 59(10):2539 2553, 2010. 2 [10] T. Hue, J. Kim, and M. Fahriddin. Hand bone radiograph image egmentation with roi merging. In Proceeding of the 13th IASME/WSEAS international conference on Mathematical Method and Computational Technique in Electrical Engineering conference on Applied Computing, page 147 154. World Scientific and Engineering Academy and Society (WSEAS), 2011. 2 [11] M. Ka, A. Witkin, and D. Terzopoulo. Snake: Active contour model. International journal of computer viion, 1(4):321 331, 1988. 2 [12] S. Kumar and M. Hebert. Dicriminative random field: A dicriminative framework for contextual interaction in claification. In IEEE International Conference on Computer Viion (ICCV), 2003. 5 [13] G. Lang, P. Pelochek, and H. Bichof. Am driven nake in rheumatoid arthriti aement. In Image Analyi, page 454 461. Springer, 2003. 2 [14] S. Mahmoodi, B. Sharif, E. Cheter, J. Owen, and R. Lee. Automated viion ytem for keletal age aement uing knowledge baed technique. In IEEE International Conference on Image Proceing and It Application, 1997. 2 [15] S. Mahmoodi, B. S. Sharif, E. G. Cheter, J. P. Owen, and R. Lee. Skeletal growth etimation uing radiographic image proceing and analyi. Information Technology in Biomedicine, IEEE Tranaction on, 4(4):292 297, 2000. 2 [16] M. Maravic, C. Berge, J. Daure, M. Boiier, et al. Practice for managing a flare of long-tanding rheumatoid arthriti: urvey among french rheumatologit. Clin Exp Rheumatol, 23:36 42, 2005. 1 [17] M. Martín-Fernández, R. Cárdene, E. Muñoz-Moreno, R. de Lui-García, M. Martín-Fernández, and C. Alberola- López. Automatic articulated regitration of hand radiograph. Image and Viion Computing, 27(8):1207 1222, 2009. 1, 2 [18] D. Michael and A. Nelon. Handx: A model-baed ytem for automatic egmentation of bone from digital hand radiograph. Medical Imaging, IEEE Tranaction on, 8(1):64 69, 1989. 2 [19] J. Sotoca, J. Iñeta, and M. Belmonte. Hand bone egmentation in radioaborptiometry image for computeried bone ma aement. Computerized Medical Imaging and Graphic, 27(6):459 467, 2003. 2 [20] M. Van De Gieen, G. J. Streektra, S. D. Strackee, M. Maa, K. A. Grimbergen, L. J. Van Vliet, and F. M. Vo. Contrained regitration of the writ joint. Medical Imaging, IEEE Tranaction on, 28(12):1861 1869, 2009. 2 [21] C. Xu and J. L. Prince. Generalized gradient vector flow external force for active contour. Signal proceing, 71(2):131 139, 1998. 2 [22] A. Yukel and T. Olmez. Automatic egmentation of bone tiue in x-ray hand image. In Adaptive and Natural Computing Algorithm, page 590 599. Springer, 2009. 2