Symbolic model of Spatial Relations in the Human Brain Olivier Dameron SMI - Stanford University Mapping the Human Body Workshop Buffalo 2005
Context Why a symbolic model of brain cortex anatomy? Medical practice diagnosis: search for similar cases surgery planning: suspiscion of important functional area computer aided surgery: anatomy is a landmark Medical research anatomical variability (intra and inter individual) relation between anatomy and function
Context This model has to be usable by software formalised model AND representation reusable
What are we talking about? brain: two globally symetric hemispheres folded surface: sulci and gyri, lobes, etc grey substance and white substance
Left 2 Globally symmetric hemispheres Front Back E. Berton Laboratoire d'anatomie. Univ. Rennes Right
Top Gyrus Sulcus Back E. Berton Laboratoire d'anatomie. Univ. Rennes Front
Top Parietal Lobe Frontal Lobe Back Occipital Lobe Front Temporal Lobe
Left White substance Back Front Cortex (Grey substance) E. Berton Laboratoire d'anatomie. Univ. Rennes Right
Need for Spatial Relations
Why are Spatial Relations useful? Modeling Segmenting images Labeling entities in images Describing the location of non-anatomical entities (pathologies, brain activation areas...)
Modeling Some entities are (partially) defined by their position wrt other entities Central sulcus between Frontal and Parietal Lobes the 3 pars of Precentral Gyrus (gyrus-gyrus) superior-middle-inferior but NOT superior-inferior-middle the 3 operculum of Inferior Frontal Gyrus (gyri-sulci)
Top Parietal Lobe Back Central Sulcus Frontal Lobe Front
Top Superior pars Middle pars Inferior pars Back E. Berton Laboratoire d'anatomie. Univ. Rennes Front
Segmenting images current segmentation algorithms are dumb only rely on the value of voxels (and neighbors) no matter how complicated the algorithms are, this is still the wrong approach we have a priori knowledge about what entities should look like we have a priori knowledge about the constraints that have to be respected currently, this aspect is ignored or at best harcoded and application-specific
Labeling entities in images Current approaches rely on: image-based atlases (VoxelMan; Kikinis et al; Van Essen et al.) statistical knowledge (SPAM Montreal univ.) supervised learning (e.g. Riviere et al.) => they are highly sensitive to natural anatomical variability and to deformation => we expect qualitative description to be more robust
Describing location Describing location of entities using anatomical landmarks pathological entities may not be directly visible well known functional areas of the brain => the spatial relationships are between anatomical structures and the other entities => the spatial relationships are required for symbolic reasoning
Motor area (Precentral G.) Somesthesy area (Postcentral G.)
Which spatial relations do we need?
Spatial Relationships Between cortical structures Similar to anatomy and geography Between sulci Similar to works about cavity Between cortical structures and sulci Similar to separation by river
Top Continuity Back E. Berton Laboratoire d'anatomie. Univ. Rennes Contiguity Front
Top Branch Back E. Berton Laboratoire d'anatomie. Univ. Rennes Connection Front
Inflated Flatenned
Top Separation by Sulcus Back E. Berton Laboratoire d'anatomie. Univ. Rennes Front
Problems
Problems 1. Anatomical variability left / right asymetries between individuals There is not much we can do about it...... but symbolic models have to take it into account
Problems 2. Lack of precise definition for the entities what is a sulcus? what is a gyrus? where does it end? how do we differentiate the gyri? => we are talking about relations between things that we don't know very well; the relations are involved in the identity of these things, but how? for the relations how is the computer supposed to use this? => what do these relations mean?
Sulcus Gyrus Gyrus
Problems 3. Multiple points of view Teaching: gyri are visible parts of cortex Functional location: cortex (visible + buried) Pathol. location, Surgery: cortex + white subst.
Teaching Localization of functional activities Localization of pathologies, Surgery
Approach
Modeling Principles SNAP Stratified ontology (Borgo) separate the anatomical entities from the space region they take up (eventually) the portions of matter they are made of Spatial partitioning (transitivity + closure) Make explicit the relations Define the neighborhood relations between anatomical entities by topological relations between their regions
Anatomical variability
Handling anatomical variability Explicitely describe: which relations are mandatory which relations are optional which relations are impossible Does not require formal definitions for the relations Future work: incorporate statistical information
Stratified ontology
Space regions for anatomical entities Every physical anatomical entity is associated with exactly one space region A space region is associated to at most one physical anatomical entity seems correct for the cortex, is it always true?
Space regions for anatomical entities are one-piece (at least macroscopically) there is no equivalent to the region of the USA for anatomy Theoretical works: Borgo et al. (requires points) Cohn et al. (pointless geometry) anatomical entities take up internallyconnected space regions definitions of CON(x) and INCON(x) in [Cohn01]
Properties of the relations
Theoretical background: RCC
Theoretical background: RCC Use a primitive connection relation C(x,y) (reflexive and symmetric) Use C to define the other mereotopological relations: DC(x,y) =def C(x,y) P(x,y) =def z (C(z,x) C(z,y)) PP(x,y) =def P(x,y) P(y,x) EQ(x,y) =def P(x,y) P(y,x) O(x,y) =def z (P(z,x) P(z,y))...
Theoretical background: RCC Use C to define the other mereotopological relations: DC, P, PP, EQ, O, PO, DR, EC, TPP, NTPP,... These relations can be organized in a taxonomy We can also use C to define internally connected region (INCON(x))
Proposed definitions 2 cortical structures are continuous iff their space regions are EC 2 sulci are connected iff their space regions are EC a sulcus separates 2 gyri iff their space regions are mutually EC 2 cortical structures partially overlap iff their space regions are PO
Handling multiple points of view
Problem Depending on the context, the topological properties of the spatial relationships between anatomical entities are different
Proposed solution Use the third one (previous picture) describe additional parts cortical layer visible part of cortical layer consistently with the modeling principle, each of them is associated with exactly one space region define properties for these entities thus, we remove ambiguity
Discussion
Benefits: link with numeric information Information = data or knowledge Automatic interpretation, labelling... Use automatic segmentation and labelling for: automatic validity checking of Kn. Base obtaining new knowledge (e.g. statistical studies of variabilities)
Benefits: consistency management Some consistency constraints are direct consequences of the properties of the relations This result can be generalized to other domains (e.g. geography) Maintenance and scalability: is it possible to generate automatically all the strate about space regions from the anatomical relationships?
Benefits: combination with other domains Typically: anatomy and {pathology, brain function,...} Asking for trouble: the 3 of them (and this is specially when symblic knowledge is useful) Direct mapping between an anatomical and a pathological entity requires to specify additional consistency constraints (e.g. a tumor located in a part of an organ is located in the organ) requires cautious definitions: is it included, is it a partial overlap...?
Benefits: combination with other domains (cont.) Using the stratified approach looks ontologically more sound allows formal definition: e.g. one of the firmly connected parts of the region of the pathology (if tumors can have metastasis) is a proper part (or overlaps with) of the region of the anatomical entity scales up pretty well requires expressive formalisms (role composition, rules...)
Open questions How does this scale (specially when we go at the microscopic level)? How does it fit whith broader models? mereotopologically (FMA) taxonomically (upper-level ontologies) Do we have representation language expressive enough to support this?
Open questions Is it really useful/ worth the trouble? yes: consistency yes: automatic help for acquiring new knowledge Link with numeric data for segmentation and labeling: new approach
Open questions Take boundaries into account Fiat and/ or bona fide Crisp vs fuzzy What would be the added value? from the modeling perspective from the application perspective
Conclusion Stratified ontology: distinction between anatomical entities and their space region (approach consistent with previous ontological recommendations) Contraint on the space region Spatial relationships between anatomical entities defined by (mereo)topological relations between their space region Benefits: explicit formal definitions, generalization, extension, link with numeric info
Conclusion (and disclaimer) Recognized the need for more formal definition of spatial relations Provided a tentative that has to be perfected domain: FMA, O Bodenreider... formally: wich flavor of RCC? ontologically: upper-level Yet, demonstrated several advantages: consistency extensibility