Experiences in Developing An Intelligent Ground Vehicle (IGV) Ontology In Protégé Craig Schlenoff, NIST Randy Washington, DCS Corporation Tony Barbera, NIST July 8, 2004
Background: Agenda What is an Intelligent Ground Vehicle (IGV)? NIST 4D/RCS Methodology and Architecture Ontology Development: 4D/RCS to Ontology Mapping Interchange Formats and Upper Ontologies IGV Military Equipment IGV Behaviors IGV Conditions Current Status Issues and Lessons Learned
What is an Intelligent Ground Vehicle?
DeployToStartPoint MainRouteRecon Evaluate&ClassifyObstruction MoveToControlPoint PassVehInF ron t. PassVehInFront DriveOnTwoLaneRd PassVehInFront DriveOnTwoLaneRd NegotiateLaneConstrict 4D/RCS Mapping to IGV Instances Ontology Commander & Vehicle Instances RoadMarchToAssemblyArea RouteRecon RoadMarchToAnAssemblyArea(AA) FollowPlatoon PrepareFor PrepareDetailed Secure Organize MoveInto ToAssemblyArea RoadMarch MovementPlans AssemblyArea AssemblyArea MarchFormation Light Cavalry Troop PrepareForRoadMarch PrepareForRoadMarch MoveIntoFormation MoveIntoFormation ExecuteRoadMarch ExecuteRoadMarch ExecuteHalt ExecuteHalt OccupyAssemblyArea OccupyAssemblyArea FollowPlatoonToAssemblyArea RouteRecon SecureAssemblyArea AreaRecon OrganizeAssemblyArea KNOWLEDGE DATABASE (Executing) VALUE JUDGMENT (RightFlankRecon) DOT Driving Manuals and ARMY Field Manuals + DeployTo Locate&Secure MoveTo StartPoint LeftFlank RightFlank MainRoute ObstacleBypass Obstacle ControlPoint DominateTerrain RouteRecon RouteRecon Recon Recon Recon MoveTo Locate Perform Ford Secure MoveTo Overwatch ControlPoint WaterBypass Ford Water Area Cover/Concealed Section Recon Obstacle Position Command Group (FSO) Scout Platoon 1st Quartering Party DeployToStartPoint LeftFlankRecon Locate&SecureObstacleBypass MoveToControlPoint A Section LocateWaterBypass PerformFordRecon FordWaterObstacle SecureArea OverwatchSection Scout Platoon 3rd DeployToStartPoint RightFlankRecon DominantTerrainRecon AmbushSiteRecon MoveToControlPoint B Section C Section SENSORY INPUT SENSORY WORLD BEHAVIOR PROCESSING MODEL GENERATION (Platoon B Section Control BEHAVIOR Module) (Executing) (ReconToRoute) GENERATION. COMMANDED TASK (GOAL) STATE-TABLES Squad #2 Squad #3 Squad #7 #5 #6 #8 #4 PSG #9 #10 Assess FordTerrain ScanArea ForEnemy Cross Ford MoveTo Position AssessFordTerrain SendReport CrossFord MoveToPosition ScanAreaForEnemy NEXT SUBGOAL Domain Experts Scan Path MoveTo Water ShiftTo 4WhLo MoveTo Opposite Bank ShiftTo 4WhHi Dry Brakes Surveillanc (RSTA) FLIR, Laser Pan/Tilt Radar, Control Camera Comms Gaze Control ScanPath Mobility Primitive/ Trajectory MoveToWater ShiftTo4WheelLo MoveToOppositeBank ShiftTo4WheelHi DryBrakes Targeting Sensors Lethality (Gunner) Weapon Control Select WaterObsRecon Plan State-Table Task Decomposition Tree (Route Reconnaissance Example) Hierarchical Organization of Control Modules RightFlankRecon Selection Conditions Selected Plan NormalRouteReconSituation RightFlankRecon WaterObstacleDetected MinefieldDetected WaterObstacleRecon AssessMinefieldRecon 3 cm Saturated Ground Plant Height 6-18" LongLeafGrasses - very flat, long green leaves-purple/rose/yellow flowers. MostlyTrees,SomeBushes ExtensiveWaterSurfaceVisible SlowMovingWaterCoveredLand SignificantTractionSlip MajorGroundDeformation SwampDetected DefileDetected DefileRecon LateralRouteDetected LateralRouteRecon PLAN SELECTION TABLE 15 cm 6-48" Sedges - triangular tan/green stem plants, papyrus, narrow green to tan grasslike leaves, spikelets of inconspicuous tan-to-yellow-to-white flowers. StagnantWater OrganicMaterialOnWaterSurface IndeterminantGroundLevel Mosses,Evergreens,andShrubs BogDetected Environment Instances 2.4 cm.9 to 2.7m Cattails Object Groupings and Classifications Objects and Maps Features and Attributes Segmented Groupings ColorCameras LADAR Radar Stereo FLIR Nav SENSORY PROCESSING Six Feet of Water 1-6' 3-9' Float Objects & Attributes Reeds - tall, woody, thin, round, hollow jointed (tan-to-green) stem plants, long narrow green blade leaves, large feathery panicles (elongated clusters of tan/white/purple flowers along main stem). Bulrushes - tall tan-to-green stems, with dark brown cylindrical seed heads that explode into white down, long flat green sword shaped leaves, cattails. Water Lilies - very large floating green leaves with white flowers. SignificantTractionSlip WaterCoveredLand SomeWaterVisible ExtensiveGrassyVegetation MajorGroundDeformation WaterDepthToSixFeet TractionSlip FlowingWater NarrowWidth,IndeterminantLength NonVegetatedWaterInMiddle ErodedEarthEmbankments LargeSurfaceAreaOfStillWater OrganicMaterialMayBeOnWaterSurface LargeAreaWithoutGrasses,Trees,Shrubs BoundedBySwamp,Marsh FlowingWater SignificantWidth,IndeterminantLength NonVegetatedWaterInMiddle ErodedEarthEmbankments WaterSheenOnGroundSurface RuttedWithStandingWater LittleToNoVegetation SignificantTractionSlip SignificantGroundDeformation World States WORLD MODEL KNOWLEDGE MarshDetected LegalToPass StreamDetected Pond/LakeDetected RiverDetected MudDetected Situations NewPlan Condition Instances WaterObstacleRecon S1 sq5_nobypassonrouteside S2 sq5_nobypasstowardboundary S1 sq5_possibleforddetected S4 sq5_fordlookspassable S4 sq5_fordnotpassable S1 sq5_lateralbypassfound S1 SetupControlMeasures sq5_recontorouteflank sq6_travelingoverwatch sq8_visuallyclearobstacle-farside S2 SetupReport&NewControlMeasures sq5_recontofarflank sq6_travelingoverwatch sq8_visuallyclearobstacle-farside S0 SetupReport&NewControlMeasures sq5_movetacticallytocontrolpoint sq6_travelingoverwatch sq8_movetacticallytocontrolpoint S4 SetupReport&NewControlMeasures sq5_fordrecon sq6_nearsideoverwatch sq8_visuallyclearobstacle-farside S5 SetupReport&NewControlMeasures sq5_movetocover/concealposition sq6_nearsideoverwatch sq8_visuallyclearobstacle-farside S1 SetupControlMeasures sq5_recontorouteflank sq6_travelingoverwatch sq8_visuallyclearobstacle-farside S7 SetupReport&ControlMeasures sq5_obstaclefarsiderecon sq6_travelingoverwatch sq8_visuallycleartoroute Input Conditions Output Commands PLAN STATE-TABLE BEHAVIOR GENERATION Process Model Instances
Interchange Formats and Upper Ontologies OWL Neutral (W3C) interchange format XML base enables use XSLT transforms Provides access to emerging semantic web technologies OWL-S Rich semantics for describing complex processes (without being too complicated) Well suited to agent architectures Pieces of SUMO (Suggested Upper Merged Ontology) Class structure and properties provide a good starting point for developing domain specific ontology Native KIF format too complex for target community and not necessary for requirements capture Namespaces Used quite a bit to make ontology more manageable
IGV Conceptual Model Troop Commander Platoon Leader Section Lead Vehicle Commander Mobility System Propulsion Subsystem Platoon Leader Platoon Leader Section Lead Section Lead Vehicle Commander Vehicle Commander Survivability System Surveillance System Localization Subsystem Auxiliary Subsystem Engine Component Transmission Component External Request by a process Lethality System Automotive Subsystem Support System Navigation Subsystem Track-Drive Component Brake Component
Representing an IGV (cont.)
PassVehInFront. DriveOnTwoLaneRd PassVehInFront PassVehInFront DriveOnTwoLaneRd NegotiateLaneConstrict Tactical Behaviors Plan State-Table Selection RouteRecon Bridge Detected Scout Platoon 3rd WaterBridgeRecon LeftFlankRecon RouteReconToBridge RightFlankRecon A Section Bridge Detected C Section WaterObs Detected NoBypass OnLeft B Section WaterObsRecon SENSORY INPUT KNOWLEDGE DATABASE SENSORY PROCESSING Positive (AND) Gears In Reverse ommanded Velocity Negative (AND) Gears In Forward Recon ToCP Squad/Veh #2 Squad/Veh #3 GENERIC 4D/RCS AGENT CONTROL MODULE Traveling Overwatch Visually Clear Route Squad/Veh #7 VALUE Support Recon to Visually Traveling Route Recon Route Recon RightFlank Traveling ClearObs Overwatch To CP Overwatch Far-Side StartUpAndOperate #4 PSG #9 #10 #5 #6 #8 COMMANDED COMMANDED TASK (GOAL) TASK (GOAL) New StartupAndOperateCommand S1 proc_startengine S1 EngineStarted WORLD BEHAVIOR MODEL GENERATION S3 GearChanged NEXT S2 SUBGOAL COMMANDED SUBGOALS BEHAVIOR GENERATION JUDGMENT S2 GearChangeRequired STATE-TABLES S3 proc_changegear Positive But Gear Is In Reverse (or) Negative But Gear Is In Forward Gear Change Required. StartUpAndOperate S2 NewCommandedVelocity S4 proc_adjustenginethrottle S4 EngineThrottleAdjusted S2 New StartupAndOperateCommand S1 proc_startengine S1 EngineStarted S2 GearChangeRequired S3 proc_changegear S2 S3 GearChanged S2 S2 ShutDownRequested S5 proc_setgeartopark S5 GearInPark S6 ShutDownEngine S2 S6 EngineShutDown S0 Done Input Conditions S2 NewCommandedVelocity S4 proc_adjustenginethrottle S4 EngineThrottleAdjusted S2 ShutDownRequested S5 proc_setgeartopark S5 GearInPark S6 ShutDownEngine Output Input Conditions Commands Output Commands S2 S6 EngineShutDown S0 Done
Representing a Propulsion
Propulsion Graph
More Visualization Features
DriveOnTwoLaneRd PassVehInFront PassVehInFront DriveOnTwoLaneRd NegotiateLaneConstrict Conditions RouteRecon Bridge Detected Scout Platoon 3rd WaterBridgeRecon LeftFlankRecon RouteReconToBridge RightFlankRecon A Section Bridge Detected C Section WaterObs Detected NoBypass OnLeft B Section WaterObsRecon SENSORY INPUT SENSORY PROCESSING Recon ToCP Squad/Veh #2 GENERIC 4D/RCS AGENT CONTROL MODULE Traveling Overwatch Visually Clear Route Traveling Overwatch Route Recon To CP Squad/Veh Squad/Veh #3 #7 #4 PSG #9 Positive (AND) Gears In Reverse #10 Support Route Recon KNOWLEDGE DATABASE VALUE JUDGMENT Negative (AND) WORLD BEHAVIOR MODEL GENERATION Gears In Forward COMMANDED TASK (GOAL) COMMANDED SUBGOALS #5 Recon to RightFlank #6 BEHAVIOR GENERATION Traveling Overwatch #8. Visually ClearObs Far-Side COMMANDED TASK (GOAL) PassVehInFront STATE-TABLES. NEXT SUBGOAL Positive But Gear Is In Reverse (or) Negative But Gear Is In Forward StartUpAndOperate New StartupAndOperateCommand S1 proc_startengine S1 EngineStarted S2 GearChangeRequired S3 proc_changegear S2 S3 GearChanged S2 Gear Change Required Positive (AND) Gears In Reverse Negative (AND) Gears In Forward Positive But Gear Is In Reverse (or) Negative But Gear Is In Forward Gear Change Required S2 NewCommandedVelocity S4 proc_adjustenginethrottle S4 EngineThrottleAdjusted S2 S2 ShutDownRequested S5 proc_setgeartopark S5 GearInPark S6 ShutDownEngine S6 EngineShutDown S0 Done Input Conditions Output Commands
IGV Condition Example
Model Development Status OWL entities defined Classes 175 Properties 130 Instances 700
Issues and Lessons Learned Developing an ontology is a slow iterative process It difficult to evaluate a model construct without inputting detail. It is very difficult to change the model once you have entered any level of detail. Difficult to develop consistent rules for when to use a Classes vs. an Instance in a large domain Is knowledge in class restrictions or instances? Difficult to present large models to domain experts Experiences with OWL-S shows that it has applications outside of the semantic web. Would like to get involved in its development