Angela Pawlowski Advanced Technology Laboratories 1 Federal Street

Advanced Technology Laboratories
1 Federal Street • A&E Building 3W
Camden, New Jersey 08102
Angela Pawlowski
Synopsis
• The Challenge of Multiple UAV
Manned/Unmanned Teaming
• The Role Of Intelligent Software In Meeting
The Challenge
• Steps Towards The Objective Force and
Beyond
Today
Six to ten operators manage a single UAV; human
and computer footprint is substantial.
Tomorrow
DARPA/Army plan is for one human to manage employment of
multiple UAVs, but UAV autonomy isn’t there yet – human and
computer footprint reduction is a challenge.
UAV Proliferation
UAV Proliferation Is Extensive
Intelligent Software Is The
Enabler!
The Aviation Applied Technology Directorate
and Lockheed Martin
have a vision for extending their associate
system
decision aiding software
to UAV control, exploitation,
survivability, and lethality for manned/unmanned teams,
promoting:
• Less human-intensive control
• Greater utility from the UAVs
• Enhanced UAV and manned platform survivability
• Greater coordinated team lethality
What’s An Associate?
• Definition of Terms
Coach
Knows how to act to make you better at your
responsibilities
Associate Knows how to help with your
responsibilities without being told
Assistant Knows how to do what it is told to do
Aid
Knows how to do only what is in a limited area of
performance
• Human Centric
• Mission Focused
Intelligent Software Systems
Vision & Teaming
5 1525
81
55
Associate Technology
• Total Mission Mgmt (Info & Action)
• Complete Cognitive Awareness
• Crew Intent Awareness
• Forward Thinking & Preparation
• Tailorable to Mission & User
• Tailorable Levels of Authority
• Completely Dependable & Trustworthy
• Capable of Autonomy
UAV Teaming
• Today: Functional Control from
Cockpit of Multiple, Simultaneous
UAVs
• Mid-Term: Control Evolution from
Operator to Supervisor
• Far-Term: Mothership
AVIATION Vision For Connectivity To
Multiple UAVs
The Airborne Manned Unmanned System
Technology (AMUST) Program brings together
UAV’s, Helicopters, and Associate Technology
Vigilante
Eagle Eye
Aviator Control of a UAV
and its Sensors is
Intuitive and User
Friendly
Hunter
ÿ Electronic Associate
Shadow 200
ÿ
ÿ
Predator
ÿ
Exdrone
ÿ
Fire Scout (Navy
A-160
UCAR
VTUAV)
ÿ
manages multiple UAVs
Common User Interface to
all UAVs
Common Control
Functionality
Associate manages UAVspecific control link
Teaming Capability is
Platform Independent
Provides ground
maneuver force with
multi-mission solutions
Associate Requirements Extend
Beyond Control
• Intelligent decision aiding must be extended to the
unmanned system platforms for increased autonomy
Today’s Capability
Manned UAV
System Grouped
Future Capability
Manned
System
UAV
Common
Goals
• Human-to-UV and UV-to-UV collaboration must be
defined and developed
UAVs
C2 on the
Ground
Dismounted
Warriors
UGVs
COMPLEXITY
COMPLEXITY
CAPABILITY
CAPABILITY
FLEXIBILITY
FLEXIBILITY
ADAPTABILITY
ADAPTABILITY
AATD Levels of Autonomous
Behavior
Autonomous - Conglomerate 10
Autonomous Teams with Unmanned
Leader/Mission Manager
Human-like Autonomy in a Mixed Team
9
8
Synergistic Multi-Mission Reasoning 7
Dynamically Mission Adaptable 6
Complex Missions Specific Reasoning
5
4 Semi- Automated Missions w/Simple
Decision Making
3 Scripted/Structured Mission
TRUSTED
TRUSTED
2 Automated Tasks & Functions INDEPENDENT
INDEPENDENT
1 Simple Automation
SELF
GOVERNING
SELF
GOVERNING
0 Manual -Remote Control
MINIMAL
INTERVENTION
MINIMAL INTERVENTION
COMPLEXITY
COMPLEXITY
CAPABILITY
CAPABILITY
FLEXIBILITY
FLEXIBILITY
ADAPTABILITY
ADAPTABILITY
Example Programs Developing
Autonomy
Only in Sci fi ‡ 9 & 10
Beyond FCS ‡ 8
UCAR ‡ 6
Autonomous - Conglomerate 10
Autonomous Teams with Unmanned
Leader/Mission Manager
9
Human-like Autonomy in a Mixed Team 8
FCS Block II ‡ 7
Synergistic Multi-Mission Reasoning 7
Dynamically Mission Adaptable 6
Complex Missions Specific Reasoning
Demo III ‡ 4
5
FCS Block 1, AMUST &
UCAV ‡ 5
4 Semi- Automated Missions w/Simple
Decision Making
3 Scripted/Structured Mission
A160 ‡ 3
TRUSTED
Shadow & Cruise Missile ‡ 2
TRUSTED
2 Automated Tasks & Functions INDEPENDENT
INDEPENDENT
1 Simple Automation
SELF
Hunter/Predator
‡ 1GOVERNING
SELF
GOVERNING
0 Manual -Remote Control
MINIMAL
INTERVENTION
MINIMAL INTERVENTION
Exdrone, RC Toys ‡ 0
Autonomy Technology Enabler:
Information
MENSA & Agents
Self-Aware
Execution
Explicit
Workflow
¸
¸
Guidance
Mission Effectiveness aNd
Survivability Agents
(MENSA)
•
•
•
•
•
Unexpected events
Vehicle safety
Impact to mission
Request deconfliction
Trigger replanning
Sharing
Collaboration
Exception
Handling
Teaming
Alerts
Self-Aware Agent Behavior
• Continuous self-monitoring
• Agents understand their own (and
other’s) capabilities
• Exception Handling
• Agent-Agent and Agent-Human
Team Interaction is created with:
• Semantic XML (DAML) Mark-Up
• CoABS Agent Development Toolkit (I2AT)
Self Awareness involves both physical and cognitive capabilities assessment.
Autonomy Technology Enabler:
UCAR
Focus is “teams” of autonomous entities (human and UAV), where
the UAVs are highly autonomous and able to collaborate with others.
Criteria for Collaboration
Common Relevant Operating Picture
Platform Capabilities Assessment
Teammate Registration
Information Needs Assessment
Comms for sharing information
Tailored information sharing
AATD Vision: Collaboration is Realized
through Distributed Associates
Airspace Mgmt
Associate
Personnel
Recovery
Associate
Lethality
Associate
Active
Intel
Associate
Weapons & SA
Associate
Link & Sensor
Associate
Mission
Associate
HAZMAT
Associate
Personnel
Associate
Collaboration Technology Enabler:
Cooperating AMUST-D Decision Aids
Mobile Commander’
Commander’s
Associate (MCA) Decision Aid:
• Situation Awareness
• Data Fusion
• Agent Based Data
Discovery
• Route Planner
• Plan Monitoring
• UAV Control & Monitoring
Warfighter’
Warfighter’s Associate
(WA) Decision Aid:
• Situation Awareness
• Data Fusion
• Route Planning
• Route Monitoring
• UAV Control
UAV:
• Radar
• Video/IR
IDM
TCDL
Collaboration Technology Enabler:
Grapevine
Grapevine Node 1
Data Fusion
Needs-based data exchange
Grapevine Node 2
Data Fusion
Grapevine Manager
Grapevine Manager
Grapevine Proxy
Agent 2
Grapevine Proxy
Agent 1
Grapevine Proxy
Agent 3
Grapevine Proxy
Agent 3
Grapevine Node3
Data Fusion
Grapevine Manager
Grapevine Proxy
Agent 1
Grapevine Proxy
Agent 2
Peer-to-peer
configuration
Collaboration Technology
Enabler: SAAVE, ISTAV, FUCHSIA
Inter-Vehicle
Collaboration
Support
• Synthetic Attention
Span for Autonomous
Vehicle Environments
(SAAVE)
• Intelligent Sharing of
Tasks among
Autonomous Vehicles
(ISTAV)
ATL develops collaboration technology for
UV-to-UV and Human-to-UV
HCI
Intelligent MultiVehicle C2 Support
• Fusion of Unmanned
Control, Health, and
Status Information
Assessment (FUCHSIA)
• Monitoring Execution
With Distributed Intelligent
Agents (MEDIA)
Collaboration Technology
Enabler: RAP Teams
Robots
Agents
People
UAVs
RAP Teams
Ground ‘Bots
Humans
Remote
Teammates
Agents
Robots, Agents, and People become interchangeable, peer components
teamed in the battlespace.
Goals - Identify functional capabilities required for peer-to-peer teaming
(e.g., collaboration, initiative, delegation, etc.).
- Define the technologies required to provide these capabilities.
Collaboration Technology
Enabler: Augmented Cognition
Intelligent Interruption + Context Recovery
In a Multi-tasking Environment
Agent
Language
Language
Generation
Generation
Text-to-Speech
Text-to-Speech
Conversion
Conversion
Audio
Audio
Server
Server
HUB
Speech
Speech
Recognition
Recognition
Frame
Frame
Construction
Construction
Serve
r
Information
Information
Access
Access
Priority
Priority
Manager
Manager
Dialogue
Dialogue
Management
Management
Alert
Alert
Management
Management
Context
Context
Tracking
Tracking
CoABS Agent
Technology
Cognitive
Workload
Assessment
Technology
Interruption
Management
• Cog. Break
Context • Negotiate
Recovery
• Immediate
Support
AugCog applies non-intrusive human workload measurement techniques
to adapt the behavior of the Decision Aid.
The Path Ahead …
• Continue to fund and develop autonomy and
collaboration-enabling programs
• Focus on a System-of-Systems approach
– Distributed, Heterogeneous, Autonomous Systems
– Associate System Integration built around Teaming
Concepts
• Coordinate with other services, DARPA, and
industry to leverage each other’s efforts and
emergent technologies