Nick Hawes Long-Term Autonomy in Everyday Environments

Long-Term Autonomy in
Everyday Environments
A New Challenge for AI and Robotics
Nick Hawes
http://nickhaw.es
@hawesie
School of Computer Science, University of Birmingham, UK
Long-Term Autonomy
in Everyday
Environments
http://strands-project.eu
Robust,
intelligent,
autonomous
behaviour
Exploitation of
structure for
improved
performance
A New Challenge
for AI and Robotics
Novel
opportunities
to learn
structure
environment
Long runtimes in
everyday
environments
Exploitation of
structure for
improved
performance
A New Challenge
for AI and Robotics
Long runtimes in
everyday
environments
Meta-room mapping
Desktop
observations
G4S Technology,
Object presence checks
UK
Door checks
Information
Hausprovision
der
Object
presence checks
Barmherzigkeit,
Door checks
Austria
G4S Technology, Challenge House, Tewkesbury, UK
690m3
Haus der Barmherzigkeit, Vienna, Austria
1030m3
Application
Specific
Task Action
Task Action
Task Action
Task Action
Routine
Executive
Control
Task Executor
Nav Learning
Monitoring
Localisation
& Navigation
Topological
Continuous
Scheduler
Application
Specific
Task Action
Task Action
Task Action
Task Action
Routine
Executive
Control
Task Executor
Nav Learning
Monitoring
Localisation
& Navigation
Topological
Continuous
Scheduler
Continuous
Topological
Continuous
Monitoring
Topological
Continuous
Nav Learning
Monitoring
Topological
Continuous
Task Executor
Nav Learning
Monitoring
Topological
Continuous
From 9:00 to 17:00
Weekdays, except 26/5/14
Check fire doors
Check fire extinguisher
Routine
Task Executor
Check all doors
Observe desks
Patrol corridors
Check fire doors
Map offices
Nav Learning
Monitoring
Topological
Continuous
Check all doors
Observe desks
Patrol corridors
Charge
Upload data
Replicate database
Process maps
Routine
Task Executor
task
Scheduler
task
task
Nav Learning
task
task
Monitoring
Topological
Continuous
Task Action
Task Action
Application
Task Action
Specific
Task
Action
Routine
Executive
Task Executor
Control
Nav Learning
Monitoring
Localisation
Topological
& Navigation
Continuous
Scheduler
Care
Security
Deployment
14/5/14 to 4/6/14
22/5/14 to 12/6/14
Working Hours
Weekdays, 8.00 to 17.00
Weekdays, 8.45am to 17.45
Distance
27.94km
20.64km
Tasks Completed
1985
963
Autonomous Time
48h 53m 17s
26h 18m 51s
System Lifetime
Max SL
171h 0m (7d 3h 0m)
91h 0m (3d 19h 0m)
Max SL working
48h 40m (2d 0h 40m)
39h 30m (1d 15h 30m)
wait
object check
door check
metric map
desktop perception
wait
patrol
object check
idle/engagement
door check
G4S Technology,
UK
Haus der
Barmherzigkeit,
Austria
A New Challenge
for AI and Robotics
Exploitation of
structure for
improved
performance
Long runtimes in
everyday
environments
Best 8 matches between straight-line and recorded times
mean time from
robot
straight line time
Worst 8 matches between straight-line and recorded times
mean time from
robot
straight line time
W3
0.1
W1
0.9
W2
action goto W2 from W1
cost mean time from all attempts
W3
0.1
W1
0.9
W2
express navigation goals in Linear Temporal Logic
e.g. (F W2) (eventually reach W2)
W3
0.1
W1
0.9
W2
╳
¬W2
true
W2
B. Lacerda, D. Parker, and N. Hawes. Optimal and Dynamic Planning for Markov
Decision Processes with Co-Safe LTL Specifications. In: IROS 2014.
B. Lacerda, D. Parker, and N. Hawes. Optimal and Dynamic Planning for Markov
Decision Processes with Co-Safe LTL Specifications. In: IROS 2014.
Qualitative Spatial Relations
(QSRs)
Akshaya Thippur et al. KTH-3D-TOTAL: A 3D Dataset for Discovering
Spatial Structures for Long-Term Autonomous Learning. In SAIS’14.
Lars et al. Bootstrapping probabilistic models of qualitative
spatial relations for active visual object search. In AAAI SS
2014 on Qualitative Representations for Robots
Hole punch
Glass
Headphone
Keys
Calculator
Lamp
Mobile phone
Laptop
Stapler
Desktop PC
Bottle
Book
Pen/Pencil
Cup/Mug
Telephone
Mouse
Keyboard
Monitor
Probability
Object Presence Probability
1
0.75
0.5
0.25
0
1.0
book
wrt. monitor
0.5
0.0
left
right
front
behind
close
distant
left
right
front
behind
close
distant
left
right
front
behind
close
distant
left
right
front
behind
close
distant
left
right
front
behind
close
distant
1.0
mug
wrt. monitor
0.5
0.0
1.0
PC
wrt. monitor
0.5
0.0
1.0
keyboard
wrt. monitor
0.5
0.0
1.0
mouse
wrt. monitor
0.5
0.0
Position of cup relative to monitor
Position of cup relative to keyboard
Supporting planes vs QSRs
10 trials
3 out of 8 tables
choose 1/500 sim. desks
L. Kunze, K. K. Doreswamy and N. Hawes.
Using Qualitative Spatial Relations for Indirec
Object Search. In ICRA’14.
Search Results (Simulation)
Objects Found (/10)
10
10
68.5
Time (secs)
Poses
70.0
10
65.0
8 55.0
7.5
52.5
6
6
5
35.0
4.8
33.6
3.1
2.5
2.3
3.2
17.5
15.6
1.1
0
0.0
Random Views
Supporting Planes
Correct QSRs
Partially Correct QSRs Misleading QSRs
Search Results (Robot)
Objects Found (/10)
10
Time (secs)
Poses
70.0
10
69.5
9
7.5
52.5
5
35.0
33.4
2.5
17.5
2.2
1.1
0
0.0
Supporting Planes
Correct QSRs
Qualitative Spatial Relations
(QSRs)
train: 19 desks, 3 scenes per desk = 57 scenes
test: 1 desk, 3 scenes per desk = 3 scenes
Classification Results (Robot)
With Visual Classification
Without Visual Classification
100.0
96.0
95.65
92.3
90.98
89.9
88.94
75.0
65.0
59.2
59.2
54.72
50.0
45.38
25.0
0.0
0
No Relations
Learnt Metric Relations
Ternary Point Calculus
Ternary Point Calculus
Ternary Point Calculus
Distance
Relative Size
Ternary Point Calculus
Distance
Relative Size
Connectivity
Lars Kunze et al. Combining Top-down Spatial Reasoning and Bottom-up Object
Class Recognition for Scene Understanding. In IROS ’14.
Rares Ambrus et al. Meta-Rooms: Building and Maintaining Long Term Spatial
Models in a Dynamic World. In IROS ’14.
Rares Ambrus et al. Meta-Rooms: Building and Maintaining Long Term Spatial
Models in a Dynamic World. In IROS ’14.
Rares Ambrus et al. Meta-Rooms: Building and Maintaining Long Term Spatial
Models in a Dynamic World. In IROS ’14.
Rares Ambrus et al. Meta-Rooms: Building and Maintaining Long Term Spatial
Models in a Dynamic World. In IROS ’14.
A New Challenge
for AI and Robotics
Exploitation of
structure for
improved
performance
Long runtimes in
everyday
environments
A New Challenge
for AI and Robotics
Robust,
intelligent,
autonomous
behaviour
Exploitation of
structure for
improved
performance
A New Challenge
for AI and Robotics
Novel
opportunities
to learn
structure
environment
Long runtimes in
everyday
environments
http://strands-project.eu
Long-Term Autonomy in
Everyday Environments
A New Challenge for AI and Robotics
Nick Hawes
http://nickhaw.es
@hawesie
School of Computer Science, University of Birmingham, UK