Slides - Agenda

!CHAOS: a Cloud of Controls
Workshop CCR – Frascati
25-29 Maggio 2015
Michele Tota on behalf of !CHAOS project
A Cloud of Controls
Design and implementation of a prototype of Control as a
Services: an infrastructure at national level which offers a
cloud of services and procedures distributed and shared
over the LAN/WAN, to monitor and control any hardware
device, system or intelligent component and which provides
resources to processing services, data logging and
archiving.
Workshop CCR – Frascati 25-29 Maggio 2015
2
Why, when & where the project started
plant
→ sensors/actuators
Workshop CCR – Frascati 25-29 Maggio 2015
→ front-end
3
→ control room
control room
device
front-end controllers
Workshop CCR – Frascati 25-29 Maggio 2015
4
control room
object caching
BSON (Binary JSON)
\16\00\00\00\02hello\00\06\00\00\00world\00\00
device
control units (CUs)
Workshop CCR – Frascati 25-29 Maggio 2015
5
control room
NoSQL DB
data
management
(CDS)
device
control units (CUs)
Workshop CCR – Frascati 25-29 Maggio 2015
6
control room
EU
control units (CUs)
Workshop CCR – Frascati 25-29 Maggio 2015
7
control room
Metadata server
system dynamic configuration
device
control units (CUs)
Workshop CCR – Frascati 25-29 Maggio 2015
8
Controls & DAQ
control units (CUs)
Workshop CCR – Frascati 25-29 Maggio 2015
9
device
Realization of a prototype of Control as a
Services open platform suited for a large
number of applications in science, industries
and society
Control
Units
GUIs
devices
Workshop CCR – Frascati 25-29 Maggio 2015
10
!CHAOS strong points
EFFICIENCY
• Custom transfer protocol
• Support for standard protocol (RESTful)
easier to use or to implement
higher efficiency
insane overhead
VS
HTTP / RESTful
Workshop CCR – Frascati 25-29 Maggio 2015
Custom protocol
11
!CHAOS strong points
HETEROGENEITY
• Data abstraction with binary serialization
(binary JSON)
• Support for heterogeneous hw
– Linux kernel 2.6+
– Others can use !CHAOS Lite
Workshop CCR – Frascati 25-29 Maggio 2015
12
!CHAOS strong points
SCALABILITY
• Specifically designed to be a flexible and a
scalable system
In-memory
cache
CU
CU
NoSQL DB
…
CU
Workshop CCR – Frascati 25-29 Maggio 2015
Persistent
storage
13
!CHAOS infrastructure
Workshop CCR – Frascati 25-29 Maggio 2015
14
Back-end services: in-memory cache
CU
EU
UI
in-memory
cache
write
read/write
read
CDS
CDS
CDS
CDS instances
• scalability
• high performance and low latency on write operations
Workshop CCR – Frascati 25-29 Maggio 2015
15
Back-end services: in-memory cache
In-memory
cache
NoSQL DB
Persistent
storage
Workshop CCR – Frascati 25-29 Maggio 2015
16
Couchbase
Back-end services: NoSQL database
•
•
•
•
•
•
Needs to retrieve data fragments from stage
Data indexing for history
Insert, update and search operations
High availability
Scalability
Atomic find/modify operation
Workshop CCR – Frascati 25-29 Maggio 2015
17
Back-end services: MongoDB
Workshop CCR – Frascati 25-29 Maggio 2015
18
Back-end services: NoSQL database
In-memory
cache
Couchbase
NoSQL DB
MongoDB
Persistent
storage
Workshop CCR – Frascati 25-29 Maggio 2015
19
Back-end services: persistent storage
• Read random access
• Write sequential access
• High availability
– Redundancy
– Fault tolerance
– Recovery
+ FUSE
• Object storage analysis
Workshop CCR – Frascati 25-29 Maggio 2015
20
Back-end services: persistent storage
Workshop CCR – Frascati 25-29 Maggio 2015
In-memory
cache
Couchbase
NoSQL DB
MongoDB
Persistent
storage
Ceph
21
!CHAOS deployment
Control Unit
Execution Unit
User Interface
OpenVPN
Cloud
!CHAOS Services
CDS
CDS
CDS
MDS
MDS
MDS
Back-end services
Ceph / FS
Cluster MongoDB
Cluster Couchbase
Private and/or Pubblic Cloud
Workshop CCR – Frascati 25-29 Maggio 2015
22
!CHAOS deployment: state of the art
Control Unit
Execution Unit
User Interface
OpenVPN
Cloud
!CHAOS Services
CDS
CDS
CDS
MDS
MDS
MDS
Back-end services
Ceph
Cluster Mongo DB
Cluster Couchbase
OpenStack (CNAF / LNF)
Workshop CCR – Frascati 25-29 Maggio 2015
23
!CHAOS on multi-site infrastructure
•
•
OpenStack availability zone
Ceph geografical
Workshop CCR – Frascati 25-29 Maggio 2015
24
!CHAOS@BTF test-bed
FROM SINGLE MAGNET CONTROL
BTF DIAGNOSTICS accessing
!CHAOS
UP TO DAFNE MAG TERMINAL
Workshop CCR – Frascati 25-29 Maggio 2015
25
!CHAOS@AulaTouschek Monitoring
http://chaost-webui1.chaos.lnf.infn.it/Ed36.html
Workshop CCR – Frascati 25-29 Maggio 2015
26
!CHAOS mobile application
Workshop CCR – Frascati 25-29 Maggio 2015
27
http://www.frascatiscienza.it/chaosContest/
Workshop CCR – Frascati 25-29 Maggio 2015
28
Who is who
+
WP
Workpackage title
Lead
WP1
MANAGEMENT &
DISSEMINATION
INFN-LNF
WP2
ARCHITECTURE
SOFTWARE
DEVELOPMENT
INFN-LNF
Uni. TV
WP3 FRONTEND DRIVERS &
INTEGRATION
INFN-LNF
INFN-LNS
ADF &NI
WP4
INFN-TV
INFN-PG
INFN-LNS
HARDWARE
DEVELOPMENT
WP5 IT INFRASTRUCTURE &
SERVICES,
ACCESS POLICY
COLLABORATIVE
TOOLS
INFN-CNAF
INFN-LNF
INFN-LNS
Contact: WP2; C. Bisegni (LNF); WP3: A. Michelotti; WP4: G.
Salina (TV); WP5 E. Fattibene (CNAF); Technical Manager: A.
Stecchi (LNF); Project leader: G. Mazzitelli.
Workshop CCR – Frascati 25-29 Maggio 2015
29