Lezione 2a - INFN Bari

Università degli Studi di Bari – Corso di Laurea Specialistica in Informatica
“Tecnologia dei Servizi “Grid e cloud computing”
A.A. 2009/2010
Giorgio Pietro Maggi [email protected], http://www.ba.infn.it/~maggi
Lezione 2a - 14 ottobre 2009
Il materiale didattico usato in questo corso è stato mutuato da quello
utilizzato da Paolo Veronesi per il corso di Griglie Computazionali
per la Laurea Specialistica in Informatica tenuto nell’anno
accademico 2008/09 presso l’Università degli Studi di Ferrara.
Paolo Veronesi
[email protected], [email protected]
http://www.cnaf.infn.it/~pveronesi/unife/
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
0
Grid: the “common market”
(definitions and implementations)
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
1
Power Grid inspiration: seamlessly delivering electricity as a
utility to users
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
2
Grid history




Name “Grid” chosen by analogy with
electric power grid (Foster and
Kesselman 1997)
Vision: plug-in computer for processing
power just like plugging in toaster for
electricity.
Concept has been around for decades
(distributed computing, metacomputing)
Key difference with the Grid is to realise
the vision on a global scale.
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
3
The Grid Revolution
The past
, the present, …
-CPU
-Disc
- Memory
- Input/Output
G
R
I
D
01011010110
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
M
I
D
D
L
E
W
A
R
E
4
The Grid Revolution
…the future: The Grid!
virtual
services
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
virtual
services
5
The Grid Metaphor
Mobile Access
G
R
I
D
Workstation
M
I
D
D
L
E
W
A
R
E
Supercomputer, PC-Cluster
Data-storage, Sensors, Experiments
Visualising
Internet, networks
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
6
The Grid Problem

Purpose:
flexible, secure, coordinated resource sharing among
dynamic collections of individuals, institutions, and
resources
 From “The Anatomy of the Grid: Enabling Scalable
Virtual Organizations”
enable “groups of users (virtual organizations)” to share
geographically distributed resources as they pursue
common goals – assuming the absence of…





central location,
central control,
omniscience,
existing trust relationships.
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
7
Virtual Organizations

Virtual organization (VO): a set of individuals and/or institutions
identified by the same set of rules, which define:





the resources shared (what);
the individual users allowed to share (who);
conditions under which sharing occurs (how).
VOs represent “community overlays” on classic organization
structures. They can be large or small, static or dynamic.
VO membership:

Single users and users of the same institution can be members of
different VOs
u3
u8
u2
u7
u9
u1
u4
VO1
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
u5
u6
VO2
VO3
8
Virtual Organizations (cont)

Examples:




An industrial consortium
Students from different university departments using
computing power for their simulation projects
physicists from different research institutions involved
in a the same experiment implementation, using the
Grid for analysis of the data generated by the
experiment
Astronomers from various research institutes
analyzing data gathered by multiple telescopes all
over the world
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
9
Authentication and authorization
Resource sharing requires owners to make
resources available, subject to contraints on
when, where and what can be done
 This requires:




Policies and mechanisms to express them in Policy
Decision Points
Authentication: the establishment of the identity of a
consumer
Authorization: determining whether an operation is
consistent with resource sharing rules applicable to
the consumer at Policy Enforcement Points
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
10
Some Important Definitions
1.
2.
3.
4.
5.
6.
7.
Grid Computing
Resource
Protocol
Network enabled service
Application Programmer Interface (API)
Software Development Kit (SDK)
Syntax
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
11
1. Grid Computing (1/4)

Early definition:


“We will probably see the spread of computer utilities, which, like present electric
and telephone utilities, will service individual homes and offices across the
country” (Len Kleinrock, 1969)
The Grid:

“A computational Grid is a hardware and software infrastructure that provides
dependable, consistent, pervasive and inexpensive access to high-end
comptational capabilities” (I.Foster, C.Kesselman: The Grid: Blueprint for a New
Computing Infrastructure”, 1998)

and: “because of the focus on dynamic cross-organization sharing, Grid
technologies complement rather than compete with esisting distributed
computing technologies” (I.Foster et al., The Anatomy of the Grid, 2001)
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
12
1. Grid computing: characteristics (2/4)

The three fundamental properties of Grid computing:
1.
Large-scale coordinated management of resources belonging to
different administrative domains (multi-domain vs single domain)
 Grid computing involves multiple management systems
2.
Standard, open, multi-purpose protocols and interfaces that provide a
range of services (standard vs proprietary)

3.
Grid computing supports heterogeneous user applications
Delivery of complex Quality of Service (QoS): Grid computing allows its
constituent resources to be used in a coodinated fashion to deliver
various types of QoS, such as respons time, throughput, avaiability,
reliability, security, etc.
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
13
1. Grid Computing (3/4)

Examples of non-Grid systems:

Cluster management systems on a parallel computer
or on a Local Area Network



Sun Grid Engine
Load Sharing Facility (LSF, by Platform)
Portable Bach System (PBS, by Veridian)
Complete kwowledge of system state and user’s requests  centralized control

The World Wide Web:


Open
Based on general-purpose protocols accessing distributed
resources
No coordinated use of independent resources (no protocols for negotiation and
sharing, yet)
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
14
Grid as a multi-institutional infrastructure
Intra-site resource
local credential
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
Inter-site
seamless collaboration
Virtual organization
Grid-level credential
15
The three main capabilities of a Grid
middleware

Virtualization of users
and resources

Mapping virtual
resources to physical
resources

Mapping virtual users
to physical users
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
Grid system
16
Site A
Site B
16
1. Grid Computing (4/4)

The importance of standardization:

The Grid:


A Grid:


open, general-purpose and using standard protocols
no standardization and interoperability between services –
current situation
Similarly: an Internet (based on proprietary protocols,
as in the early ages of networking) vs the Internet
(based on the IP protocol)
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
17
2. Resource

An entity that is to be shared


Does not have to be a physical entity


E.g., computers, storage, data, software
E.g., Condor pool, distributed file system, …
Defined in terms of interfaces, not devices


E.g. scheduler such as LSF and PBS define a
compute resource
Open/close/read/write define access to a distributed
file system, e.g. NFS, AFS, DFS
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
18
3. Grid Protocol

A formal description of message formats and a set of rules for message
exchange, which defines one of the basic mechanisms of Grid Computing



Protocols, some examples:







Rules may define sequence of message exchanges
Protocol may define state-change in end-points triggered by a given sequence of
exchanged messages, e.g., file system state change
Management of credentials and policies in case of multi-domain resources
Secure remote access
Co-allocation of multiple resources
Information query protocols
Data management protocols
Good protocols are designed to do one thing; for this reason, the Grid
architecture relies on layering of protocols. i.e. through the composition of
multiple, simple protocols.
Examples of protocols

IP, TCP, Transport Layer Security (was Secure Socket Layer), HTTP, Kerberos
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
19
4. Network Enabled Services

Services are defined by:



Examples:



the protocol spoken, as protocols allow interaction between different
services
the behaviour implemented.
Resource access service, Resource discovery, Co-scheduling, Data
replication, etc.
Services hide the complexity of resource implementations
Examples: FTP and Web servers
FTP Server
Web Server
FTP
Telnet
Protocol Protocol
HTTP Protocol
TCP Protocol
TCP Protocol
IP Protocol
IP Protocol
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
Transport Layer
Security Protocol
20
5. Application Programming Interface (API)

A specification for a set of routines to facilitate
application development

Refers to definition, not implementation (several
implementations of the same API are possible)
APIs are a complement of protocols, as without
protocols interoperability between APIs would be
solved only case by case with specific
implementations
 Specification of an API is often language-specific



Routine name, number, order and type of arguments;
mapping to language constructs
Behavior or function of routine
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
21
6. Software Development Kit

A particular instantiation of an API

Software Development Kits consist of libraries
and tools


Provides implementation of API specification
One API can have multiple SDKs
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
22
7. Syntax

Rules for encoding information, e.g.



Distinct from protocols


XML, Condor ClassAds, Globus Resource Specification
Language
X.509 certificate format (RFC 2459)
One syntax may be used by many protocols (e.g., XML) and
be useful for several purposes
Syntaxes may be layered


E.g., HTML  XML  ASCII
Important to understand layerings when comparing or
evaluating syntaxes
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
23
APIs and Protocols are Both Important

Standard APIs/SDKs are important



They enable application portability
But without standard protocols, interoperability is hard
(every SDK speaks every protocol?  infeasible)
Standard protocols are important



Enable cross-site interoperability
Enable shared infrastructure
But without standard APIs/SDKs, application portability
is hard (different platforms access protocols in different
ways)
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
24
A Protocol can have Multiple APIs
TCP/IP APIs: BSD sockets, Winsock, …
 The protocol provides interoperability: programs
using different APIs can exchange information
 I don’t need to know remote user’s API

Application
Application
WinSock API
Berkeley Sockets API
TCP/IP Protocol: Reliable byte streams
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 002a
25