The CMDB is the core component in creating agile infrastructures Richard Croucher

The CMDB is the core component in creating agile
infrastructures
Richard Croucher
London
July 9th, 2008
Smart Infrastructure Solutions
Citihub Ltd 32 Threadneedle Street, London EC2R 8AY
t + 44 (0)20 763 6070 f + 44 (0)20 763 6074
www.citihub.com
Citihub Inc 275 Madison Avenue, 4th floor
t + 1 212 878 8840 + 1 212 878 8839
Agile Infrastructure – why bother?
• Your Data Centre infrastructure is
increasing in cost
– To support Business growth
– Increasing energy costs
– Increasing complexity demands
expensive skills
• It takes too long to support new
business initiatives
• You can save $$millions
optimizing resource usage and
through automation
(c) – copyright Citihub 2008
2
Loadalancer/Firewall
Loadalancer/Firewall
Logical
Environments
Loadalancer/Firewall
Loadalancer/Firewall
Agile Infrastructure
Servers are allocated out of the
pool, the network and storage
devices are
Automatically configured to
create the desired environment.
Physical
Environment
3
(c) – copyright Citihub 2008
Why OS Virtualization is not enough
•
•
Operating System virtualization - reduced CAPEX , but increased
OPEX
–
More systems to manage (virtual + physical)
–
Increased complexity
–
New environments to manage, skill up, patch e.g. VMware ESX
How to reduce the management cost?
–
Too many humans in the loop, too much effort to •
•
•
Build out new systems, Create virtualized environments, load balance by tweaking the configurations
and migrating to new physical servers
Market data bursts, online services both suffer from sudden load
demands.
–
Open Loop demand v. Traditional enterprise Closed Loop demand applications
–
OS Virtualization only helps whilst there is spare capacity on the physical server
SOA and Internet Services - how to size?
–
Do you spend on “just in case sizing” or risk bad service if undersized
(c) – copyright Citihub 2008
4
Resizable executable partitions
Creates variable sized execution partitions
Each one executes a specific application or
environment.
Workload
Execution
Environment
Resources are allocated to partitions based
on Service Level Objectives
Workload
Execution
Environment
Partitions are isolated except for predefined
endpoints connections
Examples of partitions: Fixed Income Grid,
F/X Grid also for SOA application, Web
Services
Uncommitted
Capacity
Workload
Execution
Environment
Enables resources to be moved between
them, based on business needs
Typical time to provision is 5mins to 1 hour
Headroom of uncommitted capacity kept
available to deal with surges
Live examples in the
Online Services World
(c) – copyright Citihub 2008
5
Automation for Agile Infrastrcuture
Reporting, Logging, Trend Analysis
Workload
Execution
Environment
Uncommitted
Capacity
Workload
Execution
Environment
Virtual Resources
Vmware, XEN, VLAN, LUN, NAS
Physical Resources
Server, Storage, Network, InfiniBand
Configuration Management Database – Relationships, Discovered, Expected and Desired states
6
(c) – copyright Citihub 2008
HW
Prov
Orchestration
Workload
Execution
Environment
Effectors
Partition Level Policy
SW
Prov
Provisioning
Sensors - Monitoring, Observers
Drag and Drop deployment
1.
2.
3.
4.
Drag and drop from previously defined components and connect together to create a desired deployment state.
Workflow based approval process.
Automatically verify available resources.
Instantiates environment using Effectors to configure the infrastructure.
(c) – copyright Citihub 2008
7
Physical lifecycle
Demand
Specify
Requirement
Evaluate and
Test sample
SKU
Definition
Receiving
Purchasing
Order SKU
Add Item
Create type
Build out
Change
location
Retire
Remove
Item
Mark
Available
Yes
No
Pass?
Change
Owner
Change
Status
Soak
Testing
Allocated to
Business
Faulted
8
Page 8
(c) – copyright Citihub 2008
Physical Device States
Enables life cycle management
•
Pre-Rack – the device has been, or is about
to ordered, but it has not get been received.
Enables configuration to be pre-defined e.g.
intended location
•
In Build – the device has been received and
is currently being assembled, racked or
cabled. Communication to the device is
unstable.
•
In Test – the device is built and is ready for
or currently undergoing testing
•
Healthy – the device is in working order. It
is available for use, or may already have
been allocated.
•
Failed - Faults have been logged on this
device such that it is no available for use.
Partial failures, where a device could still be
used, will remain as healthy.
•
In Repair – the device is currently queued
for, or undergoing repair.
•
Disposal – the device is marked for
disposal. It should not be re-allocated or
tested.
9
Burn In
& sizing
CMDB
Supplier
RMA
(c) – copyright Citihub 2008
Pass?
No
Yes
Available
Resource Pool
Resource
Req/de-alloc.
Managed Elements Environment switch
•
Expresses how a ME is
currently being used
•
Acts as a switch for Alarm
routing and escalation
•
Permits allocation of physical
resources to either Test or
Production roles – facilitates
DR
– Assumes appropriate isolation
mechanisms are in place
•
Enables instantiation of PreProd environments on
Demand with subsequent
promotion to Prod following
successful Test.
(c) – copyright Citihub 2008
10
Service States
Describes the State of a Service, used with Monitoring and
Alarm systems
•
Off – The Service is turned off.
•
Standby – The Service is configured ready to provide service but is currently
blocked from receiving any requests. It is currently Out of Service (OOS)
•
Re-Configuration – the Service is currently out of service currently and is being
deployed or is undergoing re-configuration.
•
Running – it is currently in service
•
Failed – the service has failed. This may translate to difference significance
depending on the level, since a horizontally scalable Service can still be
considered Running, even though some it’s Instances may be in a Failed state.
The ServiceHealth is a way to determine this partial status, since large Services
will almost always have some level of underlying failure.
•
Unknown – this is used to represent that the current service state is unknown.
(c) – copyright Citihub 2008
11
Every Managed Entity needs to be :Observers
Observable - independent viewer, watchdog
and/or performance metrics
Monitor
Monitored – reports status and performance
Provision
Provisionable – deploy software and/or
firmware
Configure
Configurable via automation, Web service API
Discover
Discoverable – determine actual configuration
Managed
Entity
(Server, switch, device,
OS, Application)
12
(c) – copyright Citihub 2008
Configuration States
Alarms
Change request
Ought-ness
Desired State
CLM/SLM
Expected
State
reconcile
allocate
request
drift
Provision
Automation
+ Rules
Actual
State
Discovery
Build
Entity
is-ness
Monitoring
Alarms
13
(c) – copyright Citihub 2008
Page 13
CMDB level of detail
•
Physical Configuration
– Describe the physical devices
– All ports to enable full cabling plan to be defined
– Location vector (Building-Hall-Cage-Rack-Slot-Port)
– Device State – Pre-rack, Build, Test, Use, Available, Failed, Repair, Retirement
•
Logical Configuration
– An instantiated server and how they are interconnected
– Network endpoints - exposed and accessed
– Software package descriptions – configuration and personalization
•
Service Configuration
•
Organization
•
Health
– Environment – Test, Production
– Access permissions, escalation, responsibilities
–
Includes performance, Links back to incident management , often federated access to an
external system
+ Relationships
14
(c) – copyright Citihub 2008
Physical
Server
Example
15
(c) – copyright Citihub 2008
Logical Server example
(c) – copyright Citihub 2008
16
Logical based deployment descriptions
•Each ServerGroup has a description of the software packages necessary to deploy and its
resource requirements.
•Endpoints describe service access points, annotated with attributes to include LB and
Multicast properties.
•ReqConnections describe Endpoints it needs to be connected to.
•Described using a recognized modelling syntax
•Graphical tools to compose , XML, XSD and APIs for automation
(c) – copyright Citihub 2008
17
Model Based Automation
18
•
Model based automation enables the physical configuration to be derived from a
logical representation of the required configuration which is then instantiated through
standardised and well tested drivers to configure the devices and provision the
servers, storage and network.
•
It requires a robust description of the entire data centre infrastructure that is
accessible via rich APIs to automation tools. This can be provided by a
Configuration Management DB (CMDB).
•
A number of standards exist for modelling the DC
–
Common Information Model (CIM) defined by the DMTF
(http://www.dmtf.org/standards/cim/cim_schema_v216/)
–
Service Modelling Language (http://www.serviceml.org), draft w3c standard.
•
CIM is supported using WEBEM tooling. Heavily supported by Microsoft (Windows
Management Instrumentation (WMI) which is CMI based, Sun and Cisco.
•
Standardization activities are starting to converge around Service Modelling
Language (SML) promoted by BEA, BMC, Cisco, Dell, EMC, HP, IBM, Intel, Microsoft
and Sun. And now accepted by w3c as a draft standard. Potential to eventually
replace CIM. SML is a meta language for producing XML descriptions within
Component Model Libraries in a portable way that facilitates integration between
different products which have adopted SML. CML are described as XML Schema
Documents (XSD). Existing XML/XSD tooling can be utilized.
(c) – copyright Citihub 2008
CMDB Requirements for Agile infrastructures
• Desired, Expected and Discovered Configuration State
• Sufficiently detailed configuration data to enable full
provisioning of Servers, Network Switches and devices,
Storage Arrays etc.
• Physical Device States and Environment Switch
• Relationships between them to identify Service impact
and assist root cause correlation.
• Web services API’s to allow loosely coupled automation
– bi-directional for both import and export
(c) – copyright Citihub 2008
19
Agility maturity model
• Chargeback
• Cost recovery
• Software
provisioning
• Storage
virtualization
• Virtual machine
management
• OS virtualization
• Script-based
automation
• Server standards
• Paper policies
• Builds standards
• Asset
Management
• Configuration
discovery
• Configuration
management
• Bare-metal
provisioning
• VM Live
migration
• Manual
orchestration
• Configuration
enforcement
• Event-triggered
actions
• Proactive alarm
prevention
• Alarm
correlation
• Fine-grained
chargeback
• Rental
chargeback
• Multiple service
providers
• Policy engine
• Derived rules
• Declarative
rules
• Multiple DC
instances
• Workflow
orchestration
• Automated state
reconciliation
• Config. states
• Derived physical
configurations
• Model-based
deployment
ADVANCED
AUTOMATED
INTERMEDIATE
BASIC
FOUNDATIONAL
(c) – copyright Citihub 2008
20
About Citihub
Citihub is a specialist IT infrastructure consultancy for
the financial services industry
Our people are proven industry practitioners with an
appetite for complex challenges, particularly around
applications infrastructure
We'
re passionate about our clients'success; the
industry’s leading companies come back to us time
and again because we take ownership and we
deliver results
[email protected]
(c) – copyright Citihub 2008