© Copyright 2015 EMC Corporation. All rights reserved. 1

© Copyright 2015 EMC Corporation. All rights reserved.
1
EMC BUSINESS DATA LAKE SOLUTIONS
SESSION 5101
-MOTI ASHKENAZI - PIVOTAL
© Copyright 2015 EMC Corporation. All rights reserved.
2
STEPS TO HARNESS BIG DATA
Build New Applications,
Products, & Business Models
Leverage New Analytics To
Predict The Future
Gather as Much Data
As Possible
© Copyright 2015 EMC Corporation. All rights reserved.
WHAT IS A DATA LAKE?
JUST COLLECTING AND ANALYZING DATA IS NOT A
DATA LAKE
• A Data Lake is a complex ecosystem of
tools
• Security isn’t secondary, it’s core
functionality
• Data is discoverable and its usage is
traceable
• It should provide access to all business
users
• It should follow business rules and policies
© Copyright 2015 EMC Corporation. All rights reserved.
DATA LAKES INCREASE DATA GRAVITY
TO HARNESS BIG DATA OPPORTUNITIES
KEEP APPS CLOSE TO DATA
Deliver data consistently & in a
standardized way
Compute intensive workloads with Intel
best in class performance and get at the
data quickly
Build views and applications for each
user
ANALYZE EVERYTHING
All of the data
More sophisticated analyses
New combinations and
correlations
EMC Solutions are Powered by Intel® Xeon ®
Processors
© Copyright 2015 EMC Corporation. All rights reserved.
ANALYTICS
APPS
KEEP DATA TOGETHER
Structured, unstructured, dark
Generated by the enterprise,
imported from outside
SPEED
Historic & real-time
DATA
EMC
BUSINESS
DATA LAKE
THE INDUSTRY’S FIRST
FULLY-ENGINEERED
ENTERPRISE-GRADE
DATA LAKE SOLUTION
© Copyright 2015 EMC Corporation. All rights reserved.
STORAGE FOR ANY WORKLOAD,
EVERY APPLICATION
Consolidate data silos into
one simple, scalable,
shared storage system
Highly scalable, easy to
deploy converged
infrastructure
Increased performance
for high-demand,
high-speed Hadoop
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
© Copyright
© Copyright
2015 EMC
2015
Corporation.
EMC Corporation.
All rights
Allreserved.
rights reserved.
STORAGE FOR ANY WORKLOAD,
EVERY APPLICATION
Consolidate data silos into
one simple, scalable,
shared storage system
Highly scalable, easy to
deploy converged
infrastructure
Increased performance
for high-demand,
high-speed Hadoop
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
© Copyright
© Copyright
2015 EMC
2015
Corporation.
EMC Corporation.
All rights
Allreserved.
rights reserved.
SIMPLER, FASTER HADOOP
GET STARTED QUICKLY
ANALYTICS TOOLBOX
HADOOP
OPEN DATA PLATFORM
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright
© Copyright
2015 EMC
2015
Corporation.
EMC Corporation.
All rights
Allreserved.
rights reserved.
COMPREHENSIVE ANALYTICS
GAIN RAPID INSIGHTS
ANALYTICS TOOLBOX
PIVOTAL HADOOP
BIG DATA SUITE
ADVANCED ANALYTICS
DATA PROCESSING
APPS AT SCALE
GREENPLUM DATABASE
OPEN DATA PLATFORM
SPRING XD
SPARK
HAWQ
REDIS
PIVOTAL HD
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
EXTENSIBLE ECOSYSTEM
FLEXIBLE AND SCALABLE
ANALYTICS TOOLBOX
PIVOTAL BIG DATA SUITE
(THIRD PARTY APPLICATIONS)
ADVANCED ANALYTICS
GREENPLUM DATABASE
APPS AT SCALE
HAWQ
REDIS
DATA PROCESSING
SPRING XD
SPARK
PIVOTAL HD
HADOOP
OPEN DATA PLATFORM
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
PLATFORM AT YOUR SERVICE
APPLICATION DEVELOPMENT, DEPLOYMENT AND OPERATIONS
ANALYTICS TOOLBOX
(THIRD PARTY APPLICATIONS)
PIVOTAL BIG DATA SUITE
ADVANCED ANALYTICS
APPS AT SCALE
GREENPLUM DATABASE
HAWQ
REDIS
DATA PROCESSING
SPRING XD
SPARK
PIVOTAL HD
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
HADOOP
OPEN DATA PLATFORM
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
PIVOTAL CLOUD FOUNDRY
THE RIGHT DATA AT THE RIGHT TIME…
INGEST, INDEX AND STORAGE – THE BEDROCK OF THE DATA LAKE
ANALYTICS TOOLBOX
DATA
MANAGER
(THIRD PARTY APPLICATIONS)
PIVOTAL BIG DATA SUITE
ADVANCED ANALYTICS
APPS AT SCALE
GREENPLUM DATABASE
HAWQ
REDIS
INGEST
DATA PROCESSING
SPRING XD
SPARK
PIVOTAL HD
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
HADOOP
INDEX &
SEARCH
OPEN DATA PLATFORM
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
PIVOTAL CLOUD FOUNDRY
…IN THE RIGHT WAY
IS THE DATA READY, RECOGNIZED, AND RELEVANT?
ANALYTICS TOOLBOX
DATA
MANAGER
DATA
GOVERNOR
(THIRD PARTY APPLICATIONS)
PIVOTAL BIG DATA SUITE
ADVANCED ANALYTICS
APPS AT SCALE
GREENPLUM DATABASE
HAWQ
REDIS
INGEST
DATA PROCESSING
SPRING XD
SPARK
PIVOTAL HD
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
POLICY
MGMT
HADOOP
INDEX &
SEARCH
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
SECURITY
& ACCESS
CONTROL
OPEN DATA PLATFORM
PIVOTAL CLOUD FOUNDRY
DEPLOYED IN AS LITTLE AS 1 WEEK
RAPIDLY MOVE FROM TINKERING TO MONETIZATION
DATA AND ANALYTICS CATALOG
MANAGEMENT
ANALYTICS TOOLBOX
DATA
MANAGER
DATA
GOVERNOR
(THIRD PARTY APPLICATIONS)
PIVOTAL BIG DATA SUITE
ADVANCED ANALYTICS
APPS AT SCALE
GREENPLUM DATABASE
HAWQ
REDIS
INGEST
DATA PROCESSING
SPRING XD
SPARK
PIVOTAL HD
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
POLICY
MGMT
HADOOP
INDEX &
SEARCH
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
SECURITY
& ACCESS
CONTROL
OPEN DATA PLATFORM
PIVOTAL CLOUD FOUNDRY
SELF-SERVE DATA SERVICES
PROVIDE ANALYTICS SERVICES NOT JUST ACCESS TO DATA
DATA SERVICES
DATA AND ANALYTICS CATALOG
MANAGEMENT
ANALYTICS TOOLBOX
DATA
MANAGER
DATA
GOVERNOR
(THIRD PARTY APPLICATIONS)
PIVOTAL BIG DATA SUITE
ADVANCED ANALYTICS
APPS AT SCALE
GREENPLUM DATABASE
HAWQ
REDIS
INGEST
DATA PROCESSING
SPRING XD
SPARK
PIVOTAL HD
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
POLICY
MGMT
HADOOP
INDEX &
SEARCH
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
SECURITY
& ACCESS
CONTROL
OPEN DATA PLATFORM
PIVOTAL CLOUD FOUNDRY
CUSTOMER CHOICE
FLEXIBILITY IN SELECTION TO MEET NEEDS
DATA SERVICES
DATA & ANALYTICS CATALOG
MANAGEMENT
ANALYTICS TOOLBOX
DATA
MANAGER
DATA
GOVERNOR
(THIRD PARTY APPLICATIONS)
PIVOTAL BIG DATA SUITE
ADVANCED ANALYTICS
APPS AT SCALE
GREENPLUM DATABASE
HAWQ
REDIS
INGEST
DATA PROCESSING
SPRING XD
SPARK
PIVOTAL HD
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
POLICY
MGMT
HADOOP
INDEX &
SEARCH
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
SECURITY
& ACCESS
CONTROL
OPEN DATA PLATFORM
PIVOTAL CLOUD FOUNDRY
THE EMC BUSINESS DATA LAKE PLATFORM
DATA SCIENCE WITH OUT A DATA SCIENTIST
DATA & ANALYTICS CATALOG
DATA SERVICES
MANAGEMENT
ANALYTICS TOOLBOX
DATA
MANAGER
DATA
GOVERNOR
(THIRD PARTY APPLICATIONS)
PIVOTAL BIG DATA SUITE
APPS AT SCALE
ADVANCED ANALYTICS
GREENPLUM DATABASE
HAWQ
REDIS
INGEST
DATA PROCESSING
SPRING XD
SPARK
PIVOTAL HD
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
POLICY
MGMT
HADOOP
INDEX &
SEARCH
SECURITY
& ACCESS
CONTROL
OPEN DATA PLATFORM
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
PIVOTAL CLOUD FOUNDRY
© Copyright 2015 EMC Corporation. All rights reserved.
FOCUS ON BUSINESS OUTCOMES,
NOT ON LOW LEVEL TECHNOLOGY DECISIONS
INGEST
ANALYZE
Capture data from
a wide range of
sources, traditional
and new
Use advanced
algorithms to discover
new, predictive
patterns
STORE
Store everything in
one environment for
cross data analysis
© Copyright 2015 EMC Corporation. All rights reserved.
ACT
Build data-driven
applications to meet
business needs
SURFACE
Share insights
with business
domain experts
START WITH THE END IN MIND
INGEST AND STORAGE SHOULD SUPPORT THE WORK
YOU WANT TO DO LATER
© Copyright 2015 EMC Corporation. All rights reserved.
HOW DO WE ASK THE RIGHT QUESTIONS?
THE REAL CHALLENGE OF ANALYTICS
© Copyright 2015 EMC Corporation. All rights reserved.
DON’T TRY AND “FIX ON THE FLY”
ANALYTICS ENVIRONMENTS NEED TO BE SEAMLESS AND SCALABLE
© Copyright 2015 EMC Corporation. All rights reserved.
THE NEED FOR SOMETHING MORE
ARE THE RESULTS RELIABLE, REPEATABLE, AND
REDISTRIBUTABLE?
•
•
•
Can we reuse these insights?
How do we move towards monetizing the insights
we’ve developed (Data as a Service)?
How do we keep up with the data
and the constant pace of change?
Data
Monetization
Business
Monitoring
© Copyright 2015 EMC Corporation. All rights reserved.
Business
Insights
Business
Optimization
Business
Metamorphosis
AVOIDING A DATA SWAMP
HOW TO STAY OUT OF THE MUD AND KEEP INSIGHTS FLOWING
Keep data relevant and reliable
Make insights and results repeatable and reusable
Provide a service-based approach to data
Policy-driven data storage and retention
Content and meta-data indexing for all data types
© Copyright 2015 EMC Corporation. All rights reserved.
EMC BUSINESS DATA LAKE PLATFORM
DATA & ANALYTICS CATALOG
DATA SERVICES
MANAGEMENT
ANALYTICS TOOLBOX
DATA
MANAGER
DATA
GOVERNOR
(THIRD PARTY APPLICATIONS)
PIVOTAL BIG DATA SUITE
APPS AT SCALE
ADVANCED ANALYTICS
GREENPLUM DATABASE
HAWQ
REDIS
INGEST
DATA PROCESSING
SPRING XD
SPARK
PIVOTAL HD
RABBITMQ
GEMFIRE
BDS ON
PIVOTAL
POLICY
MGMT
HADOOP
INDEX &
SEARCH
SECURITY
& ACCESS
CONTROL
OPEN DATA PLATFORM
VIRTUALIZATION
EMC II STORAGE
DATA LAKE FOUNDATION: ISILON | ECS
VCE VBLOCK | XTREMIO
EMC Solutions are Powered by
Intel® Xeon ® Processors
© Copyright 2015 EMC Corporation. All rights reserved.
PIVOTAL CLOUD FOUNDRY
WHERE ARE YOU?
Let EMC Help You Discover
YOUR KILLER USE-CASE
with a Big Data Vision Workshop
Work with EMC and Your Team to
IMPROVE YOUR BUSINESS
with a 8-12 Week Proof-of-Value Project
Order and
START TODAY
by Deploying the EMC Business Data Lake
© Copyright 2015 EMC Corporation. All rights reserved.
© Copyright 2015 EMC Corporation. All rights reserved.