Streamline SAP workload and speed data access with IBM/PBS NLS solutions

Streamline SAP workload and speed data
access with IBM/PBS NLS solutions
Tame the Data Demon!
Date 19th November 2015
Presented by: Dr. Klaus Zimmer, PBS Software and Elke Hartmann-Bakan, IBM
1
© 2015 IBM Corporation
DB2 Tech Talk series host and today’s presenter:
Jessica Rockwood, DB2 Tech Talk Host
Senior Manager, DB2 Systems of Record - OLTP
Today’s Technical Presenters . . .
Dr. Klaus Zimmer, PhD Physics
International Sales and Alliance management
PBS Software
E-mail: [email protected]
Elke Hartmann-Bakan
IBM Analytics Platform, Business Development, DB2 LUW for SAP
Phone: +49-7034 2747340
E-mail: [email protected]
A few details ….
1. The presentation for this Tech Talk:
Click here to download
© 2015 IBM Corporation
Disclaimer
The information contained in this presentation is provided for informational purposes only.
While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is
provided “as is”, without warranty of any kind, express or implied.
In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM
without notice.
IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any
other documentation.
Nothing contained in this presentation is intended to, or shall have the effect of:
• Creating any warranty or representation from IBM (or its affiliates or its or their suppliers and/or licensors); or
• Altering the terms and conditions of the applicable license agreement governing the use of IBM software.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment.
The actual throughput or performance that any user will experience will vary depending upon many factors, including
considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage
configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve
results similar to those stated here.
© 2015 IBM Corporation
Agenda
• PBS Software – Company and solutions portfolio
• Motivation
– Manage your data volume actively
• Architecture
– IBM DB2 with BLU acceleration
• Solution and Feedback – PBS CBW NLS solution for SAP BW
• Solution and Feedback – PBS add on solutions for SAP ERP
• Hints for project realization
• Next steps – Key things to take home
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
4
PBS Software GmbH
SAP add-on software development
Founded in 1991, privately owned
Worldwide 1500+ customers
5000+ PBS solutions installed
For nearly 25 years, PBS Software has been a leading provider
of SAP add-on solutions for data archiving, ILM, compliant
storage and system decommissioning
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
5
PBS Co-operations
SAP Software Solutions Partner since 1999
World-wide cooperation with several partner companies
who are specialized in ILM and SAP archiving projects
Cooperation with IBM for compliant storage since 2005
Enhanced cooperation with IBM since 2014: IBM DB2 with BLU Acceleration
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
6
PBS Software Solutions Portfolio
Nearline Storage Solutions
Add-on Solutions for SAP ERP
Seamless access to archive and database
Transactions, reporting, audits
Analytical and operational systems
Fast query access, big data processing
PBS
ILM
Solutions
Compliant Storage
ArchiveLink and WebDAV
Structured and unstructured data
Data Extraction / Decommissioning
Data extracts from archive and database
SAP independent data access
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
7
Motivation
• PBS Software – Company and solutions portfolio
• Motivation
– Manage your data volume actively
• Architecture
– IBM DB2 with BLU acceleration
• Solution and Feedback – PBS CBW NLS solution for SAP BW
• Solution and Feedback – PBS add on solutions for SAP ERP
• Hints for project realization
• Next steps – Key things to take home
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
8
Avoid the Data Tsunami
Without archiving
Transfer growth to low-cost archive
Large data compression > 1:5
Smaller footprint in online SAP DB
Single instance – no mirrors
Reduce DB costs > 60%
Continuous growth of the SAP database
Poor retrieval/system performance
TCO explosion, unstable system
Backup/DR? Audits? Compliance?
With archiving and PBS/IBM
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
9
ILM Project Goals : Tame the Data Demon
SAP
Database
SAP
Database
Archive / NLS
Control the data volume,growth rate and the retrieval performance
Enable complete and effective data analysis for end user queries
Meet legal requirements for analysis and audits
Cut the SAP operation costs!
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
10
Active Data Volume Management
SAP DB
Manage your data volume actively!
Control the data growth rate
Improve current hardware/resource usage
Keep your system managable
Limit the TCO increase
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
11
Robust Data Aging Strategy
Data Category
“HOT”
Frequent access,
updates/changes
“WARM”
Occasional access,
mature data
“COLD”
Rare access,
fix data
“Online” Database
Near-Line Storage
(Read Only)
Classic Archive
(Read Only)
Seamless user access to ALL data
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
12
IBM DB2 with BLU Acceleration
• PBS Software – Company and solutions portfolio
• Motivation
– Manage your data volume actively
• Architecture
– IBM DB2 with BLU acceleration
• Solution and Feedback – PBS CBW NLS solution for SAP BW
• Solution and Feedback – PBS add on solutions for SAP ERP
• Hints for project realization
• Next steps – Key things to take home
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
13
Columnar – Makes the Difference
• Column-Store
• Data is stored from top to bottom, column by column
• Row store – data is stored by tuple
• High Compression Rate
• Similar attribute values within a column
• No Indexes and Aggregates
• Self-indexing and uses less disk space when compared to an RDBMS
• Very Fast for analytical Queries
Tuple
Figure taken form Column-Oriented Database Systems, VLDB 2009 Tutorial; Stavros Harizopoulos, Daniel Abadi, Peter Boncz
14
DB2 with BLU Acceleration - Overview
• Hybrid in-memory technology
•
•
CPU-optimized unique runtime handling
•
Unique encoding for speed and compression
•
Columnar storage, vector processing (e.g. SIMD)
•
Built directly into the DB2 kernel
Evolution and Revolution together
•
BLU tables coexists with traditional row tables
•
Query any combination of row or BLU tables
•
Easy conversion of tables to BLU tables
•
Transparent for SQL and apps without changes
• Just „Load and Go“
•
Faster deployment and less ongoing management
•
Simple upgrade: Conversion from row-organized tables to BLU
Acceleration is easy
DB2 with BLU Acceleration
Runtime
Classic
DB2 runtime
BLU
runtime
Classic DMS
BLU DMS
(non-BLU tables)
(BLU tables)
Classic DB2 bufferpool
CPUs with SIMD
Storage
classic row
structured table
C1 C2 C3 C4 C5 C6 C7 C8
compressed, encoded
columnar
C1 C2 C3 C4 C5 C6 C7 C8
15
Summary Introduction to DB2 10.5 BLU
•Unmatched Innovation from IBM Research & Development Lab
Dynamic In-Memory
Actionable Compression
•
•
•
Allows processing without decompressing
•
Compression savings beyond DB2 10.1
In-memory columnar processing with
dynamic movement of data from storage
C1 C2 C3 C4 C5 C6 C7 C8
Encoded
•Instructions
•Data
Parallel Vector Processing
Data Skipping
•
•
Multi-core and SIMD parallelism
(Single Instruction Multiple Data)
Skips unnecessary processing of irrelevant data
•Results
16
Full usage of Unique DB2 Features for NLS
Faster Query Performance
Minimized HW Footprint
• DB2 10.5 with BLU Acceleration
• Compression rates in the range of 95%
• Support of SAP BW Flat Info Cubes (BW 7.40
SP8)
• Optimal resource utilization: less memory, less
IOPS, less disk space
Highly Automated and Secured
Low TCO and Fast ROI
• STMM – self tuning memory management
• DB2 NLS data can be kept on less expensive
hardware
• Real time statistics, online reorg …
• Data Encryption
• Customer can reuse his existing hardware, fully
virtualized
Mature and Robust Technology
• DB2 OEM license with all features in PBS CBW
NLS bundle
• HADR with Cluster Management (TSA*)
• Mature backup and restore process
Flexible design to adjust implementation based on customers requirements
17
*TSA=Tivoli Systems Automation for Multi Platforms
SAP BW Support on DB2 10.5
Supported BW objects
•
InfoCubes / Aggregates
•
Flat InfoCubes (BW 7.40 SP8 or
higher – SAP Note: 2155374)
•
DSO and PSA
•
InfoObjects
•
NLS InfoCubes and DSOs
•
BW temporary tables
Advantages
SAP BW 7.0 and higher
DB2 Near-Line Store for SAP BW
SAP Strategic Enterprise Management (SAP SEM)
•
Faster reporting on InfoCubes
•
Faster reporting on DSOs
•
Faster ETL processing
•
Significant space reduction
•
Less ETL processing steps
•
No index tuning required
18
NLS Performance Tests
Query 1
Query 2
CO-PCA-Cube
Smart Meter Cube
11,558 Mio records
29 Mio records
Runtime - NLS Query (sec)
0,76
Sybase IQ
Query 2
0,27
DB2 BLU
3 x faster
0,81
Sybase IQ
Query 1
DB2 BLU
0,58
28% faster
0
DB2 BLU:
Sybase IQ:
HW:
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
no performance tuning
was tuned for best performance
Same HW for both databases, Intel x86_64, 4 cores, 16 GB RAM, Suse-Linux
IBM Confidential
19
SAP BW NLS on DB2 – Backup
• Database backups of the NLS system can be performed in
less frequent intervals using different infrastructure (e.g.
Online Backup vs. Flash copy)
BW
database
NLS / DB2
database
tape
tape
tape
tape
tape
Flash
Copy
Flash
Copy
Flash
Copy
Flash
Copy
Flash
Copy
tape
tape
t
20
Solution and Benefits – 4BW
• PBS Software – Company and solutions portfolio
• Motivation
– Manage your data volume actively
• Architecture
– IBM DB2 with BLU acceleration
• Solution and Feedback – PBS CBW NLS solution for SAP BW
• Solution and Feedback – PBS add on solutions for SAP ERP
• Hints for project realization
• Next steps – Key things to take home
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
21
SAP Nearline Interface: Data Archiving Process - DAP
Data transfer via SAP NLS into the nearline database and/or ADK files
Data compression > 80 %, in many cases ≥ 90 %
Transparent process for reload and updates
CUBES
NLS
DB2 BLU
Fast data transfer
Huge compression
Fast retrieval
Archive
Large compression
Compliant storage
Long-term archiving
DAP
DSO
PSA
SAP BW DB
PBS CBW NLS specific enhancement: archiving/NLS option for PSA data
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
22
PBS CBW using IBM DB2 with BLU Acceleration
Query with simultaneous access to online database and NLS: OLAP
splits query in two parts retrieving the data from respective sources
NLS Database
DB2 BLU
BW Query
NLS and DB
InfoProvider
Cube - DSO
O
L
A
P
SAP
Nearline
Provider
PBS CBW
Nearline
Services
SAP BW
Database
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
23
CBW Query Results – User Perspective
CBW NLS + DB2 BLU
SAP BW database
InfoCube with 29 million records: 14 million in SAP db, 15 million in NLS
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
24
CBW Customer Feedback
SME customers (BW < 1 TB) as well as „big data“ companies (> 30 TB)
Typical CBW Compression
Typical CBW Runtime Improvement
SAP DB
NLS DB
200
2,000
> 90 %
Compression
seconds
Gigabyte
250
150
100
1,000
Retrieval typically 8-times faster!
50
0
2.8 TB
224 GB
0
Initial Query
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
Drilldown
Filter
25
Example: Company: 1 mio customers and 100 sales org.
Evaluate annual revenue by sales organization
Expected result set: List output with 100 rows
Standard NLS: read up to 1 mio. records (volume per customer),
aggregate by sales organization in the OLAP processor.
PBS Value Add: Copy MD attributes (here: sales organization) to
NLS, process the aggregation on the NLS database server
OLAP Engine
NLS
DB2 BLU
SAP BW
Database
Characteristics
Key figures
MD attributes
Navigation attributes
Hierarchies
FAST SLOW Value Add with PBS Master Data Snapshots
Characteristics
Key figures
PBS DSP – Data Snapshot Process
PBS enhancement
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
MD attributes
Navigation attrib.
Hierarchies
26
Query example with standard SAP NLS processing:
Large NLS result set, aggregation by SAP OLAP
Example runtime ~ 300 sec.
Query example with advanced PBS CBW master data snapshot
Small NLS result set, aggregation in NLS db (DB2 BLU)
Example runtime ~ 3 sec.
Query runtime
300
300 sec
FAST SLOW Value Add with PBS Snapshots – Typical Results
100 x Faster!
200
100
3 sec
0
Standard NLS
PBS improved NLS
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
27
Solution and Benefits – 4ERP
• PBS Software – Company and solutions portfolio
• Motivation
– Manage your data volume actively
• Architecture
– IBM DB2 with BLU acceleration
• Solution and Feedback – PBS CBW NLS solution for SAP BW
• Solution and Feedback – PBS add on solutions for SAP ERP
• Hints for project realization
• Next steps – Key things to take home
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
28
PBS NAI Nearline Solution for SAP ERP
PBS NAI
SAP ERP
SAP ADK
PBS
Archiving
&
Nearline
PBS add-ons
Retrieval
Application Modules
PBS archive add ons
PBS Index
Interface
Nearline Database
IBM DB2 BLU
ADK
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
ArchiveLink
WebDAV
PBS Index
SAP ERP
Database
Nearline DB:
PBS Index
AS Index (ZARIX)
Any SAP table
Compliant
Storage
ADK
29
Combine NLS + Classical Archiving for Effective Retrieval
Store archive index data in NLS db
Improve index structures (efficiency)
Achieve fast archive indexing
Enable seamless integration:
Display transactions of PBS add-ons
Huge archive data compression
Very fast data access, even for „big data“
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
30
NAI & PBS archive add ons: Customer Feedback
Very short index generation runtime
Significantly improved archive retrieval performance
Reduced administration efforts
Data Retrieval
Index Generation
400
ADK Indexing
NAI Indexing
5
x 7,5
x 13
Runtime (sec)
Runtime (h)
10
300
200
7min 2s
x 210
100
0
23s 3,5s
x 6,5
0
COPA
PCA
COPA
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
PCA
31
Integration of External Data into SAP
Example: PBS NAI CPOS for POS data
#1
PBS NAI
+
EDS
Data analysis
Reporting
Validation/audits
#2
...0111011
Legacy
POS data
Customer system:
85 billion line items /y
~ 85 TB /y
NAI
IBM DB2
SAP DB
3.6 TB/y
#n
Archive
Compliant
archiving
SAP ERP
Goal: Use legacy BIG DATA within the SAP infrastructure
Insert footer information such as internal use only (Insert>Header & Footer)
32
Project Realization
• PBS Software – Company and solutions portfolio
• Motivation
– Manage your data volume actively
• Architecture
– IBM DB2 with BLU acceleration
• Solution and Feedback – PBS CBW NLS solution for SAP BW
• Solution and Feedback – PBS add on solutions for SAP ERP
• Hints for project realization
• Next steps – Key things to take home
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
33
How a Sound Archiving/Nearline Project Could Look Like
Analyze the need for nearline/archiving, define sizing
approx.
2-3 days
Deploy software and NLS database
approx.
1-2 days
Customize NLS/archiving processes
- depends on data -
Generate test data - test all components involved
- week(s) -
Transfer to QA
approx.
Repeat and enhance test cases, mass data tests
- depends on data -
Transfer to production
approx.
Total project duration: typically from few weeks to several months
1-2 days
1 day
Projects performed by PBS partner network or IBM Services.
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
34
Hardware for NLS Options with IBM + PBS NLS
SAP BW Server
PBS CBW NLS Server
NLS with DB2 BLU
SAP
Gateway
PBS CBW NLS
Server Instance
CBW NLS
Minimum Test Hardware*
4-8 Core
CPU
32-64 GB
RAM
200 GB
HDD
CBW NLS supports:
IBM DB2 BLU
on AIX and Linux
NLS Database Instance
Minimum PROD Hardware*
4-8 Core
CPU
32-64 GB
RAM
200 GB+
HDD
*depending on used HW (Intel/Power) and performance requirements
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
35
Next Steps
• PBS Software – Company and solutions portfolio
• Motivation
– Manage your data volume actively
• Architecture
– IBM DB2 with BLU acceleration
• Solution and Feedback – PBS CBW NLS solution for SAP BW
• Solution and Feedback – PBS add on solutions for SAP ERP
• Hints for project realization
• Next steps – Key things to take home
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
36
Key Things to Take Home
• Reduce the data footprint and control the growth rate
• Manage your data volume actively!
• IBM/PBS archiving/NLS offering for SAP BW and SAP ERP
• High data compression rate: in average 90%
• Performance Improvement with PBS unique add-on solutions:
• BW query performance:
approximately 10x times faster
• ERP retrieval performance: approximately 10 to 100x times faster
IBM and PBS Nearline Storage Solutions for SAP BW and SAP ERP
37
DB2 Tech Talk: DB2 - The Right Database Choice
Next Steps Roadmap
Step One
Learn more about IBM DB2
Step Two
IBM DB2 for Linux, Unix and Windows for SAP:
http://www.ibm.com/solutions/sap/us/en/landing/J233701
A22235G06.html
DB2 with BLU Acceleration:
http://www.ibm.com/solutions/sap/us/en/landing/J23
3701A22235G06/db2sap.html
PBS Partner Network
https://www.pbs-software.com/en/licensing/index.html
For Reference
Call IBM to schedule a demo
or learn more
• 1 800 966-9875 (U.S)
• 1-888-746-7426 (Canada)
• 1800-425-3333 (India)
• Or visit http://www.ibm.com/planetwide/
for contact information worldwide
SAP Developer Network: DB2 for Linux, Unix, Windows
www.sdn.sap.com/irj/sdn/db6
R
Forum: www.sdn.sap.com/irj/sdn/forum?forumID=187
IBM Redbook: Architecting and Deploying DB2 with BLU
Acceleration in Your Analytical Environment:
www.redbooks.ibm.com/Redbooks.nsf/RedpieceAbstracts/s
g248212.html?Open
Deploying the IBM Banking Data Warehouse to IBM DB2
10.5 with BLU Acceleration:
www.ibm.com/developerworks/community/wikis/home?la
ng=en#!/wiki/Wc9a068d7f6a6_4434_aece_0d297ea80a
b1/page/Deploying%20the%20IBM%20Banking%20Data
%20Warehouse%20to%20IBM%20DB2%2010.5%20wit
h%20BLU%20Acceleration
38
© 2015 IBM Corporation
Upcoming Tech Talks
Don’t miss these in-depth DB2 feature talks!
Next DB2 Tech Talk:
DB2 pureScale in VMWare
December 17th 2015 at 12:30 PM EST
•Speaker: Enzo Cialini, IBM Senior Technical Staff Member,
Worldwide Core Database Technical Sales
•Registration: http://www.idug.org/p/cm/ld/fid=703
How to register :
DB2 Tech Talks web site
IDUG North America Tech Conference 2016
•May 24-27, 2016 | Austin, Texas
http://www.idug.org/p/cm/ld/fid=797
Dates and topics subject to change and modification.
39
© 2015 IBM Corporation
Find a local user group
chapter!
http://bit.ly/1z2RYcl
40
© 2015 IBM Corporation
Questions
Listening live? Submit questions at the top of this webcast
Listening in replay and have questions?
Email our DB2 Tech Talk Coordinator : Ammar Naji at [email protected]
41
© 2015 IBM Corporation
Thanks for attending!
Download this presentation on the right hand side of your window under the
“resources” widget.
42
© 2015 IBM Corporation