n IBM Tivoli Network Manager with DB2 Best Practices v1.1
by user
Comments
Transcript
n IBM Tivoli Network Manager with DB2 Best Practices v1.1
Network Manager 3.9, 4.1, 4.1.1 IBM Tivoli Network Manager with DB2 Best Practices v1.1 n Note: Before using this information and the product it supports, read the information in “Notices” on page 33. Contents About this Guide...................................................................................................iii Chapter 1: How Network Manager uses the Database........................................1 Discovery.......................................................................................................................................................1 Poller...............................................................................................................................................................1 GUI..................................................................................................................................................................2 Chapter 2: Network Manager Configuration for DB2 Database.........................3 Network Manager Backend Processes...........................................................................................................3 Network Manager GUI and TCR Reports.....................................................................................................3 DB2 User Accounts...........................................................................................................................................4 How to configure password changes.........................................................................................................4 Chapter 3: IBM DB2 Configuration Settings for Network Manager....................9 Application Connection Problems..................................................................................................................9 Transaction Log Size Packages........................................................................................................................9 Configuration Settings...................................................................................................................................10 Registry keys................................................................................................................................................10 Database instance settings..........................................................................................................................11 Database Manager Settings........................................................................................................................12 DB2 Autoconfigure Option........................................................................................................................12 Database Maintenance...................................................................................................................................14 Chapter 4: Troubleshooting................................................................................17 Troubleshooting Tips.....................................................................................................................................17 Buffer Pool....................................................................................................................................................17 DB2 commands useful in troubleshooting..................................................................................................18 Diagnostic queries..........................................................................................................................................19 Appendix A: Using the Network Manager GUI to set the Cognos datasource passwords............................................................................................................23 Appendix B: DB2 deprecates System Managed Spaces for permanent tablespaces..........................................................................................................27 What does this mean for Network Manager?.............................................................................................27 Appendix C: Performance Analyst Tool.............................................................29 Appendix D: References....................................................................................31 Notices..................................................................................................................33 Trademarks......................................................................................................................................................35 This edition applies to 3.9, 4.1,4.1.1 of IBM Tivoli Network Manager and to all subsequent releases and modifications until otherwise indicated in new editions. © Copyright IBM Corporation 2013. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. About this Guide The database is central to the functionality of IBM Tivoli Network Manager IP. It stores all the data discovered and processed by the discovery engine to build the network topology views, historical data collected by the poller, as well as reference and operational data used by the Network Manager processes. Network Manager 3.9 supports the use of DB2, Oracle, MySQL, and Informix databases. We will focus on DB2 (9.7,10.1,10.5) specifics in this article, although most of the information referring to the use of databases with Network Manager applies to all databases. This document is intended to help you tune and maintain the DB2 database. In many cases the default values for DB2 are sufficient and very little attention is required. For customers with large environments it is common to use the services of a central Database Administration, and that can affect performance as the DB administrators attempt to optimize their resources by sharing the DB2 server with other applications. Knowing how Network Manager uses the database can help avoid heavy traffic problems. Using the information in this article, you can optimally set up DB2 with Network Manager to maintain optimum performance levels. This article is written for administrators responsible for maintaining Network Manager and assumes some basic familiarity with Network Manager and DB2. See Appendix D: References for links to information on Network Manager. © Copyright IBM Corp. 2013 iii Chapter 1: How Network Manager uses the Database Discovery At the end of the final phase of discovery (building the relationships and connections), the Model process merges the new topology into the NCIM tables, followed by the Poller process reacting to these changes to adjust the active policy scopes. This final phase of discovery is an intensive period that can take 30 minutes or more to complete as the Model and Poller processes deal with hundreds of thousands of network entities. After discovery, the Model process starts a RUNSTATS operation on the NCIM database to rebuild the statistics and this can have an impact on the database performance while it is running. This applies to deployments using Oracle or Informix databases for Network Manager 3.9 and was added for DB2 for Network Manager 4.1. The total amount of discovered data is generally not an issue for the DB2 database. However, as you increase the number of domains and entities, beyond say five domains, or if you are storing a lot of historical polled data, you will need to pay attention to the performance of the DB2 database. Poller When each poller starts up, and at the end of discovery, there occurs many minutes of intensive database queries and updates on the NCIM, NCMONITOR, and NCPOLLDATA tables. If configured, the poller stores historical polled data in NCPOLLDATA and this can put a steady, heavy load on the database if large numbers of data points are collected and stored at high frequency. The poller runs a pruning service every hour to maintain the most recent data points to a configured limit – by default, 5 million. Refer to the About Historical Polled Data Collection documentation especially if you plan on storing a lot of data. See: http://www.ibm.com/support/knowledgecenter/SSSHRK_4.1.1/com.ibm.netw orkmanagerip.doc_4.1.1/itnm/ip/wip/common/welcome.html Under normal circumstances the pruning takes only a few minutes, but the time can be affected by poor database performance caused by inefficient transaction log maintenance. It is possible to place the NCPOLLDATA schema on a separate supported database instance or server, but there are advantages and disadvantages to consider. On the plus side, you might want to consolidate your reporting services with a dedicated database, but if the database is located remotely, you have to balance that against the possible impact of a slower network link and additional architectural complexity. © Copyright IBM Corp. 2013 1 GUI The GUI has a short but intense period as the view caches are built as a user first opens each Network View. The dynamic update engine runs periodically (default 5 minutes) to keep the dynamic views and caches up to date. This is not usually intense, but in extreme cases can involve locking conflicts during peak periods with NCPGUI and NCIM schemas. The more concurrent GUI users with network views that are open, the more load on the database. Recent refactoring of the Network View caching for Network Manager 3.9 FP3 has improved performance and lightened the load on the NCP GUI and NCIM schemas for large topology databases considerably, both for large network views and the number of active users. 2 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 Chapter 2: Network Manager Configuration for DB2 Database This chapter explains how and where each Network Manager component’s database connection details are configured. This helps you confirm the details of the DB2 instance, the database name within the instance, and the user name specified. Network Manager Backend Processes The following files hold all the database connection data for the different schemas used by the Network Manager backend processes and many Perl scripts. $NCHOME/etc/precision/DbLogins.<domain>.cfg $NCHOME/etc/precision/MibDbLogin.<domain>.cfg (NCMIB schema only) These files contain the following information that you will need when accessing the database or changing its settings: • Database name: The name that the administrator specified when creating the database instance for Network Manager. An administrator creates the database, before installing Network Manager, using the create_db2_database.sh (UNIX) or create_db2_database.bat (WINDOWS) script. • Schema: The following schemas are created in the database: NCIM, NCPGUI, NCMONITOR, NCPOLLDATA, and NCMIB. Note that there is a separate record for each schema in these files. All the schemas are expected to be co-located, except NCPOLLDATA, which can be in a separate database. • Username: Specifies the account used for read-write access to the database instance. By default, the same account is configured for all the schemas. • Password: Specifies the password for the user associated with Username. By default, the password is encrypted. You can use the ncp_crypt utility to handle encryption/decryption of this password. Network Manager GUI and TCR Reports The Network Manager GUI reads the same database configuration information as the backend processes from these two files: $NCHOME/precision/profiles/TIPProfile/etc/tnm/tnm.properties $NCHOME/precision/profiles/TIPProfile/etc/tnm/ncpolldata.properties Note that the passwords in these files, when encrypted, use a different format from those in DbLogins and cannot be copied from DbLogins, nor can they be encrypted/decrypted using the ncp_crypt utility. The best way to set or change these passwords is by using the Network Manager GUI. See DB2 User Accounts The database connections for the Tivoli Common Reporting (TCR) reports are set up separately for BIRT and Cognos reports. The TCR trcmd.sh utility is used to configure TCR. For convenience, the Network Manager configureTCR.sh script uses this utility to deploy the Network Manager BIRT © Copyright IBM Corp. 2013 3 and Cognos reports, as well as configuring the data sources for the DB2 database. DB2 User Accounts The database accounts are created and maintained using the operating system. If your servers have password regulations that enforce periodic changes or can expire, then you must also update Network Manager with the new passwords when they change. When the database passwords expire, the Network Manager processes will fail when they try to open new connections and this can catch Network Manager administrators off guard. How to configure password changes The following sections explain how to configure the passwords for the Network Manager backend, Network Manager GUI, TCR Reports, and Network Manager Topology API. Note: The Topology API is only used by the IBM Tivoli Network Configuration Manager when integrated with Network Manager. Configuring the passwords for the Network Manager Backend Follow these steps to configure the password for the Network Manager Backend. The DbLogins file (in step 2) has a separate record for each database schema. Note the username defined for the ncim schema – the new password should be for this account. Normally the same account is used for all the schemas, and you can replace the same password in each of these records. 1. For the database account name being used, encrypt the new password using the ncp_crypt utility, for example, if the new password is netcool: > ncp_crypt –password netcool '@44:O2aJesKCAX6Af0aOKeTBMRWk2ru8soEOe9PFEv6smwc=@' 2. netcool Cut and paste the encrypted string (without the single quotes) into the following files for each schema record using this database account name: $NCHOME/etc/precision/DbLogins.<domain>.cfg $NCHOME/etc/precision/MibDbLogin.<domain>.cfg 3. Run the following command to verify that the connection information is valid and working for each schema: ncp_perl $NCHOME/precision/scripts/perl/scripts/ncp_db_access.pl –domain <domainname> 4. Restart the ncp processes by executing the following commands: itnm_stop ncp itnm_start ncp 4 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 Configuring the passwords for the Network Manager GUI Use the Network Manager GUI to change the password for the database account. The GUI will handle encryption, if configured, automatically. Select the following task: Administration->Network->Database Access Configuration Change the password on both NCIM and Historical Polling panels and press the Save button for each. Verify the database connection by opening a Network View. Database connection errors will appear in the ncp_topoviz log in: $NCHOME/precision/profiles/TIPProfile/logs/tnm/ Configuring the passwords for the TCR Reports If you are not using Tivoli Data Warehouse for the historical polled data, then you can simply use the configTCR.sh script located here: $NCHOME/precision/products/tnm/bin/configTCR.sh Caution: Network Manager 3.9 Fix Pack 2 or 3 If you do not want to reinstall the default package of BIRT and Cognos reports, then you must edit the configTCR.sh file and comment out the following lines: Report "Configuring ITNM Reports." # cd ${TCR_COMPONENT_HOME}/bin 1>/dev/null 2>&1 ${TCR_COMPONENT_HOME}/bin/trcmd.sh -import -bulk $ {ITNMHOME}/products/tnm/itnmreports.zip -username $ {IAGLOBAL_WASUserID} -password "$ {IALOCAL_WASPassword_XML_FRIENDLY}" 1>>${LOGFILE} 2>&1 # ${TCR_COMPONENT_HOME}/bin/trcmd.sh -import -bulk $ {ITNMHOME}/products/tnm/itnmcognos.zip -username $ {IAGLOBAL_WASUserID} -password "$ {IALOCAL_WASPassword_XML_FRIENDLY}" 1>>${LOGFILE} 2>&1 For later versions of Network Manager, unless you add the new command line option to install the reports, this command will only set the passwords. Run the following script to modify all the data sources for the BIRT and Cognos reports: ./configTCR.sh –p tipadmin_password –d new_db_password However, if you are using Tivoli Data Warehouse, you must use the TCR trcmd.sh utility to set the passwords for the data sources separately. Note: Executing the configTCR.sh script resets the connection details to the local Chapter 2: Network Manager Configuration for DB2 Database 5 database for the NCPOLLDATA and PARAMETER data sources. To recover, you must use the trcmd.sh script for these two data sources, with the appropriate connection values, in the BIRT and Cognos reports sections below. If you choose not to use the configTCR.sh script, see the BIRT and Cognos sections below, which describe how to use the TCR script (trcmd.sh) to set the database readonly account and password for the BIRT and Cognos reports. BIRT Reports To set the account name and password, run the trcmd utility once for each of the data sources: NCIM, NCPOLLDATA, and PARAMETERS. The trcmd.sh script is located here: $TIPHOME/../tipv2Components/TCRComponent/bin/trcmd.sh For example, using the account name db2inst and password netcool, run each of these commands: ./trcmd.sh -modify -dataSources -reports -username tipadmin -password tipadmin_password -dataSource name=NCIM -setDatasource odaUser=db2inst odaPassword=netcool ./trcmd.sh -modify -dataSources -reports -username tipadmin -password tipadmin_password -dataSource name=PARAMETERS -setDatasource odaUser=db2inst odaPassword=netcool ./trcmd.sh -modify -dataSources -reports -username tipadmin -password tipadmin_password -dataSource name=NCPOLLDATA -setDatasource odaUser=db2inst odaPassword=netcool If you are using Tivoli Data Warehouse, run the first command for the NCIM data source against the local DB2 database and the other two commands for the PARAMETERS and NCPOLLDATA data sources to change the password for the Tivoli Data Warehouse database account. For more information, see http://www.ibm.com/support/knowledgecenter/SSSHRK_4.1.1/com.ibm.networkmanage rip.doc_4.1.1/itnm/ip/wip/admin/task/nmip_adm_configuringdatasourcesbirt.html Cognos reports To set the account name and password, run the trcmd utility once for each of the data sources: NCIM, NCMONITOR, NCPGUI, NCPOLLDATA, and PARAMETERS. The trcmd.sh script is located here: $TIPHOME/../tipv2Components/TCRComponent/bin/trcmd.sh For example, if the database account is db2inst and the new password is netcool, the trcmd.sh script would be executed as follows: 6 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 ./trcmd.sh –dataSource –add NCIM –dbType DB2 –dbName NCIM -user tipadmin -password tipadmin_password -openSessionSql "SET CURRENT SCHEMA = NCIM" –dbLogin db2inst –dbPassword netcool –force ./trcmd.sh –dataSource –add NCMONITOR –dbType DB2 –dbName NCIM -user tipadmin -password tipadmin_password –openSessionSql "SET CURRENT SCHEMA = NCMONITOR" –dbLogin db2inst –dbPassword netcool -force ./trcmd.sh –dataSource –add NCPGUI –dbType DB2 –dbName NCIM -user tipadmin -password tipadmin_password –openSessionSql "SET CURRENT SCHEMA = NCPGUI" –dbLogin db2inst –dbPassword netcool -force ./trcmd.sh –dataSource –add NCPOLLDATA –dbType DB2 –dbName NCIM -user tipadmin -password tipadmin_password –openSessionSql "SET CURRENT SCHEMA = NCPOLLDATA" –dbLogin db2inst –dbPassword netcool -force ./trcmd.sh –dataSource –add PARAMETERS –dbType DB2 –dbName NCIM -user tipadmin -password tipadmin_password –openSessionSql "SET CURRENT SCHEMA = NCPOLLDATA" –dbLogin db2inst –dbPassword netcool -force If using Tivoli Data Warehouse, the database details must match the TDW database only for the NCPOLLDATA and PARAMETERS data sources. The Cognos administration in the Network Manager GUI can be confusing. See the appendix in this article for step-by-step instructions on how to change the password from the Network Manager GUI. The Network Manager GUI also offers the ability to test the database connection for Cognos. Otherwise, the only way to verify that the Cognos connection to the database is set up correctly is to run a report. Configuring the passwords for the Network Manager topology API Note: The Network Manager topology API is used only when Network Configuration Manager is integrated with Network Manager. The API connects to the Network Manager NCIM database through a datasource that needs to be refreshed when the Network Manager database password changes. Follow these steps to configure the password for datasource used by the Network Manager topology API. 1. Run the following command to refresh the password used by the API after you have changed the password from the Network Manager GUI as described above. $NCHOME/precision/bin/createITNMDatasource.sh <tip_administrator_username> <tip_administrator_password> This script reads the new password value from $NCHOME/precision/profiles/TIPProfile/etc/tnm/tnm.properties and automatically updates the data source. As a reference, full instructions for integrating NCM and ITNM can be found at the following link http://www.ibm.com/support/knowledgecenter/SS7UH9_6.3.0/com.ibm.netc Chapter 2: Network Manager Configuration for DB2 Database 7 ool_configurationmgr.doc_6.3.0/ncm/wip/int/concept/ncm_int_integrating.ht ml 2. Restart TIP by executing this command: itnm_start tip 3. Test the datasource after the change by executing the following command: $TIPHOME/bin/wsadmin.sh -lang jython -username tipadmin -password <TIP ADMIN Password> -f $NCHOME/precision/bin/createITNMDatasource.jy -test The output should contain the following: WASX7217I: Connection to provided datasource was successful. 8 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 Chapter 3: IBM DB2 Configuration Settings for Network Manager The DB2 default configuration settings are usually adequate for a standard DB2 9.7, 10.1, or 10.5 deployment, with the default settings made by Network Manager when creating the database instance. However, if you have several domains, or store a lot of historical polled data, or share the database server with other products, then you may need to be more aware of the database performance. If you upgraded your DB2 database, it may still have some of the previous settings, so it would be wise to check your settings according to the suggestions below. This chapter assumes you are familiar with issuing DB2 commands. When you use the create_db2_database.sh script to create the Network Manager database, it also applies a set of configuration settings. Additional DB2 configuration settings were added for Network Manager 4.1. Before looking at the general recommendations for the database configuration settings, let's consider two common problems with Network Manager associated with application connections and transaction log size. Application Connection Problems By default, Network Manager 3.9 sets the number of application connections permitted to 100. This is usually sufficient for up to about five domains. As you increase the number of domains, you may see database connection fail errors (SQLSTATE=57030) exceeding the number of application connections allowed. To avoid database connection fail errors, set MAXAPPLS to automatic, as in the following example: db2 update db cfg for ITNM using MAXAPPLS AUTOMATIC Note: Network Manager 4.1.1 sets MAXAPPLS to auto by default now. If the DB2 server is shared with other products, you might consider increasing the number of connections to 200 for the Network Manager database instance, as in the following example: db2 update db cfg for ITNM using MAXAPPLS 200 In either case, restart the database instance for the changes to take effect. © Copyright IBM Corp. 2013 9 Transaction Log Size Packages If the transaction log size is insufficient, you may see errors during periods of many uncommitted row changes such as: The transaction log for the database is full. SQLSTATE=57011 For a tech note on this, see http://www-01.ibm.com/support/docview.wss?uid=swg21635447 By default, DB2 sets the logging to be circular (logarchmeth1=OFF, logarchmeth2=OFF). With this method, you will need to either increase the log file size or the number of secondary log files available. The following DB2 commands provide suggested settings for the different log parameters: db2 db2 db2 db2 update update update update db db db db cfg cfg cfg cfg for for for for <dbname> <dbname> <dbname> <dbname> using using using using LOGBUFSZ 2150 LOGFILSIZ 5000 LOGPRIMARY 10 LOGSECOND 20 The performance of the database can be significantly affected if the transaction log files are switching too frequently – generally, aim for a few per hour. You can monitor the current active log id over a period of time by periodically running the following command: db2 "Select MEMBER, CUR_COMMIT_DISK_LOG_READS, CURRENT_ACTIVE_LOG from table(mon_get_transaction_log(-1)) as t order by member asc" The previous command gives you an idea of how often the active log id is changing. Configuration Settings The following sections have been taken from the Best Practices for System Performance for IBM Tivoli Service Management Products for settings that have been found useful for Network Manager. https://www.ibm.com/developerworks/community/blogs/a9ba1efe-b731-43179724a181d6155e3a/entry/new_best_practices_for_system_performance_7_5x_whi te_paper9?lang=en Registry keys Set the following registry keys individually. Using the database instance owner account (or one with administrative privileges) run the db2set commands: db2set DB2_SKIPINSERTED=ON db2set DB2_SKIPDELETED=ON db2set DB2_EVALUNCOMMITTED=YES 10 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 db2set db2set db2set db2set DB2_USE_ALTERNATE_PAGE_CLEANING=ON DB2_INLIST_TO_NLJN=YES DB2_MINIMIZE_LISTPREFETCH=YES DB2_FMP_COMM_HEAPSZ=65536 Database instance settings For optimal performance, use the following DB2 database configuration settings: db2 db2 db2 db2 db2 db2 update update update update update update db db db db db db cfg cfg cfg cfg cfg cfg for for for for for for <dbname> <dbname> <dbname> <dbname> <dbname> <dbname> using using using using using using DFT_QUERYOPT 5 (default) LOGBUFSZ 2150 LOGFILSIZ 5000 LOGPRIMARY 10 LOGSECOND 20 LOCKLIST AUTOMATIC (default) db2 db2 db2 db2 db2 db2 db2 update update update update update update update db db db db db db db cfg cfg cfg cfg cfg cfg cfg for for for for for for for <dbname> <dbname> <dbname> <dbname> <dbname> <dbname> <dbname> using using using using using using using LOCKTIMEOUT 300 MAXFILOP 61440 (UNIX / Linux 64bit) MAXFILOP 65535 (Windows 64bit) NUM_IOCLEANERS AUTOMATIC (default) NUM_IOSERVERS AUTOMATIC (default) SOFTMAX 1000 STMTHEAP 20000 db2 db2 db2 db2 update update update update db db db db cfg cfg cfg cfg for for for for <dbname> <dbname> <dbname> <dbname> using using using using CUR_COMMIT ON AUTO_REVAL DEFERRED DEC_TO_CHAR_FMT NEW STMT_CONC LITERALS (default) (default) Note that while the recommended setting for SOFTMAX has a positive impact on runtime performance, crash recovery time will increase. If you use DB2 HADR, this would not be cause for concern. In addition, if you use online backup, there may be a spike in I/O while data is being flushed to disk when the backup starts. While these settings are generally the default settings in DB2 9.7 and higher, you must ensure that they are set, especially if you upgraded from a previous version of DB2 to DB2 9.7. Note that when using DB2 9.7 with the statement concentrator (STMT_CONC), IBM recommends applying DB2 fixpack level 5 or later to avoid a potential performance issue with the statement concentrator. Use the following database configuration settings to take advantage of the self-tuning aspects of the database, and to reduce the overall management cost for the installation. You must have enough system resources for automatic tuning to work properly. If you are constrained on system resources, tune the settings manually: db2 db2 db2 db2 update update update update db db db db cfg cfg cfg cfg for for for for <dbname> <dbname> <dbname> <dbname> using using using using DATABASE_MEMORY AUTOMATIC PCKCACHESZ AUTOMATIC DBHEAP AUTOMATIC STAT_HEAP_SZ AUTOMATIC In some instances, setting PCKCACHESZ AUTOMATIC can result in an overly Chapter 3: IBM DB2 Configuration Settings for Network Manager 11 large package cache that can have adverse effects on performance. If you suspect that you are experiencing this problem, set PCKCACHESZ=524288. Database Manager Settings For optimal performance, use the following DB2 database manager settings: db2 db2 db2 db2 update update update update dbm dbm dbm dbm cfg cfg cfg cfg using using using using AGENT_STACK_SZ 1024 (UNIX / Linux default) AGENT_STACK_SZ 1000 (Windows) RQRIOBLK 65535 HEALTH_MON OFF (not applicable for DB2 10.1) Set the following database manager configuration parameter to avoid potentially encountering a DB2 heap exception (com.ibm.db2.jcc.c.SqlException: Not enough storage is available in the MON_HEAP_SZ heap to process the statement): db2 update dbm cfg using MON_HEAP_SZ AUTOMATIC (default) In addition, DB2 fenced processes can cause excessive memory use in some cases. Set the following parameter: db2 update dbm cfg using keepfenced NO If you need fenced processes and decide to use this feature of DB2, monitor the db2fmp process closely to ensure that memory usage does not become excessive. When running DB2 on AIX 5.2 or above, set the AIX network parameter lo_perf=0 to avoid an issue where loopback traffic becomes slow, and ping responds with out-of-order packets. See APAR IY63671 for more details. For assistance in performing DB2 administration tasks, see Appendix D: References for links to the DB2 Information Center. DB2 Autoconfigure Option The DB2 autoconfigure option calculates and displays initial values for the buffer pool size, database configuration, and database manager configuration parameters, with the option of applying these recommended values. The following command shows the recommended values without applying them. Remove the option “apply none” to apply these values, or set each desired configuration value individually. > db2 autoconfigure using mem_percent 80 apply none Current and Recommended Values for Database Manager Configuration Description Parameter Current Value Recommended Value ----------------------------------------------------------------------------------------Application support layer heap size (4KB) 12 (ASLHEAPSZ) = 15 15 No. of int. communication buffers(4KB)(FCM_NUM_BUFFERS) = AUTOMATIC AUTOMATIC Enable intra-partition parallelism NO (INTRA_PARALLEL) = NO IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 Maximum query degree of parallelism (MAX_QUERYDEGREE) = ANY Agent pool size (NUM_POOLAGENTS) = AUTOMATIC(100) Initial number of agents in pool Max requester I/O block size (bytes) (NUM_INITAGENTS) = 0 Sort heap threshold (4KB) (RQRIOBLK) = 32767 (SHEAPTHRES) = 0 1 AUTOMATIC(100) 0 32767 0 Current and Recommended Values for Database Configuration Description Parameter Current Value Recommended Value ----------------------------------------------------------------------------------------Default application heap (4KB) (APPLHEAPSZ) = 256 256 Catalog cache size (4KB) (CATALOGCACHE_SZ) = 300 718 Changed pages threshold (CHNGPGS_THRESH) = 80 Database heap (4KB) (DBHEAP) = 4648 Degree of parallelism (DFT_DEGREE) = 1 Default tablespace extentsize (pages) Default prefetch size (pages) (DFT_EXTENT_SZ) = 32 Default query optimization class (DFT_PREFETCH_SZ) = AUTOMATIC Max storage for lock list (4KB) (DFT_QUERYOPT) = 5 Log file size (4KB) (LOCKLIST) = AUTOMATIC (LOGFILSIZ) = 1024 Number of primary log files Number of secondary log files (LOGPRIMARY) = 13 (LOGSECOND) = 12 80 5886 1 32 AUTOMATIC 5 AUTOMATIC 1024 9 11 Max number of active applications (MAXAPPLS) = 100 AUTOMATIC Percent. of lock lists per application (MAXLOCKS) = AUTOMATIC AUTOMATIC Number of asynchronous page cleaners Number of I/O servers Package cache size (4KB) (NUM_IOCLEANERS) = 1 1 (NUM_IOSERVERS) = 3 7 (PCKCACHESZ) = AUTOMATIC AUTOMATIC Percent log file reclaimed before soft chckpt (SOFTMAX) = 520 360 Sort list heap (4KB) (SORTHEAP) = AUTOMATIC AUTOMATIC SQL statement heap (4KB) (STMTHEAP) = 8192 8192 Statistics heap size (4KB) (STAT_HEAP_SZ) = 4384 4384 Utilities heap size (4KB) (UTIL_HEAP_SZ) = 59789 311136 Self tuning memory Automatic runstats (SELF_TUNING_MEM) = ON ON (AUTO_RUNSTATS) = ON ON Sort heap thres for shared sorts (4KB) (SHEAPTHRES_SHR) = AUTOMATIC AUTOMATIC Log buffer size (4KB) 2151 (LOGBUFSZ) = 2150 Current and Recommended Values for Bufferpool(s) Chapter 3: IBM DB2 Configuration Settings for Network Manager 13 Description Parameter Current Value Recommended Value ----------------------------------------------------------------------------------------BUF32K Bufferpool size = 8192 114922 IBMDEFAULTBP Bufferpool size = -2 14028 . . . Database Maintenance Database maintenance consists of tasks such as reorganizing tables and indexes, and archiving and deleting historical data. Note: Network Manager 3.9 runs statistics on the NCIM schema tables in the Oracle and Informix databases after discovery is finished. However, starting with Network Manager 4.1, Network Manager now runs statistics on the NCIM schema tables in the DB2 database. You should not need to run statistics on NCIM schema tables if Network Manager is doing it for you. If you are storing historical polled data in the NCPOLLDATA schema of any database type, consider reorganizing and running statistics regularly on the polldata table. 1. REORG Should run at least weekly 2. RUNSTATS Should run at least weekly 3. LOG file pruning Should run at least every three days in a large environment if you are storing the historical polled data in Tivoli Data Warehouse. Reorganizing tables and indexes DB2 effectively reuses space within tables that become fragmented with updates over time. But running the REORG utility periodically is useful to optimize performance by keeping data aligned on the indexes. You can run the first command to first generate an input file for all the tables, and then run the second command to execute them: db2 -x "select 'reorg table',substr(rtrim(tabschema)||'.'|| rtrim(tabname),1,60),' ;'from syscat.tables where type = 'T' order by tabschema asc"|tr -s " "|tee reorg_all.sql db2 -tvf reorg_all.sql Update Catalog Statistics The RUNSTATS utility updates the catalog statistics that the DB2 optimizer 14 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 uses when executing queries. When there is any data modification to the user tables, the catalog statistics tables are not automatically modified. A RUNSTATS command has to be performed on tables and indexes so that columns in the catalog tables are updated with the latest information. Run the first command to generate the RUNSTATS commands for all the tables, and the second command to execute them, as follows: db2 -x "select 'runstats on table',substr(rtrim(tabschema) ||'.'|| rtrim(tabname),1,60),' with distribution and detailed indexes all ALLOW WRITE ACCESS;'from syscat.tables where type = 'T' order by tabschema asc" |tr -s " " |tee runstats_all.sql db2 -tvf runstats_all.sql Log history maintenance Regular log history maintenance is important if you are inserting and deleting rows constantly in large numbers, such as the historical polled data in the ncpolldata.pollData table and storing it in TDW. Otherwise, there may be no need to regularly do this. Check the log history file and purge it, if necessary, as in the following example: db2 'list history all for <db>' If you see more than approximately 5000 records, then that could start to impact database performance. DB2 keeps a list of all the log files that have been created, but you really only need the ones that relate to the last few backups, so as long as you have recent backups, you can purge the log files before a point in time (for example, the last 3 backups). Purge the log files by running the following command: db2 'prune logfile prior to <time>' If necessary, this should be performed regularly as part of the database maintenance. Automatic Maintenance In general, IBM Tivoli service management products do not recommend turning on automatic maintenance settings, such as AUTO_MAINT, AUTO_TBL_MAINT, AUTO_RUNSTATS, and AUTO_STMT_STATS. Instead, schedule database maintenance activities during regular maintenance windows to avoid adversely affecting end user performance. Chapter 3: IBM DB2 Configuration Settings for Network Manager 15 Chapter 4: Troubleshooting This chapter covers some basic DB2 troubleshooting steps when things go wrong as well as a summary of useful DB2 commands when performing troubleshooting tasks. Troubleshooting Tips When you see a DB2 error message in one of the Network Manager log files and the meaning is not obvious, execute the following command: db2 ? <Error Message ID> This command provides additional information on the error to make it clearer. Another place to look is the DB2 log, db2diag.log, located under the DB2 instance account directory (similar to /home/db2inst1/sqllib/db2dump/db2diag.log) and see what the error messages in context suggest. Buffer Pool This section describes how to troubleshoot a buffer pool failure. For instance you might see a failure of a buffer pool, for example, BUF32K, to get the memory it needs. DB2 offers automatic allocation but this feature may not be active for all buffer pools and this can result in unexpected issues. To fix this, reset the buffer pool to automatic. Create a file that contains the following select statement: select substr(a.tbspace,1,20) as tbspace, a.datatype, substr(b.bpname,1,20) as bpname, b.npages, a.pagesize, b.pagesize from sysibm.sysbufferpools b full join sysibm.systablespaces a on a.bufferpoolid = b.bufferpoolid order by 2, a.pagesize; Execute this statement from the instance owner account, or one with appropriate privileges: su – db2inst1 db2 connect to NCIM db2 –tvf buffer.sql TBSPACE DATATYPE BPNAME -------------------- -------- -------------------SYSCATSPACE A IBMDEFAULTBP © Copyright IBM Corp. 2013 NPAGES PAGESIZE PAGESIZE ----------- ----------- --------2 4096 4096 17 USERSPACE2 SYSTOOLSPACE USERSPACE1 TEMPSPACE1 LARGETEMP2 SYSTOOLSTMPSPACE A L L T T U BUF32K IBMDEFAULTBP IBMDEFAULTBP IBMDEFAULTBP BUF32K IBMDEFAULTBP 8192 -2 -2 -2 8192 -2 32768 4096 4096 4096 32768 4096 32768 4096 4096 4096 32768 4096 7 record(s) selected. The setting of -2 for NPAGES means it is set to automatic allocation, while the other buffer pool (BUF32K) has a fixed size. If there is a space allocation issue with a buffer pool not already set to automatic allocation, then you can alter the setting. Then restart DB2 to pick up the change to automatic allocation. As the db2inst1 user: db2 connect to NCIM db2 alter bufferpool buf32k size automatic db2stop force db2start The following example shows the settings for BUF32K to -2 : TBSPACE DATATYPE BPNAME NPAGES PAGESIZE PAGESIZE -------------------- -------- -------------------- ----------- ----------- ----------SYSCATSPACE A IBMDEFAULTBP -2 4096 4096 USERSPACE2 A BUF32K -2 32768 32768 SYSTOOLSPACE L IBMDEFAULTBP -2 4096 4096 USERSPACE1 L IBMDEFAULTBP -2 4096 4096 TEMPSPACE1 T IBMDEFAULTBP -2 4096 4096 LARGETEMP2 T BUF32K -2 32768 32768 SYSTOOLSTMPSPACE U IBMDEFAULTBP -2 4096 4096 DB2 commands useful in troubleshooting Here are some useful commands to add to your cheat sheet. See Appendix D: References for a link to the DB2 documentation for more information. Source environment for db instance owner “db2inst1”: . ~db2inst1/sqllib/db2profile Connect to database: db2 connect to <DBNAME> user <username> using <pwd> List the tables of a schema: db2 "select char(tabschema,10) tabschema, char(tabname,33) tabname, stats_time from syscat.tables where tabschema='NCIM' order by 1,2,3" Stop all processes using DB2: db2 terminate ; db2 force applications all ; db2stop force ; Clean all DB2 interprocess communications after stopping all processes: 18 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 ipclean -a Start database “ITNM”: db2start db2 restart db ITNM ; db2 activate db ITNM ; Remove all rows in a table without dropping it: db2 "ALTER TABLE NCPOLLDATA.POLLDATA ACTIVATE NOT LOGGED INITIALLY WITH EMPTY TABLE“ Show all database names in the database instance: db2 list db directory Show the details of, say, the entityData table: db2look -d <dbname> -e -z NCIM -t ENTITYDATA -i <user> -w <pw> Show configuration settings: db2 get dbm cfg db2 get db cfg for <dbname> db2set -all Debug information for IBM Support Use the following command to gather debug information when asking IBM Support for help. The command creates a file called db2support.zip that you can upload. db2support . –d <dbname> -c –s Diagnostic queries Here are some useful queries that can help you identify a problem from the recent executions of SQL queries against the DB2 database. Identify long running commands. List the last 10 queries by average execution time. This can indicate a problem if the average execution times are excessive. It may indicate locking problems or if other demands on the database resources prevent normal running of the database commands. db2 "select NUM_EXECUTIONS, STMT_TEXT from sysibmadm.TOP_DYNAMIC_SQL order by AVERAGE_EXECUTION_TIME_S desc fetch first 10 rows only" | tr –s " " List the last 10 queries by number of executions. Some of the queries that run in the normal operation of Network Manager can be very repetitive. If these are also showing longer than normal execution times, it can expose a problem. Run this command to show the most frequently executed queries: db2 "select NUM_EXECUTIONS, STMT_TEXT from sysibmadm.TOP_DYNAMIC_SQL order by NUM_EXECUTIONS desc fetch first 10 rows only" | tr -s " " Identify poor sorting. Use this command to understand the queries that had Chapter 4: Troubleshooting 19 more than X number of executions and by SORTS_PER_EXECUTION to know which queries might indicate missing or poor indexing. Poor indexing may indicate a need to run REORG to rebuild the indexes, or indicate where an additional index may be useful. If poor indexing is really the problem, rather than corrupted indexes, it may be useful to discuss with IBM Support so that if it does not impact other areas (such as inserts), the indexing in the product can be improved. db2 "select NUM_EXECUTIONS, SNAPSHOT_TIMESTAMP, AVERAGE_EXECUTION_TIME_S,SORTS_PER_EXECUTION,STMT_TEXT from sysibmadm.TOP_DYNAMIC_SQL where NUM_EXECUTIONS > 100 order by SORTS_PER_EXECUTION desc"| tr -s " " Investigate the efficiency of a query using Explain tables. Writing reports often involves creating complex SQL queries, maybe including custom tables. Optimizing the SQL query to run efficiently often involves tradeoffs with indexes, sorts, even the order the joins are evaluated. To help understand how the SQL compiler will execute the query we can use tools such as the DB2 Design Advisor or the Explain function. These are powerful tools that can provide insight into the benefit that additional indexes would provide. For example: cd ~/sqllib/misc db2 -tvf EXPLAIN.DDL db2advis -d <dbname> -n <schema> -s "SELECT DISTINCT tunnel.entityId TUNNELID, COALESCE(tunnel.displayLabel,tunnel.entityName) TUNNELNAME, dm.domainMgrId DOMAINMGRID FROM ncim.entityData tunnel INNER JOIN ncim.domainMembers dm ON dm.entityId = tunnel.entityId INNER JOIN ncim.networkPipe outerPipe ON outerPipe.entityId = tunnel.entityId INNER JOIN ncim.connects outerPipeConnects ON outerPipeConnects.connectionId = outerPipe.connectionId INNER JOIN ncim.mplsTETunnel headend ON headend.entityId = tunnel.entityId LEFT OUTER JOIN ncpgui.pathView pv ON pv.pathId = tunnel.entityId WHERE tunnel.entityType = 36 AND pv.viewId IS NULL " Command to find the diagnostic data directory. Sometimes it is a challenge to find the directory being used for the DB2 logs depending on where the DB2 client or server is installed. You can run the following command to see the directory path: db2 get dbm cfg|grep -i diag For example, the previous command might show: /opt/IBM/tivoli/netcool/platform/linux2x86/db2client/sqllib/db2dump/ For diagnostic information like memory usage and problem queries, check these files: <instance_name>.nfy For general problems, including information on deadlocks, look in the following log file: 20 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 db2diag.log Useful information for diagnosing locking problems, including a deadlock. The following commands and queries will gather information at the time of the problem that would be very useful for IBM Support. For details on a deadlock, run the db2detaildeadlock script to understand the details associated with the deadlock. Find the db2detaildeadlock script by looking under the database instance owner’s directory. It might be embedded in the Network Manager installed directory if set up by the Network Manager installer, or located under the instance owner’s home directory. For example: db2evmon -path /opt/IBM/tivoli/netcool/platform/linux2x86/db2client/ncim/NODE0000/SQL 00001/MEMBER0000/db2event/db2detaildeadlock > db2detail_dead_log.txt Use the following commands to monitor locking and to identify any long running locks, including deadlocks: db2 "select l.agent_id, substr(l.appl_name, 1, 15) as appName , substr(s.stmt_text, 1, 70) as stmt_text, l.lock_object_type, l.tabname from sysibmadm.locks_held as l, sysibmadm.long_running_sql as s where l.agent_id = s.agent_id and s.stmt_text is not null" Monitor the lock situation using the following commands: db2pd -wlocks -db <dbname> db2 get snapshot for locks on <dbname> db2 get snapshot for applications on <dbname> Chapter 4: Troubleshooting 21 Appendix A: Using the Network Manager GUI to set the Cognos datasource passwords 1. Select Reporting->Common Reporting, and click on Launch at the top left and select Administration. 2. Click on the Configuration tab. 3. Repeat the following steps for each of the data sources: NCIM, NCPOLLDATA, PARAMETERS, NCPGUI, and NCMONITOR. 4. Drill down into the data source. © Copyright IBM Corp. 2013 23 5. Select the check box and click on More… then click on Set Properties. 6. Click on the Signon tab, and then click on “Edit the signon...”. 7. Enter the database username and password on the following panel. Click Ok to save it. 24 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 Appendix A: Using the Network Manager GUI to set the Cognos datasource passwords 25 Appendix B: DB2 deprecates System Managed Spaces for permanent tablespaces System Managed Spaces (SMS) for permanent table spaces have been deprecated in DB2 10.1. For details, see: http://www.ibm.com/support/knowledgecenter/api/content/SSEPGG_10.1.0/com.ibm. db2.luw.wn.doc/doc/i0058748.html Network Manager 3.9 and 4.1.1 use DB2 10.5. SMS for permanent tables spaces is still supported in DB2 10.5, but may be dropped in a future release. http://www.ibm.com/support/knowledgecenter/SSEPGG_10.5.0/com.ibm.db2.luw.sql. ref.doc/doc/r0000929.html?cp=SSEPGG_10.5.0%2F2-9-7-102 For new databases, permanent table spaces should use Automatic Storage Table (AST) managed space. Tests for Network Manager in the Lab using AST showed some efficiency during intense database activity, for example moving the discovery data to the NCIM database after discovery. It did not show measurable improvements working with topology views in the GUI. While not tested as I write this, we would expect some similar improvement in cases where the ncpolldata database is heavily used. What does this mean for Network Manager? For existing DB2 10.1 and 10.5 databases, there is no requirement to take action. It is intended that future releases of Network Manager will use AST managed tablespaces with no action required on the part of the deployer. If you are deploying a new instance of Network Manager you can take advantage of the new efficiency and create the new database using AST tablespace. Follow these steps to edit the script used to create the DB2 database: 1. Edit the file create_db2_database.sh. This is located in either of these locations, $NCHOME/precision/scripts/sql/db2/ or in the install media under PrecisionIP/scripts/ 2. Replace the line, create tablespace userspace2 pagesize 32k managed by system using ('usercont2') bufferpool buf32k with, create tablespace userspace2 pagesize 32k managed by automatic storage bufferpool buf32k 3. Create the DB2 database using this script prior to installing Network Manager as described here: http://www.ibm.com/support/knowledgecenter/SSSHRK_4.1.1/com.ibm.networkmana gerip.doc_4.1.1/itnm/ip/wip/install/task/nmip_ins_installdb2unix.html © Copyright IBM Corp. 2013 27 Appendix C: Performance Analyst Tool This is a very useful Tivoli tool that can analyze database snapshots to identify the worst query offenders, and expose database tuning issues. Visit the home page and download this free tool from: http://ibm.biz/BdxDbg The following URL provides a comprehensive paper on using the Performance Analyst tool: https://www.ibm.com/developerworks/community/files/form/anonymo us/api/library/75dbdf46-1a08-429c-9742bd340d7d1fd3/document/7de6f5ac-ae00-4620-b207ca933b4f2939/media/DB2_Query_Tuning_and_Index_Creation.pdf To get started, run the PerfAnalyst tool and select File..Open from the menu to display a list of plugins. Select the DB2::Snapshot plugin and you will see the commands to use to generate the snapshot. Click OK and it will prompt you for the snapshot. After loading the snapshot into the tool, the Statement tab shows the queries sorted by total CPU time. In this example, you can see that a significant portion of the database CPU utilization (total_cpu_util 53%) was being consumed by a SELECT statement which throttled the delete processing from the same pollData table. Compare the average CPU (avg_busy_time) with average execution time to gauge the wait time. If there is a significant delta, the reason is probably because the query execution is spending a lot of time waiting for locks. © Copyright IBM Corp. 2013 29 30 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 Appendix D: References DB2 9.7 Information Center http://www.ibm.com/support/knowledgecenter/SSEPGG_9.7.0/com.ibm.db2.l uw.kc.doc/welcome.html DB2 10.5 Information Center http://www.ibm.com/support/knowledgecenter/SSEPGG_10.5.0/com.ibm.db2. luw.kc.doc/welcome.html IBM Tivoli Network Manager Information Center http://www.ibm.com/support/knowledgecenter/SSSHRK_4.1.1/com.ibm.netw orkmanagerip.doc_4.1.1/itnm/ip/wip/common/welcome.html DB2 Best Practises Tuning and Monitoring database https://www.ibm.com/developerworks/community/wikis/home? lang=en#!/wiki/Wc9a068d7f6a6_4434_aece_0d297ea80ab1/page/Tuning %20and%20Monitoring%20Database%20System%20Performance Top 10 Db2 Performance tips http://www.ibm.com/developerworks/data/library/techarticle/hayes/0102_haye s.html 10 useful DB2 redbooks https://www.ibm.com/developerworks/community/blogs/SusanVisser/entry/to p_ten_ibm_redbooks_for_db2_infosphere1?lang=en © Copyright IBM Corp. 2013 31 Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY 10504-1785 U.S.A. For license inquiries regarding double-byte character set (DBCS) information, contact the IBM Intellectual Property Department in your country or send inquiries, in writing, to: Intellectual Property Licensing Legal and Intellectual Property Law IBM Japan Ltd. 1623-14, Shimotsuruma, Yamato-shi Kanagawa 242-8502 Japan The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those © Copyright IBM Corp. 2013 33 Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact: IBM Corporation 958/NH04 IBM Centre, St Leonards 601 Pacific Hwy St Leonards, NSW, 2069 Australia IBM Corporation 896471/H128B 76 Upper Ground London SE1 9PZ United Kingdom IBM Corporation JBFA/SOM1 294 Route 100 Somers, NY, 10589-0100 United States of America Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee. The licensed program described in this document and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any equivalent agreement between us. Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. All statements regarding IBM's future direction or intent are subject to change or withdrawal without notice, and represent goals and objectives only. 34 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2 All IBM prices shown are IBM's suggested retail prices, are current and are subject to change without notice. Dealer prices may vary. This information is for planning purposes only. The information herein is subject to change before the products described become available. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. The sample programs are provided "AS IS", without warranty of any kind. IBM shall not be liable for any damages arising out of your use of the sample programs. Each copy or any portion of these sample programs or any derivative work, must include a copyright notice as follows: © (your company name) (year). Portions of this code are derived from IBM Corp. Sample Programs. © Copyright IBM Corp. _enter the year or years_. If you are viewing this information softcopy, the photographs and color illustrations may not appear. Trademarks IBM, the IBM logo, ibm.com, Netcool, Netcool/OMNIbus,and Tivoli are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml. Adobe, Acrobat, and Portable Document Format (PDF) are trademarks or registered trademarks of Adobe Systems Incorporated in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Notices 35 Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. 36 IBM Tivoli Network Manager: IBM Tivoli Network Manager with DB2