Warren Gill – [email protected] 20 March 2012

3603 Defining Cross Platform Dependencies with Shadow Jobs

© 2012 IBM Corporation

Workload Automation

Overview

Introduction Description of cross-engine dependencies Installation Overview of the necessary installation steps Configuration Configuring TWS/d and TWS/z to manage cross-dependencies, creating the remote engine workstation Running jobs Creating and running shadow jobs

2

© 2012 IBM Corporation

Cross-engine dependencies Local engine

Remote engine

JS1

JS3

shadow_job

job3 JS2.job2 follows JS1.shadow_job

JS2 job2

3

Feature introduction The feature introduces shadow jobs:  A shadow job is a job, defined on the local engine, that maps a remote job defined on another Tivoli Workload Scheduler engine, either Distributed or z/OS.

 Once in the plan, the shadow job binds to an instance of the remote job in the remote plan and starts tracking the remote job status changes.  A dependency from the remote job is defined locally as a dependency from the shadow job.

4

Remote engine vs local engine  The job to monitor (remote job) runs on a remote engine  The shadow job:

– maps the remote job – runs on the local engine – is defined on a workstation of type REM-ENG

Remote Engine remote_job

Local Engine shadow_job

 Both the remote and the local engines can be distributed Master Domain Managers or z/OS controllers

5

Installation and configuration: TWS/d  Tivoli Workload Scheduler 8.6 or later is required on both local and remote engines  No extra installation or configuration is needed

6

Planning for your configuration: TWS/z You must configure your controller for HTTP or HTTPS • Decide whether or not to protect your controller using SSL and set the HTTPOPTS statement in the controller parameter library accordingly. • Define a HTTP/HTTPS destination for each remote engine with which the controller will communicate. • The choice between HTTP and HTTPS destinations depends on how the remote engine is configured. If the remote engine is a TWSz controller with a stand-by controller, configure them to listen on the same VIPA address and port

7

TWS/z: Connections using SSL To secure the HTTP connection using SSL, or if a remote engine only accepts HTTPS requests, first define an SSL key ring • •

Get the proper certificates. If you are in a test environment you can bypass this step and use the default security certificates stored into the SEQQDATA library: o EQQCERCL: the security certificate for the client. o EQQCERSR: the security certificate for the sever.

•

Create in the RACF a key ring containing these certificates. o You can use the sample job EQQRCERT to import the certificates. To run this job, ensure that you use the same user ID that RACF associates with the controller started task. TWS/z also supports key database USS files.

•

8

TWS/z: Enable HTTP{S} Enable HTTP and/or HTTPS communications both in the local and in the remote controller. Specify in the HTTPOPTS statement: o The address and port where the controller waits for incoming requests: HTTPOPTS

HTTPOPTS HOSTNAME('1.111.111.111') HTTPPORTNUMBER(4567)

HOSTNAME('2.222.222.222') HTTPPORTNUMBER(511)

o If you are using SSL, you must also specify the name of the key ring containing the certificates: HTTPOPTS HOSTNAME('1.111.111.111') HTTPPORTNUMBER(4567) SSLPORT(4568) SSLAUTHMODE(CAONLY) SSLAUTHSTRING(tws_test) SSLKEYRINGTYPE(SAF) SSLKEYRING(KEYRINGNAME)

9

HTTPOPTS HOSTNAME('2.222.222.222') HTTPPORTNUMBER(511) SSLPORT(512) SSLAUTHMODE(CAONLY) SSLAUTHSTRING(tws_test) SSLKEYRINGTYPE(SAF) SSLKEYRING(KEYRINGNAME)

TWS/z: Set the routing table Specify in the ROUTOPTS statement:  An HTTP or HTTPS destination for each controller you will bind  Both z/OS controllers and distributed masters must have routing entries ROUTOPTS HTTP(TWS2DEST:'2.222.222.222'/511/Z) HTTPOPTS HOSTNAME('9.168.119.180') HTTPPORTNUMBER(4567)

HTTPOPTS HOSTNAME('2.222.222.222') HTTPPORTNUMBER(511) ROUTOPTS HTTP(TWS2DEST:'2.222.222.222'/511/Z) HTTP(MDMDEST:'host1234.domain.com'/31115/D)

http://host1234.domain.com:31115/JobManagerRESTWeb/JobScheduler 10

Components On the local engine: 



Create a workstation of type REM-ENG Create a job (shadow job) that maps the job on the remote engine

Example Local Engine REMENG_WKS#JS.shadow_job

11

Remote Engine WKS#JS.JOB

REM-ENG creation, TWS/d ●

On the local engine, create a workstation of type REM-ENG

COMPOSER DEFINITION OF A WORKSTATION OF TYPE REM-ENG

12

CPUNAME REMENG_WKS DESCRIPTION "Sample Remote Engine Workstation" OS WNT NODE ecerasaro.romelab.it.ibm.com TCPADDR 31115 TIMEZONE Europe/Rome FOR MAESTRO HOST ECERASARO_DWB TYPE REM-ENG PROTOCOL HTTP END

REM-ENG creation: TWS/z > TWS/z Define a remote engine workstation in the local controller. A remote engine workstation is a new type of workstation used to represent a remote Tivoli Workload Scheduler engine or a remote Tivoli Workload Scheduler for z/OS controller: For each remote TWSz controller define a remote engine workstation with the following characteristics:  Work station type

===> R

Remote engine

 Reporting attribute

===> A

Automatic

 Server usage

===> N

Neither

 Destination  Remote engine type ===> Z

13

===> TWS2DEST z/OS

HTTP/HTTPS destination

REM-ENG creation: TWS/z > TWS/d Define a remote engine workstation in the local controller.  For each remote TWSd master and for its backup masters define a remote engine workstation with the following characteristics: – Work station type ===> R Remote engine – Reporting attribute ===> A Automatic – Server usage ===> N Neither – Destination ===> MDMDEST HTTP/HTTPS destination – Remote engine type ===> D Distributed  Set the backup master workstation as alternate workstation of the master workstation to transparently manage a switch master scenario.

14

Defining shadow jobs in the database To define that job C in the local engine starts after that job B in the remote engine completes successfully: • • •

Define a shadow job B-shadow in the local controller and add to job C a dependency on it A shadow job is defined on a remote engine workstation. In the remote job data section you specify the key of the remote job in the remote engine database:

Key for distributed shadow job: ● Job stream workstation ● Job stream name ● Job name

15

Key for z/OS shadow job: ● Job stream (application) name ● Job (operation) number

Defining shadow job behavior In the remote job info panel you can also specify the shadow job behavior when a bind failure occurs. – The “complete if bind fails” option indicates if the behavior must be: • •

16

N: End in error with error code FBND - the default Y: Complete successfully.

Defining shadows in TDWC Tivoli Dynamic Workload Console Two new job types created for this feature provide customized panels The look-up function allows authorized users to get remote job data directly from the remote engine database

17

Cross-engine dependencies production flow 1. On the local engine submit a shadow job 2. The local engine sends a binding request to the remote engine 3. On the remote engine, the binding process starts and looks for the job to match 4. The remote engine sends to the local engine the binding response (matching result) 5. If a match exists, the remote engine sends the remote job status changes to the local engine 6. Remote job status changes are applied to the shadow job

18

Shadow jobs binding flow 1. Submit shadow job

Local Engine shadow_job 6. Apply job status changes

2. Binding request 4. Binding response

3. Binding process

Remote Engine remote_job

5. Status change notifications

Remote DB

19

Monitoring shadow jobs through binding  Once in the plan, the synchronization between the Shadow jobs and the remote jobs starts when the Shadow jobs dependencies are satisfied  The Shadow job status remains READY until the remote job starts  The extended status (status details) provides additional information about what is happening The controller is sending the bind request to the remote engine... … the remote engine has received the bind request and it is processing it.... ...the remote engine has found a match in the remote engine plan; either the LTP or the CP.

If the bind request fails the shadow job status is set to ERROR with error code FBND or to COMPLETE basing on the “Complete if bind fails” option.

20

z/OS shadow jobs status transition

21

Distributed shadow jobs status transition

22

Workload Automation

23

© 2012 IBM Corporation

Workload Automation Legal Disclaimer • © IBM Corporation 2011. All Rights Reserved. • The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. • References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. • If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete: Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. • If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete: All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. • Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both. • If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete: Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. • If you reference Java™ in the text, please mark the first use and include the following; otherwise delete: Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. • If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete: Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. • If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete: Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. • If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete: UNIX is a registered trademark of The Open Group in the United States and other countries. • If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete: Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. • If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.

24

© 2012 IBM Corporation