GDG to GDGE Conversions

I've been trying since SHARE ended (March 10, 2017) to get this blog entry out. As happens to everyone, things just got in the way.

While in San Jose, I ran into a nice customer that I'd talked to last year at his site. He was asking about how to convert GDGs to GDGEs, as he had z/OS V2.2 up and running and was looking at exploiting them now*. I had talked about this customer in 2015 (or 2016?), and had shown my experiences, and even gave a little sample JCL that I used to do the conversion. Now, my JCL didn't work on non-SMS GDGs, but since my use case was SMS I was ok. (My JCL also would handle migrated data sets, btw.) The "heart" of the JCL was that an IDCAMS ALTERs would be done, along with machinations using temporary GDG's, in a lovely six step job.

The specific use case for this customer was that he had non-SMS GDGs on tape. My JCL didn't work. So, back to the drawing board to see if there was something we could improve on. This meant some emails with Mr. Catalog (aka Steve Branch from DFSMS), and some tests. The customer himself even did some tests. Steve came up with an idea of trying out TSO/E RENAME commands, since the IDCAMS ALTERs didn't handle non-SMS managed GDS (which IDC3009I told me nicely about). Steve was onto something, but said the tape data sets were still going to be a no-go with this method.

Cue up some more JCL and some tests... I wanted to show you what I tried out and what the customer tried out, in hopes that it might help you in planning for GDGEs. I'll call the old job "IDCAMS ALTER" and the new job "TSO/E RENAME" and show you what I saw. They both still have several steps, but what's a couple more job steps when you can get up to 999 generations?

Case A: NON-SMS, on DASD:
IDCAMS ALTER step: Failed as expected on NEWNAME, The explanation was: AN ATTEMPT WAS MADE TO ALTER A NON-SMS MANAGED GDS TO A NON-GDS NAME.
TSO/E RENAME: Success!

Case B: SMS on DASD.
IDCAMS ALTER step: Success, as expected, since this was my 2015 experience. All MIGRAT1 data sets were recalled, fwiw.
TSO/E RENAME: Success, as expected. I did have some MIGRAT1 and MIGRAT2 data sets, and they were all recalled when over.

For the tape cases, I didn't not try this out personally as the customer shared with me his experience with TSO/E RENAME in his environment:
Case C: TAPE - NON-SMS:
IDCAMS ALTER step: Expected to fail, because not on DASD, and did not try.
TSO/E RENAME: Customer: It failed as expected, because not on DASD. The messages the customer got were: DATA SET 'HHH.TAPEGDG.XXXX.G0001V00' NOT ALLOCATED, REQUIRED VOLUME NOT MOUNTED VOLUME OR CVOL NOT ON SYSTEM AND CANNOT BE ACCESSED
-and- when trying again with a DD statement so that the volume would be mounted for the RENAME:
DATA SET 'HHH.TAPEGDG.XXXX.G0001V00' NOT ON A DIRECT ACCESS DEVICE, NOT SUPPORTED.

However, for Case C if you are really wanting to do the conversion, you could do it with some time and effort. You could write some REXX to produce JCL that would individually: uncatalog the tape GDG generations, delete the GDG base, define it as a GDGE, recatalog all the tape generations as GDGE.

My conclusion: I like TSO/E RENAME over the IDCAMS ALTER as it handles both non-SMS and SMS on DASD nicely (and hmigrated too, and recalls). Still though, if you've got to convert non-SMS tape generations, you are going to be into a small-ish project. Might still be worth it, though, if you have a lot of JCL that you don't want to change and don't know where all the changes would be found.

Finally, Lizette Koehler and I have been in some contact about this. Lizette was wondering about the internal differences between a GDG and a GDGE, and why the IDCAMS ALTER "direct" method was difficult to implement (instead of the six step method). Back to Mr. Catalog, and Steve shares his thoughts with you and me:

--- beginning of Steve's response --
To explain the internal differences of GDGs vs GDGEs, it is best to first know the limitations of a GDG. GDG base records contain a GAT (generation aging table) cell which has a GAT limit contained in 1 byte. This limits the GAT count to 255. The GAT cell also has a table that keeps track of the order of the generations. Each entry in the GAT table has a corresponding GDS sub-record cell that has the necessary info to generate the GDS name and its volumes. To locate a GDS, all that is necessary is to read the GDG base record.

The GDGE record design does two things: makes the GAT limit field 2 bytes and removes the concept of GDG sub-records. All GDS are kept as nonVSAM records. In theory, the GAT count could be much larger than 999; the 999 was chosen because it is the largest number JCL can currently have without making programming changes.

Here is what I would think we would need to do a GDG to GDGE conversion:

  1. Create nonVSAM records for each GDS sub-record.
  2. Remove the sub-records from the GDG and delete any GDG extension records.
  3. Reformat the GDG base to a GDGE creating a 2 byte GAT limit field and shifting the GAT table over.
  4. And this is a real issue/headache - provide back out in case of a failure. This was the real Achilles' heel for VSAM-ICF Catalog conversion, once started, the user was committed. The user was warned to make a full volume copy, and in case of a problem, restore the volume! We currently provide DEFINE back out which is not too difficult because in case of failure, you simply delete what was defined so far. For GDG to GDGE conversion, you must detect at what point 1, 2 or 3, the failure occurred and have code to get back to the original GDG with all GDG extensions. We have similar code for ALTER NEWNAME which is convoluted at best. 

--- end of Steve's response --

Here's the complete "new" TSO/E RENAME JCL that Steve has proposed and which we both have tested. It doesn't use any IDCAMS ALTERs. As always, my TPS reports are my old GDG data set and I wish to convert them to GDGEs, because you can never enough of those TPS reports.

//* ******************************************************
//* CONVERT A GDG TO A GDGE (NON-SMS OR SMS) ON DASD
//* USES TSO/E RENAME INSTEAD OF IDCAMS ALTER NEWNAME
//* ******************************************************
//STEP01 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
 LISTC ENTRY(MWALLE.TPS.REPORTS) ALL  
 LISTC LVL(MWALLE.TPS.REPORTS) ALL  
 DEFINE GDG (NAME(MWALLE.TPSTEMP.REPORTS) LIMIT(255) SCRATCH PURGE)
//STEP02 EXEC PGM=IKJEFT01
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
 RENAME 'MWALLE.TPS.REPORTS.*' 'MWALLE.TPSTEMP.REPORTS.*'
//STEP03 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
 DELETE MWALLE.TPS.REPORTS GENERATIONDATAGROUP
//STEP04 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
 DEFINE GDG (NAME(MWALLE.TPS.REPORTS) EXTENDED LIMIT(999) -
 SCRATCH PURGE)
//STEP05 EXEC PGM=IKJEFT01
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
 RENAME 'MWALLE.TPSTEMP.REPORTS.*' 'MWALLE.TPS.REPORTS.*'
//STEP06 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
 DELETE MWALLE.TPSTEMP.REPORTS GENERATIONDATAGROUP
 LISTC ENTRY(MWALLE.TPS.REPORTS) ALL
 LISTC LVL(MWALLE.TPS.REPORTS) ALL
//*

 

*Warning: you need to have z/OS V2.2 everywhere for GDGEs, so this is something to consider exploiting after getting rid of all z/OS V2.1 and R13 from your enterprise.

May the generations be with you, 

Marna

1 Comment

A follow on comment....

April 21, 2017 02:49 PM by Marna Walle

A huge thanks and recognition to a customer (Richard Jakobek, from the UK) who has graciously written some REXX to do just what I mentioned above!  I'm going to paste it below, as I cannot attach any files on this blog.  He has offered to share it with any that are interested...

 

The JCL (for JES3):

//RJAKGDG JOB 'JCL gen',MSGCLASS=X,MSGLEVEL=(1,1) NOTIFY=&SYSUID
//*MAIN SYSTEM=SY2,CLASS=MVS
//LISTCAT EXEC PGM=IDCAMS
//SYSPRINT DD DISP=(,PASS),UNIT=VIO,DSN=&&LIST
//SYSIN DD *
LISTCAT LEVEL(RJAK.TAPEGDG.TESTING) VOLUME
//*
//JCLGEN EXEC PGM=IKJEFT01
//SYSPROC DD DISP=SHR,DSN=RJAK.LP2.JCL
//INFILE DD DISP=SHR,DSN=*.LISTCAT.SYSPRINT
//* Use comment to write the job to Sysout or to internal reader.
//OUTFILE DD SYSOUT=*
//* OUTFILE DD SYSOUT=(A,INTRDR)
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
GDGxtend 366
//

 

The REXX:

/* Rexx to rebuild a tape GDG with EXTENDED feature. */
/* This rexx will build the JCL to uncatalog all of the tapes in a */
/* GDG. Then delete that GDG and redefine it with as EXTENDED. It */
/* then recatalogs the tapes to the new GDG. */
/* The GDG name and the new LIMIT value for the GDG are passed from */
/* the calling job. */
jcl.1="//RJAKGDGR JOB 'GDG Rebuild',MSGCLASS=X,MSGLEVEL=(1,1)"
jcl.2='//*MAIN SYSTEM=SY2,CLASS=MVS'
jcl.3='//UNCAT EXEC PGM=IDCAMS'
jcl.4='//SYSPRINT DD SYSOUT=*'
jcl.5='//SYSIN DD *'
nn = 6 /* The number of the next JCL stem variable */
new_limit = ARG(1) /* New value for limit. Passed from job */
gdg_base = 0
'EXECIO * DISKR INFILE (FINIS '
Do while queued() > 0
pull line
if pos('LISTCAT',line)>0 then
do
cp = pos(')',line)
dsn_length = cp-19
gdg_name = substr(line,19,dsn_length)
end
/* Look out for a second GDG which shares the same HLQs as the one */
/* being processed. If a second name is found, jump to JCL build. */
if pos('GDG BASE',line)>0 then
do
gdg_base = gdg_base+1
if gdg_base>1 then
do
SAY 'A second GDG base found matching mask'
SAY 'The second GDG name is:' word(line,4)
SAY 'This GDG will be ignored.'
signal step2
end
end
/* ------------------------------------------------------------------ */
if pos('NONVSAM ------- ',line)>0 then
do
gds_name = word(line,3)
jcl.nn = ' DELETE 'gds_name ' NOSCRATCH '
nn=nn+1
end
End
/* ------------------------------------------------------------------ */
step2:
DELSTACK
jcl.nn= '//* -------------------------------------------'
nn=nn+1
jcl.nn= '//GDGE EXEC PGM=IDCAMS,COND=(0,NE) '
nn=nn+1
jcl.nn= '//SYSPRINT DD SYSOUT=* '
nn=nn+1
jcl.nn= '//SYSIN DD * '
nn=nn+1
jcl.nn= ' DELETE 'gdg_name
nn=nn+1
jcl.nn= ' DEFINE GDG ( - '
nn=nn+1
jcl.nn= ' NAME('gdg_name') - '
nn=nn+1
jcl.nn= ' EXTENDED SCRATCH LIMIT('new_limit') - '
nn=nn+1
jcl.nn= ' ) '
nn=nn+1
jcl.nn = '/*'
nn=nn+1
jcl.nn = '//* ----------------------------------------- '
/* ----------------------------------------------------------------- */
/* Third step, builds the IDCAMS JCL to recatlog the tapes. */
nn=nn+1
jcl.nn='//RECAT EXEC PGM=IDCAMS,COND=(0,NE)'
nn=nn+1
jcl.nn='//SYSPRINT DD SYSOUT=*'
nn=nn+1
jcl.nn='//SYSIN DD *'
nn=nn+1
gdg_base=0
/* Read the infile again to get the names to be recataloged. */
'EXECIO * DISKR INFILE (FINIS '
Do while queued() > 0
pull line
if pos('GDG BASE',line)>0 then
do
gdg_base = gdg_base+1
if gdg_base>1 then signal Step3
end
if pos('NONVSAM ------- ',line)>0 then
do
gds_name = word(line,3)
jcl.nn = ' DEFINE NONVSAM (NAME('gds_name') -'
nn=nn+1
end
if pos('VOLSER',line)>0 then
do
volser = substr(word(line,1),19,6)
jcl.nn =' DEVT(CART) VOLUMES('volser') )'
nn=nn+1
end
End
/* ------------------------------------------------------------------ */
Step3:
DELSTACK
jcl.nn = '/*'
nn=nn+1
jcl.nn = '//'
/* ------------------------------------------------------------------ */
/* Write the generated JCL to file OUTFILE */
'EXECIO * DISKW OUTFILE (STEM jcl. FINIS '
/* ------------------------------------------------------------------ */

Recent Stories
Some Handy z/OS V2.3 (and z14!) Information

GDG to GDGE Conversions

Arriving in HOLDDATA near you: z/OS V2.3 FIXCATs!