Groovy Copybook Slurper


In order to call CICS from Java, we've traditionally used RDz to generate a J2C bean via an Ant script. But I don't have RDz anymore (it doesn't support Groovy and always running older versions of Eclipse), so my only alternative was to use a JCL to generate a bean via IBM's JZOS library.

I'm still not a big fan, it takes time to run the JCL, download the generated file from the mainframe and tweak the object names, packages, etc.. What if I could just declare my copylib structure in my Java code and call CICS? To that end, I've been working on a parser which converts a byte stream to/from a map.

Leaving the CICS call of it out for the moment, here's an idea of want the read code would look like.

def slurper = new CopybookSlurper("""\
01 TESTING-COPYBOOK.
   03 STATES OCCURS 10 TIMES .
      05 STATE PIC XX.
""")
def data = slurper.getReader('MIOHINWINVCAKYFLNCSC'.getBytes())
data.STATES.each {
  print it.STATE  //will print each two byte state
}
assert 'IN' == data.STATES[2].STATE //zero-based indexed of the STATES list

And here's the write code

def slurper = new CopybookSlurper("""\
  01 TOP-LEVEL.
     03 STATES OCCURS 2 TIMES.
        05 STATE-NUM        PIC X(3).
        05 STATE-NAME       PIC X(10).
        05 NUMERIC-STATE    PIC 9(2). """)
    
def writer = slurper.getWriter(new byte[slurper.length])
writer.with {
  STATES[0].with {
 STATE_NUM = '1'
 STATE_NAME = 'MICHIGAN'
 NUMERIC_STATE = 2
  }
  STATES[1].with {
 STATE_NUM = '2'
 STATE_NAME = 'OHIO'
 NUMERIC_STATE = 3
  }
}
assert new String(writer.getBytes(), 'IBM-37') ==
       '1  MICHIGAN  022  OHIO      03'

The slurper name and map structure is inspired by Groovy's XmlSlurper and JsonSlurper. My parser implementation is written in groovy to take advantage of the dynamic nature of the maps and closures. I used the JZOS library under the covers to handle parsing the individual data elements so it can handle binary fields (COMP-3 stuff). Unfortunately JZOS didn't provide any simple hooks for getting a data type based on a PIC definition, so I had to rewrite that logic. I know my parser isn't as robust as the IBM version, it doesn't handle some of the weird datatypes for instance, but it's good enough for the normal copybooks that we'd be using from the web.

I also wanted to make my map lazy so that the bytes are only parsed if requested, after all sometimes COBOL copybooks have a ton of data in them, and I only want a small piece. So rather than parsing the field right away, the parser creates a map of closures with instructions on how to parse the data. Likewise, the writer stores a closure for each variable with instructions on how to write data to a shared byte array. The getReader method is shown below. There's a lot of complexity involved in occurrences and groups that you can't see here, but I just wanted to show the mapping of the variable names to closures.

public Map<String, ?> getReader(byte [] bytes) {
  Map readers = [:]
  fieldProcessors.each { DataVariable variable ->
    if (variable.read) {
      readers[variable.name] = { variable.read(bytes) }
    }
  }
  return new LazyProperties(readers, bytes)
}

The LazyProperties class extends HashMap and invokes the closures from the map when referenced, then caches the results for the next call. The getWriter method works in a similar fashion with closures, though it doesn't cache results because any call to the write methods should write data to the byte stream.

That's what I have working so far, the next step is to look into the javax.resource.cci.Record and javax.resource.cci.Streamable interfaces and perhaps Spring's org.springframework.jca.cci.core.support.CciDaoSupport to call down to CICS.

12 comments

Awesome! Do you plan to release this code?
Best Regards,
Sérgio Fonseca

Reply

Yeah, I could probably do that. The version we're using was developed at work, so I consider it proprietary, but I could create a new open source version and host it on GitHub. I'll try and do that in the next week or so.

Reply

That would be great!
I'm looking forward to it.
Best Regards,
Sérgio Fonseca

Reply

Okay, I pushed some code up to github. Check out https://github.com/jedibos/CopybookSlurper. I confess as I look at the code again, I'm not sure how understandable it'll be, took a lot of effort to parse things properly, but there's lots of comments explaining what's going on.

Reply

Hi,
I looked at the code and it is very well documented, also the code is clean and easy to understand, very good work!
I will download the IBM JZOS Library and play with your library.
Thank you very much for releasing such a piece of software.

Best Regards,
Sérgio Fonseca.

Reply

Hi,

Please, can you help me with a little issue?
I am trying to do calculations with negative values, but the output always returns as a positive number.

Sample:

def slurper = new CopybookSlurper('''\
01 CUSTOMER.
03 TOTAL PIC S9(009)V99.

''')
def reader = slurper.getCopybook('0000022621-'.getBytes('IBM-37'))

println reader.TOTAL


It prints out 2262.10 instead of -2262.10

The problem seems to be at this point:
Class CopybookSlurper, line 457.
I did some tests with the CopybookSlurper class, but I wasn't able to fix the problem, I need to deep understand your class before sugesting any updates.

Any sugestions?

Best Regards,

Sérgio Fonseca
sergiofonseca.j@gmail.com

Reply
This comment has been removed by the author.
This comment has been removed by the author.
This comment has been removed by the author.

Huh, I haven't seen that format before. Is (009) equivalent to (9)? So the full declaration would be equivalent to S9(9)? Or S9(3)? The code is supposed to handle for signs already. It makes me wonder what's happening in the case statement in the CopybookSlurper class. There's a check on line 192 that is supposed to figure out how big the integer number is, (009) will be transformed into 9. But the sign is on the far right isn't it...

I get what you're doing with your fix, but I don't feel like it should be necessary. Maybe I'm loading up the jzos parser incorrectly. I'll see if I can take a closer look.

Reply

I think the problem is the EBCDIC encoding. I ran this code:

CopybookSlurper slurper = new CopybookSlurper('01 CALCULATION PIC S9(009)V99.')
Copybook copybook = slurper.copybook
copybook.'CALCULATION' = -2262.10
println copybook.getDataString();

and the result was 0000022621} not 0000022621-. Next I tried to print out the hex codes of both strings.

println copybook.getBytes().encodeHex().toString()
println '0000022621-'.getBytes('IBM-37').encodeHex().toString()

The results were similar, but as expected the last byte was different. f0f0f0f0f0f2f2f6f2f1d0 versus
f0f0f0f0f0f2f2f6f2f160


Finally I created your string in a testdata member on the mainframe and then used 'HEX ON' to see what the hex codes would look like on the mainframe.


0000022621-
FFFFFFFFFF6
00000226210


As expected, the '-' translated to '60' in hex, which in ASCII on a windows machine looks like a '}'. I thought about creating a different version of this code that could handle encodings other than IBM-37, but the JZOS library uses EBCDIC everywhere, so I'd have to rip all of that out.

Reply

Hello,

Thanks for doing these tests, you are correct.
The problem was the encoding, your code works just fine.

About your question:
Is (009) equivalent to (9)?
Yes, same thing.

I did some tests with other formats, all they worked.

9(004)V9(06)
9(009)V9999
9(04)V9(4)

Again, thanks for the help and sorry for the delay.

Best Regards

Reply

Post a Comment