Writing A Stacking PC Routine in Metal C

Every now and then, I encounter a need to invoke a "privileged operation" from a "business application" in z/OS. As an example, I recently needed to invoke the PCKMO (Perform Cryptographic Key Management Operation) machine instruction, to encrypt a key under the current LPAR wrapping key, making it a so-called "protected" key. As to why I needed to do this from a "business application" is a discussion for another day. Just go with me on this - it's a useful, simple example.

The PCKMO machine instruction is described on page 10-78 of the "POP": z/Architecture Principles of Operation

In order to invoke this PCKMO instruction from a "normal" business application, the application would need to be APF-authorised, and it would need to switch to "supervisor state" to issue the instruction, then switch back to "problem state".

In a secure z/OS environment, business applications are not APF-authorised. APF-authorised libraries are strictly controlled, and the code they contain is not called directly by business applications. Instead, z/OS provides the "Program Call" (PC) mechanism to call from the "problem state" application in one address space, to a PC routine (fetched from an APF-authorised library) in another address space (or at least, loaded by a task in another address space). The PC routine runs in supervisor state (typically) and performs the required "privileged operation", before returning to the calling application.

Being z/OS, there is more than one way to "skin this cat". Back in the day, I might have employed a supervisor call (SVC) or some other cross-memory mechanism, but the "modern" way is to employ what's called a "stacking" PC routine.

Thanks to z/OS doing the heavy lifting of saving the caller's state and invoking the PC routine in supervisor state (if that's what you want), writing the PC routine is straightforward, as you will see. However, registering the PC routine so it can be called from any (other) address space is relatively complex. Also, some thought is needed around deciding how the calling application(s) will locate the (dynamically loaded and registered) PC routine.

In this article, I present what I believe to be the simplest approach. I will construct 3 components:

• The stacking PC routine that "does the work".

• A "started task" which registers the stacking PC routine with z/OS and advertises the PC routine's presence in a way that the stub can find it and call it.

• A "stub" subroutine which simplifies the process of calling the PC routine from my business application.

Stacking PC Routine

The requirements for a stacking PC routine are pretty straightforward:

• A routine written in Metal C needs a dynamic storage area (DSA). Rather than allocate this DSA in the PC routine, I will pass it as a parameter from the stub. The stub will provide a DSA in 31-bit storage (for reasons that will become obvious).

• The caller's "state" (including registers) is saved on the "linkage stack" by z/OS, so there is nothing for the PC routine to do on exit, except issue a PR (Program Return) instruction to "pop" the caller's state off the linkage stack and return control to the caller.

• The PC routine receives a parameter list, pointed at by general register 1 (GR1), as per usual, and the routine performs its processing based on the passed parameters, as any subroutine would. GR0, GR1 and GR15 (which normally holds the "return code" of the PC routine) can be set by the PC routine to communicate its success (or otherwise). All other registers are restored to their original values on exit, by the PR instruction.

Something that is not required, but is worth considering, is to compile the stacking PC routine for 64-bit addressing mode. I have done this to support callers in any addressing mode.

Metal C Prolog and Epilog

Since the stub will provide a DSA, and z/OS saves the caller's "state" before the PC routine is called, the prolog and epilog are trivial.

The prolog looks like this:

And the epilog looks like this:

What could be simpler than that?

Passing Parameters

How to pass parameters from the stub to the PC routine is a matter of personal taste. Because I want to provide a way of invoking several different "privileged" operations, I will build one PC routine and many stubs. Each stub will have the "natural" parameter list for the operation being performed, but the PC routine will expect these parameters to be loaded into a parameter block (as character or binary values, or 64-bit pointers).

Security

Because my PC routine will invoke a privileged machine instruction, I should (some would say "must") test the caller's access to a RACF profile corresponding to the operation being performed. I will invoke the RACROUTE REQUEST=FASTAUTH macro for this purpose.

Using FASTAUTH, rather than AUTH, for the access check means the class holding the profile must be RACLISTed. Accordingly, I have chosen the FACILITY class for my profile.

I'm not going to go into chapter and verse about issuing a RACROUTE. The interested reader can examine the scriptures for detailed information: RACROUTE REQUEST=FASTAUTH

Here's my subroutine that performs the access check:

Note that the RACROUTE macro only supports 24-bit and 31-bit addressing modes. Here's where allocating the DSA for the PC routine from 31-bit storage is essential. All the parameters passed to the RACROUTE macro are local/automatic variables (hence reside in 31-bit storage). Also note the SAM31 and SAM64 instructions which switch to/from 31-bit addressing mode to keep RACROUTE happy.

Given I am testing permission to invoke the PCKMO instruction, my "fastauth()" subroutine returns "true" if the caller has READ access to profile MACHO.PCKMO in the FACILITY class. It is vital that this access checking is performed in the PC routine and not in the stub. If I performed the access checking in the stub, anyone with knowledge of how to locate the registered PC routine could call it directly from "problem state" code, bypassing my security check!

The following TSO commands give the required access to a user, FRED:

Processing

I have created a header file to be included by the PC routine and the stub(s) that call it. In particular, this header defines the parameter block formats, which overlay each other:

The PC routine determines which verb is being requested, checks access, then invokes that verb:

Recovery

In general, a PC routine must provide and register a Functional Recovery Routine (FRR), which is invoked by the operating system if the PC routine crashes (ABENDs) before returning to its caller. The FRR is responsible for releasing or cleaning up any "resources" the PC routine has acquired, unseen by the calling process (hence the calling process can't clean them up). A good example is a latch that a PC routine might be holding, to synchronise access to some resource. The FRR should release this latch, to prevent a "hang" in the system. Fortunately, in my PC routine, I don't acquire any resources that need cleaning up, so I don't need a FRR. If your PC routine does acquire resources that need cleaning up in the event of an ABEND, you need a FRR (see: Providing recovery).

Started Task

I have chosen to use a started task to load and register my PC routine with z/OS. The other common method is to create a "subsystem" - this makes the registration process more complex/tedious, but the stub(s) can locate the PC routine by walking control blocks (which is fast).

Let me lay out the registration process, then I will describe each step in detail:

• Load PC routine into "global" storage, so any address space can call it.

• Reserve a system linkage index

• Construct an entry table holding the "launch settings" for the PC routine

• Connect the entry table to the linkage index

• Record the PC number where stubs can find it

• Wait around until told to terminate, leaving the PC routine available to all address spaces

• Delete the name/token pair holding the PC number

• Destroy the entry table

• Free the system linkage index

• Unload the PC routine

I've written my started task in "normal" 31-bit C (what I call "LE C", as opposed to Metal C), so I have access to the full C runtime library. This makes embedding the Assembler macros required to perform the registration steps a bit more complex, but it's "doable".

Almost all these steps require the caller to be in supervisor state and/or key 0 to 7. This means my started task must be APF-authorised. My practice is to switch to supervisor state only when necessary and switch back immediately when supervisor state is no longer required. Accordingly, I need a subroutine I can call switch to supervisor state (and key 0):

And a subroutine I can call to switch back to problem state (and "user" key):

Load the PC Routine

I used the LOAD macro with the GLOBAL=YES option to load the PC routine into the pageable Common Service Area (CSA), where every address space can get to it through a single, common entry point address.

Reserve a System Linkage Index

I used the LXRES macro to reserve an extended system linkage index. This 8-byte index is effectively the "PC number" that I will pass to the PROGRAM CALL (PC) instruction from my stub, to invoke the PC routine. The last byte of this index is the "entry table" index (EX) for the PC routine I'm calling. LXRES sets the last byte to zero. Given my entry table (discussed next) will only contain a single PC routine definition, this is already the correct EX value.

I'm not going to dive into the minutia of PC numbering, linkage indices, entry tables and the like. If you are really interested (or can't sleep), consult page 10-99 of the POP.

Here's how I reserve a system linkage index:

The 8-byte extended linkage index is returned in lxlist[1] and lxlist[2].

Create an Entry Table

I use the ETDEF macro, first to create a "template" entry table with a single "slot" for a PC routine definition:

The above code is awkward because of limitations of embedding Assembler into LE C. I end up with an entry table copied into a local variable, etparm, with the length of the table in local variable, etplen, and the offset of the first (and only) PC routine slot in local variable, etpc1off.

From here, I use ETDEF TYPE=SET to load in the entry point address of my PC routine and provide the launch settings for the PC:

Equally important as the parameters you can see, notably SSWITCH=NO (meaning that z/OS will invoke the PC routine in the caller's address space), are the parameters you do not see:

• The absence of EK (or EKM) parameters means that the PC routine is invoked with the PC caller's storage key, not key 0 (for example).

• The absence of the ASCMODE parameter means the Address Space Control (ASC) mode defaults to PRIMARY, meaning that all storage references are presumed to be in the PC caller's address space. I'm not doing any recording of call statistics (for example) in my started task, so I don't need to reference storage in the started task's address space from inside my PC routine.

Invoking the PC routine under the PC caller's storage key is vital to system integrity. This prevents the PC routine from reading or writing to storage that the PC caller is not authorised to access. If a PC routine is entered in a system key (0 to 7) or subsequently sets the PSW key to 0 (say), it is the responsibility of the PC routine to check that its caller is not trying to hoodwink it into reading or writing to "protected" storage. If you have ever wondered what the IBM z/OS Authorized Code Scanner is all about, it dynamically scans for PC routines (and other system routines) that are not properly checking that their callers are passing memory addresses to which the caller has legitimate access.

If you want to explicitly check that your PC routine is being passed legitimate memory addresses by its callers, you can/should use the MVCDK and MVCSK instructions (see POP pages 10-69 and 10-75, respectively) to access the addresses passed by the caller under the caller's storage key.

My PC routine is entered under the caller's storage key and I never set the PSW key, so any improper address passed by the caller will produce a "protection exception" (system code 0C4 - affectionately known as a "sock four"), thereby enforcing system integrity.

Finally, having prepared a suitable entry table in local storage, I "create" the entry table in the system using the ETCRE macro, which returns a "token" which is how the entry table is referenced in subsequent steps:

Connect the Entry Table to the Linkage Index

I use the ETCON macro to "connect" the entry table to the system linkage index, so the PROGRAM CALL instruction issued by the stub(s) can locate the launch settings for my PC routine:

Record the PC Number

At this point, the PC routine is registered and ready to be called. What remains to be done is to provide a mechanism for the stub(s) to locate the 8-byte PC number. z/OS provides "system name/token pairs" as a fast, lightweight mechanism for sharing small (16-byte) "tokens" of information between address spaces.

I use the IEANTCR service to create a name/token pair to hold the PC number:

Wait Around Until Told to Terminate

I use the EXTRACT macro to locate the ECB that will be posted when my started task is told to stop (e.g. by way of a STOP operator command). Then I use the WAIT macro to suspend my started task until it is told to stop.

Once the ECB is posted, I unwind the registration process (in reverse order)...

Delete Name/Token Pair

I use the IEANTDL service to delete the name/token pair. The stub(s) need to be able to deal with the situation that the name/token pair cannot be located.

Destroy the Entry Table

I use the ETDES macro with PURGE=YES to destroy my entry table and disconnect it from the system linkage index. A return code of 4 is expected, indicating that the entry table had one or more connections to linkage indices, which have now been disconnected.

Free System Linkage Index

I use the LXFRE macro to free the system linkage index.

Unload the PC Routine Module

Finally, I use the DELETE macro to unload the PC routine from CSA.

A typical PROC for this started task would look like this:

The Stub

The role of the stub is to provide a callable service or API which a problem state caller (e.g. a COBOL program) can invoke to perform the privileged operation inside the PC routine. The stub uses the IEANTRT service to retrieve the extended linkage index of the PC routine and then issues the PROGRAM CALL instruction. If the stub is invoked in 64-bit addressing mode, it allocates a work area for the PC routine in 31-bit memory to keep RACROUTE happy (as discussed above).

The PCKMO instruction looks in GR0 for an integer "function code" which defines the operation to be performed. GR1 holds a pointer to the parameter block for this function (see the POP for details). This stub also returns a count of clock ticks elapsed and provides a separate return code to indicate the success, or otherwise, of the mechanics of calling the PC routine. A number of things could go wrong, as indicated by the error codes I defined:

Testing

I used REXX to invoke the stub to test my PC routine:

Once the PC was successfully loaded by the started task and my user ID was granted READ access to MACHO.PCKMO in the FACILITY class, this REXX produced the following output:

So my 128-bit DES key was successfully wrapped by the LPAR wrapping key and returned in the first 16 bytes of the parameter block.

If I issue a STOP command to the started task, then repeat this test, I see:

As expected, a return code of MACERR_NOTREG (16) is returned, indicating that the PC routine was not registered (i.e. the stub could not retrieve the name/token pair containing the system linkage index).

Source Code

The full source code for the three components described above can be found here: Macho

• machopc.c holds the source for the PC routine

• macpcreg.c holds the source for the started task which registers the PC routine

• pckmo.c holds the source for the stub