ISO 10160/10161-1 Review White Papers

Optional Messages

• Joe Zeeman (Group Leader)
• Barbara Shuh

Version 1 and version 2 of the protocol both include a number of protocol messages whose delivery to the partner system is declared to be optional, even though the services themselves are required to be revoked. These optional messages have led to significant complications in the protocol in order to accommodate their potential absence, and have been a major contributing factor in the perceived complexity of the protocol from an implementation point of view. This white paper will seek to address the following questions.

What is the original rationale for the optional messages?

When the precursor to the ISO ILL standard, the Canadian ILL protocol was first developed in the early-to-mid 1980s there were no general distributed messaging systems. X.400 was still in the earliest phases of development, and, although the definitions of the basic internet mail protocol STMP and the message envelope defined in RFC 822 had been issued, deployment was limited to a very small number of university research establishments. Such messaging as was available was offered by a small number of national or regional commercial services, such as the Envoy 100 offered by a consortium of Canadian telephone systems or as adjunct to other large-scale shared systems, such as the OCLC or RLIN shared cataloguing systems. Access to theses systems was through terminal access via low-speed, connections, typically 1200 bps or worse.

These systems typically charged both the sender and the recipient of a message, and the charges were very high ( in the case of Envoy, some 6-7 times the cost of traditional postage). There was therefore a strong incentive for both the sender and the recipient to minimize the number of messages exchanged.

A second factor, operationally more important, than the economic one, lies in the observation that many ILL departments were, and remain, incapable of invoking the services to which the optional messages correspond. A library might, for instance, route an item in response to an ILL request directly from the shelf to the shipping office and not retain the fact that the item had been shipped. Without a change in operational procedures, this library would be unable to invoke the Shipped service and thus to send the Shipped message. This situation is true for all the optional messages - some ILL departments are simply unable to invoke the service without major upheaval to longstanding operational procedures. Such changes would be likely to increase costs through staff time needed to reorganize procedures and also through the continuing need for additional staff involvement in invoking the service. An increase in costs through implementing the protocol is clearly a disincentive, and optional messages were introduced to allow these libraries to continue existing practices while formally being able to claim compliance with the protocol.

In practice, of course, such formal conformance is unhelpful, since the service is not invoked and a status query directed to such a library would be likely to receive a misleading response, such as that the request is still pending, even though the item has, in fact been shipped.

Has the environment changed sufficiently (since 1982!) to invalidate the original rationale?

It is quite clear that the messaging environment has changed beyond recognition since the early 1990s. Internet mail is, at an institutional level, ubiquitous and the communication cost of sending or receiving a message is effectively nil.

It is our view that the argument that some systems will be unable to claim conformance with the standard in the absence of optional messages is spurious. Even if the messages remain optional but are not sent, such systems are in fact non-conforming because they are unable to invoke the service, regardless of whether the message is sent or not. Support for the service is mandatory in the protocol, and a system that is incapable of invoking the service is by definition non-conforming.

It must be pointed out, however, that even in systems that routinely do invoke the services, operator error may mean that a mandatory service is not invoked. Robust systems will be able to accommodate the occasional failure of a service to be invoked. It should also be possible to define protocol procedures whereby partners in a transaction can resynchronize each other in the event of such operator error. The current state tables enable such resynchronization.

What requirement is there for backward compatibility?

We see no backward compatibility issues in making the optional messages mandatory. The messages are already defined and systems are required to support them.

Does there still need to be any accommodation for lost out-of-sequence messages?

The issue of accommodating lost and out-of-sequence messages is independent of the issue of supporting optional messages. In fact, optional messages are accommodated using the identical mechanisms for lost and out-of-sequence messages. As long as there is a requirement to support transmission on potentially unreliable networks, there will be a need to cope with lost and out-of-sequence messages.

How would the state table look if there were no optional messages?

Since lost and out-of-sequence messages will still need to be accommodated, eliminating optional messages would result in very few state table changes.

Options for version 3:

1. Leave the messages optional. There seems little merit in this, since little is gained if the service is invoked but the message is not sent. The economic argument for not sending messages has disappeared.
2. Make the messages mandatory but make no other changes. This will not require a new protocol version, since the bits on the wire will be identical. If messages are mandatory, the request-optional-messages and responder-optional messages are redundant, but their presence causes little harm, since systems will be free to ignore them in all cases. This is a minimal change option.
3. Make the messages mandatory and make the requester-optional-messages and responder-optional-messages parameters optional. This will necessitate a new protocol version, since the bits on the wire are no longer identical. This option seems to offer few advantages over either option 2 or option 4.
4. Make the messages mandatory and remove the requester-optional-messages and responder-optional-messages parameters. This will necessitate a new protocol version that removes redundant, and thus possibly confusing, information.
5. Make the messages mandatory, remove the parameters and recast the state tables based on the assumption that all mandatory services will be invoked (and the delivery channel is reliable). This will necessitate a new protocol version. It will also require the introduction of error handling to accommodate occasions when operator error leads to services not being invoked or messages fail to be delivered.