IPIG
ILL Protocol Implementation Test Beds
For More Information,
contact
the ILL Application Standards
Maintenance Agency,
Library and Archives Canada
Last Update:
2001/02/19
Interlibrary Loan Application Standards Maintenance Agency
Communication Protocols for Circulation Information, including Patron Information
Consolidation of Postings to the IPIG Listserv,
Friday, 16 January - Monday, 02 February 1998
Mark Wilson posted the following message: I had a brief conversation at ALA with Randy M. of ALS about patron authentication. Although each of us will have some proprietary issues and the whole issue is well beyond protocol issues scope, it may be useful to get a subgroup together to discuss the issue on the list.
Art Zemon asked: Well *O* *K* then! :-) What did you talk about at ALA? I have my own thoughts about how to do patron authentication and would *really* like to have it be compatible with you guys. That way the ILS vendors only need to produce one sort of "server" and they can work with all of us ILL vendors.
Randy Menakes asked: Can you provide some specifics on the issues of patron authentication you would like to tackle on the list? Are you thinking about further discussion of Patron Externals (ILL-Supplemental-Client-Info)?
Mark Hinnebusch, FCLA wrote: At the last CNI, Cliff Lynch indicated CNI was going to try to get a bunch of people together to work on the verification problem. Also, Denis Lynch and I worked out something we were going to do between SilverPlatter and FCLA that we were going to generalize. That particular project is waiting on me, but it could be expedited if others have needs now.
We plan on using Z39.50 (what else?) to request patron verification information. The client sends the patron barcode as the search and an element set name of B (brief). The server returns either an 8-byte user id for ERL to use internally for access control/billing or an 80-byte error message for display to the patron. If the client specified F (full record) then the sever sends a GRS-1 record containing the data items that are defined in the draft ISO patron data elements standard, very similar to Z39.63(?) I believe.
Obviously, the brief record is very specific to our project and needs to be generalized. We also need to define other ways to search for the patron than just a barcode. In the case of the full patron record, we need some access control to protect the data from rogue clients and to meet Buckley amendment requirements (I'm sure other countries have similar privacy requirements).
I have a **very rough** draft of the GRS record for the full patron record and also a first shot at a patron data attribute set for searching. They can be cleaned up without too much trouble.
Ian Ibbotson wrote: Paul Harvey and Myself (Fretwell-Downing) are about to start work on an EC project "PRIDE", People and resources identification for a Distributed environment. I think we would be V. interested in whatever discussions are taking place wrt patron authentication. If a Sub-Group were created, would it be possible to make the deliberations open to a wider community? I Think there is a PRIDE web site, Paul can give details (pharvey@fdgroup.co.uk), but basically PRIDE is looking at technologies like LDAP, X.500, WHOIS++ & SET wrt Z39.50 and ILL applications. People interested in more info?
Mark Wilson wrote: Judging from the response to my original posting, there appears to be wide-spread interest in the topic of patron identification and authentication, and, with Mark H.'s comments, some cross-protocol issues. I would like to take a shot at outlining what I feel to be the issues. As usual, I expect Jo and Ruth to gently correct me and tell me what it is I really meant to say.
I think we are attempting to define what it means to be an IPIG-profile compliant ILL system. Everything below is directed toward that end.
Patron authentication should be a non-issue in mediated systems. The only time we care is if a PIR [Patron-initiated-request] service is active. Even PIR is a non-issue in a consortia using local library systems built by the same vendor and becomes an issue only when such a requests escape to other systems. One solution would be to forbid exporting PIRs without mediation. It certainly seems an unfair burden to force another system to authenticate PIRs from your system.
Unfortunately, one does not have to look far to find situations for which this model will not work. Consider a consortium composed of several local library systems with a centralized ILL office -- a common construct here in the United States. Thus, we may expect our customers to demand some means of dealing with authenticating patrons from other systems.
Randy of ALS showed me his proprietary means of patron authentication for libraries using an ALS system. I expect that most local systems will develop some sort of proprietary id-ing system. The real key will be how to expose each system. Mark H. and others offered Z39.50 as an avenue. Although it makes sense to use a database-query protocol to query a patron database, we may need to explore other technologies.
The Z39.50 approach mandates a Z39.50 patron authentication server at the local library system level and a Z39.50 client inside each ISO 10160 system. This seems severe requirement for IPIG-profile compliance. There should be some way for one ILL installation to query another about patron status without imposing the actual means for resolving that query. Some systems might choose to implement a local Z39.50 connection, others might choose some other means for resolving the query and responding to the querying ILL system.
It also occurs to me that this discussion has parallels in the Policy Directory discussions, maybe the two issues should be discussed together. After all 'directory' services can be seen as a flavour of database searching services, and we would not want to end up with having to support two different directory/database architectures if a single one could serve both requirements.
The PRIDE project takes this point of view and is looking at directory services based on directories holding Patron Information and Resources & Service Descriptions (i.e. policy).
I know the UK academic community is also looking into the issue of an authentication system that can be used globally, so patrons can use services of more than one institution.
Thus, the IPIG profile issues appear to be:
- Should the profile even speak to the issue of patron authentication?
- If it does, should all authentication become the responsibility of the patron's sponsoring ILL system?
- If not, should we consider extending ISO 10160 either within an APDU [standard compliant] or by an entire APDU [non-standard compliant] to deal with patron authentication *within* the ILL system? i.e., should we signal that a request is PIR and un-authenticated? Should we provide support for a Patron-Is-OK-Request and a Patron-Is-OK-Reply APDUs?
- If not, should the profile force systems to resort to Z39.50 in order to be IPIG compliant?
Ed Davidson: Yes, judging by the correspondence so far - it should.
Dennis Sherman: No. Or more accurately, not specifically. More below at #2. Automated patron identification is one way in which vendors might differentiate their services in the marketplace. The profile, IMHO, should attempt to ensure different systems can interoperate, but not dictate what a system does within its own domain.
Ed Davidson: In an ideal world this looks like the cleanest solution - but I bet there are lots of exceptions out there.
Dennis: Authenticating the patron should *always* be the responsibility of the requester system, whether that function is done by the software automatically or in some automated manner. The profile should speak to the issue of patron authentication only to make explicit that an ILL-Request APDU should not be sent until and unless the patron data has been authenticated
Ed: Well, I thought we had agreed to borrow the access control PDUs from Z39.50 as a starting point for exchanging username/password between ILL peers.
Dennis: No. The responding system should be able to assume the request contains authenticated patron data, whether the patron initiated the request or not.
Dennis: You can probably guess my answer... the profile should only require that patron data be authenticated before an ILL-Request is sent, and make no statement as to what method is used to authenticate that data.
Further comments from Ed: Although, as a further observation, I can see other cases where an ILL system and a local Library Management System might need to exchange data and possibly functionality with one another. Getting Patron status info is our first glimpse of this. But what about the issuing of returnable items to the patron? or the sending of notices to the patron for recall and overdues? These tasks are already provided by local circulation systems, but they are functions required of requester ILL systems too. It seems unfortunate that the ILL system has to duplicate this circulation functionality, when it would be much neater if it could request these services of the circulation system via a well-defined protocol.
Mark H: So, the issue of "mix-and-match" protocols is perhaps not to be avoided.
But all this is outside of the ILL protocol proper, it's stuff the ILL peers have to do in order to process an ILL request. I think our profile should concentrate on what authentication info needs to be exchanged within a request in order to let the responder process it. Surely, HOW this patron info is processed is way out of scope.
Andrew Braid responded to Mark's options: Regarding Mark's four options as below, my vote is for number 2. It is a personal view and should not be regarded as the official British Library position and it does not cover the position of OINC's (of which I understand there are many in Canada at the present time!)
We have decided that, for the time being at least, in our proposed implementation we will only accept ILL Protocol requests from pre-registered users, which amounts to the same thing as the second choice.
We do receive requests from non-registered users at present but we figure that if they can't be bothered to register they certainly can't be bothered to use the ILL protocol.
Randy Menakes stated Ameritech position: Here are some principles that we at Ameritech Library Services think should guide the IPIG's work on Patron Authentication:
1) It is always the responsibility of the requester to authenticate the patron who is requesting an ILL service - never that of the responder. This is true whether or not the original request was initiated into a system directly by patron (PIR). It's true whether the request went out mediated or unmediated. And it's true whether the organizational environment is consortial or not. If the initiating patron needs to be validated, this is accomplished prior to the ILL-Request APDU being sent.
2) There should be some recognized way for two systems to perform patron authentication. This need applies to much more than Interlibrary Loan services. A library may, for example, want to verify a patron before authorizing the patron to search particular databases. We need a means for one system to query another for patron 'information'. The target system may need to return patron data. Or it may need to return 'approval' for some specific activity, such as 'may initiate a loan request', may initiate a request for a non-returnable copy costing more that $10.00', or 'may search a locally mounted proprietary database'.
Because TRLN couldn't get patron data into our system in a way that would work interactively, we keep duplicate patron data, and update it regularly. Something of an annoyance, but a not unreasonable work-around until real interactive patron authentication is available.
Both of these put the issue of patron authentication somewhat outside of 10160/10161. But patron authentication is certainly a crucial aspect of ILL systems today, and it may be very appropriate for the IPIG to help drive the process.
On 26 January, Mark H. cross-posted a message on a circulation communications to ZIG and IPIG lists: I am cross-posting this [to ZIG and IPIG] as it seems of interest to the two communities.
For the Z list, a little background. The issue first arose of how to do patron identification for ILL. I mentioned that one option that has been raised for non-ILL applications is Z39.50 search of a patron database. A later message mentioned that circ. type functions also need patron data. I mentioned that NISO might again look at a wire protocol for circ transactions. This is the context of this message:
One of the questions that comes up is whether or not a separate circ transactions standard is needed or if using Z39.50 against patron and circulation files is sufficient, given database update extended service. While I tend to lean to a separate protocol, some have suggested the alternative. My concern is that you can model almost anything in the DP world as search/retrieve/update; this argument makes Z39.50 the only wire protocol ever needed. And this worries me.
Anyway, I thought we wanted Z39.50 to be a generalised Search/Retrieve/(Update) protocol rather than leave it as a purely bibliographic retrieval system.
Mark H.: One would guess that ILL could also be modeled as search/retrieve/update in the same way.
Art Zemon responded to Mark H: I would prefer that we use a protocol other than a BER encoded, one, particularly one which has as many inherent vagaries as Z39.50.
For instance, I would prefer to use LDAP for patron identification and something CORBA or DCOM based for updates.
Many people here at FD play the "It's all just database updates" and "You can do everything with Z3950" cards. The truth is, these statements are made with no reference at all to maintainability, portability, or reuse. What you get from technologies like CORBA and DCOM is a model, which people can implement, understand and enhance. These technologies were invented to realise message based object models. Why not use them? Going even deeper, I think a world where the ISO-ILL Vertical Application Framework is available from the OMG is an attractive one. I think there is a lot more chance of a protocol being accepted by the wider (Non libraries) community if it's based on CORBA rather than neat BER (Which at the end of the day CORBA uses). Not to mention that I think there is a helluva lot more chance that OS vendors will get ORB's into their products than per-protocol BER codes.
Mark H also responds to Art: The question of CORBA is a red herring for this discussion. The question is what services are needed. I can (and probably will since others have not) do a CORBA implementation of Z39.50. So, if I must, I will paraphrase my question to:
If I have a CORBA implementation of the services provided by Z39.50, do we need a CORBA implementation of a "intersystem circulation" set of services or are these services realizable with the CORBA implementation of Z39.50 provided services?
Barb Shuh asked Mark H.: What is "intersystem circulation"? How does it differ from "Interlibrary Loan"?
Ed Davidson responds to Barb's query: My take on this is that there are stand-alone ILL systems (Like VDX) that would like to use Circulation system functionality (like validating User Status, issuing returnable items to patrons (requester) or issuing returnable items to institutions (responder)) and which currently have to replicate large chunks of Circulation system functionality. However, it would really be preferable to ask an existing Circulation system to do these functions on the ILL systems behalf.
This could mean that my ILL system does not need yet another replication of the patron file (along with the campus' student file and the library systems patron file) - I can get all my patron info from a central source.
Other uses for a client/server circulation protocol could be to allow library buses to exchange loan/reservation data with the parent library system.
And of course, there are third party standalone issues and return terminals that are currently using proprietary protocols that might also benefit from using a standard.
I am sure there are other uses too.
Mark H responds to Barb's query: A good question. I was trying to distinguish services from implementation techniques and architectural decisions. It may be that they are one and the same. A patron-initiated circulation request from system X to system Y that doesn't pass through an ILL process would mean that the patron's record in system X maintained circulation information about the item in system Y just as though the item was owned by system X. Does that make sense as a distinction from ILL? If not, it may be that the ILL protocol is a superset of functionality.
Dennis replied (2 February, 1998): We looked at both ways, and came to the conclusion that, since the patrons *could* hop in a car and drive to one of the other libraries to borrow an item themselves, the reciprocal borrowing method was essentially the same thing, except the library was delivering to them. The reciprocal borrowing agreement the TRLN libraries have had in place for years to allow the "drive across town" delivery method provided sufficient stick.
We decided that "reciprocal borrowing" and "interlibrary loan" really were differentiated by who an item is checked out to (and once that is known, a number of other things are affected, e.g. who is responsible for overdue, recall, and renew). If the item is checked out to an ILL office, it is interlibrary loan, while if it is checked out to a patron, it is reciprocal borrowing. We use our Document Delivery System (DDS) to pass an ILL-Request containing patron data, and the lender's DDS adds a patron record to their circulation system if needed.
Barb responds to Mark H., Ed D, Art Z.: From your answer, Mark H., it seems that the proposed circ. protocol could handle the communication between the borrowing (circ) system and a lending (circ) system. To me, that actually falls within the definition of 'Interlibrary Loan".
Dennis adds a contribution to the discussion: Our DDS has components that automate searching for candidate items to fill a request, and checking the availability. (And some if it is a real hack right now... working, but sure is ugly, and will be until some of the standards issues we're discussing are resolved by *someone*!!) In coming up with the architecture we decided on, we went in circles for quite a while.
What we finally decided is that the whole design is a lot cleaner if you make the automated system quite similar to current manual processing. There is a component at the requester that determines (via Z39.50) if a requested item exists in a target OPAC. If it does, (along with some other decisions) an ILL-Request is sent. At the responder, there is a component that checks the availability of the item -- circ status, lending policies, and so on. If that responder component determines the item is not available, it returns an ILL-Answer automatically.
What this breaks down to is:
- the requester looks around to find a place they want to borrow an item from, and send a request for it.
- the responder determines whether the thing can circulate or not.
We found that trying to build intelligence into the requester to determine circ status at the responder was much too complex.
And as I've been reading all this and thinking about it, I think this is outside the scope of IPIG. We can all agree that it is a good idea for there to be a standard way to get circulation information from a system, and push and encourage (and participate) in some other group solving the problem. But whether and how an ILL system communicates with a circ system should be a vendor-dependant issue, and is another area in which systems can compete in the marketplace. I don't think it has any real effect on a profile for the ILL protocol.
Hmmm, I think there's something odd about me (a non-commercial developer) pointing out places where vendor systems can compete... :-)
Whereas, if I understand what Ed (and Art) are thinking of something different; their interest is to establish the protocols for communication between standalone circ systems and ILL systems within the same institution (more probably the lender, but could also be the borrower).
Are these two visions compatible?
Art continues the conversation with Mark H.: I think the real red herring is the encoding method versus the protocol. Z39.50 mixes the two by specifying BER for the encoding.
You are right. I do not have any real objections to Z39.50... other than the BER encoding and the uncertainties of what services can actually be *obtained* from any given target. ISO 10160/10161 suffers somewhat the same problems <sigh>. But this isn't news to anyone.
I am simply proposing that, at least for obtaining patron information, we look at another well known standard: LDAP. This may prove sufficient and easier to deal with.
The mention of 3Ms SIP (and I know that checkpoint, for instance, has implemented this on a TCP/IP stack) looks very promising as many vendors may have to implement that protocol anyway.
Ed Davidson also comments on LDAP: I thought LDAP used BER encoding too - albeit a lightweight version that didn't encode strings. Have things changed since I last looked at LDAP (about 18 months ago)?
MarkH comments on LDAP: I have no objection to doing LDAP if it meets the need. This whole string started because I reported that I was planning on doing patron verification with SilverPlatter. We both have Z39.50 software, so doing Z39.50 is much easier for us than rolling LDAP (I speak for FCLA; I'm sure SP can speak for themselves). My guess is that this situation is true for most, if not all, of the major ILS vendors; they have Z39.50 in their systems, they don't have LDAP. None are doing CORBA in production systems and while many think it is something to work toward, I know of only one ILS vendor who is publicly talking about OR technology at all.
So, it is a pragmatic issue, I think.
Art Z comments on LDAP: I don't know if asking large universities to make a massive change to their LDAP directory once per term is unreasonable.
Perhaps we could put together a test LDAP directory and load it up with 100,000 records (or 200K or 300K or whatever) and run some tests to simulate the change activity in such an environment. I'll volunteer to try that later this spring if you or anyone else is interested.
I do agree that LDAP is inappropriate for circ stuff. That is where I would like to see a CORBA object brought into existence. I think I have seen other messages later in this thread on the value of CORBA vs. BER protocols.
Ian Ibbotson adds a comment on LDAP: WRT LDAP... I was saying to Ruth the other day...I was able to locate someone who previously worked for Fretwell-Downing but moved on to read for a PhD at Sheffield Hallam Uni, using their public LDAP service. I can't imagine this institution being that far out in front.... Does anyone actually know how many Uni's already have an LDAP/X500 service (with annual updates etc)?
I think that an ILL system needs to execute a very few operations against the integrated library system's (ILS) patron database.
- Is the given patron ID (barcode number, user name, etc.) valid?
- Is the patron blocked from submitting any new requests?
- What is the patron's type (undergraduate, staff, homebound, etc.)?
- What text should appear on a mailing label? (Note that to print a mailing label, the ILL program does *not* need to know all of the components of a patron's address! The program simply needs a single pre-formatted text string with embedded line breaks.)
- What is/are the patron's phone number(s), preferably in the order they should be called?
- What is the patron's email address?
- Given a password, is it correct?
Performing a Z39.50 search and getting back a patron record in an exhaustive format like Barb's actually makes this job more complicated and more error prone. It means that each ILL system must code the algorithms to answer the above questions correctly. And the ILL algorithms had better match the ILS algorithms or we will all have angry customers!
Since virtually all ILS systems already have this logic inside them, why not expose these methods in a usable manner, rather than exporting the raw data?
Ron Burns (CPS) contributed: Once again we've a fair amount of experience integrating our ILL system with Circulation systems.
One option, which you might not wish to totally overlook, is 3M's Standard Interface Protocol (SIP). Their proposed version two accomplishes many of the required tasks, and we've proposed a set of extensions as well.
SIP has at least one thing going for it... (version 1 at least) has already been implemented by most vendors.
True, it's not an ASN.1, BER encoded monster that'll take a year to implement... That's not always a BAD thing! :)
Thorn Roby provides a 3M contact: My latest version (2.4) of the 3M protocol came from ggmarsolek@mmm.com (3M Library Systems Laboratory, Safety and Security Systems Division, (612) 733-9708).
Barb Shuh provides more extraneous details on SIP: To aid the
IPIG discussion on Patron Information, I prepared a comparison of the applicable
data elements used in the draft NISO Z39.81, The 3M Standard Interchange
Protocol and the proposed (now registered) IPIG external. You might want
to look at this as a preview to the actual 3M standard. (A list of the
applicable 3M data elements start on page 20.)
URL: http://www.lac-bac.gc.ca/iso/ill/document/readings/patronin.pdf
Ron Burns supplied info on 3M standard.: The word is: 3M does not have the info on a web site at the moment, though I suspect we've given them something to think about there.
Until then, send a message to library@mmm.com asking for the SIP2 specification, and include some delivery information (DDI?), and they will send it right out.
MarkW: In the past, access to circulation info was only of theoretical interest to the automation community. The time may have come for the combined ILL/Z39.50 community to approach this issue in a formalized manner. I for one would be very interested in working with such a group.
MarkW wrote, on 27 Jan.: I keep hearing what appear to be disjointed messages about the problem of circ info and some confusion about messages, records, encoding and transport. I believe the issues are:
- How does my ILL application know that it has to ask a question of a circ system? What signals a need to know patron or item status, etc?
- How does my ILL application phrase that question? What is the structure of the record, its data elements, and so on?
- How does my ILL application send that question to a circ/patron system and receive an answer? What circuit connections, query protocol, and network/machine-neutral encoding schema will be used?
- What services are needed? Item there? Patron OK?
- How simple can we make this for local system vendors to implement? Discussions of CORBA and DCOM (distributed application managers), BER (an encoding schema), X500, LDAP, Z39.50, and SIP(Database inquiry protocols) compare apples and oranges.
- ILL is an unconfirmed service that may use store and forward messaging. Anybody want to suggest how we email a query to a circ system?
- Rapid implementation of ISO ILL may flounder on this and the policy-directory issue, both of which are needed to complete the ILL subsystem.
Mark responds to Art: Not quite what I meant :-} ILL systems themselves (as opposed to patrons communicating with ILL Systems) may be store and forward. Ariel already does this, and many document delivery services may be store and forward. To return to a refrain I have used before: store-and-forward MIME makes far more sense than connection-oriented ILL. Thus, I may well have an ILL system that is implemented for store-and-forward only: what does it do to talk to circ systems?
Art replies to MarkW (28 Jan.): I'm not sure if we are talking at cross-purposes or simply misunderstanding each other. Here is the way I envision things. How does this fit (or not) with your vision?
The ILL system may be store 'n' forward with other ILL systems. But the ILL system has a synchronous, real-time connection to its local ILS (circ) system.
When a patron creates a request in the borrowing ILL system, the ILL system makes a synchronous call to the local ILS to verify the patron's validity and authority to create the request.
Then the borrowing ILL system sends the request to the lending ILL system, very probably via a store 'n' forward mechanism.
Finally, the lending ILL system receives the request and processes it. Perhaps it even uses its synchronous connection to its own ILS to cause the circ system to check out the item to the borrowing institution or patron.
I'd add that the borrowing ILL system could also use its synchronous connection to the ILS to cause its circ system to issue the borrowed item to the requesting patron, as well as to send overdues and recalls, fines, charges etc.
Dennis Sherman, responding to a group of messages on circ info: I'm in agreement with Art and Ed on what kinds of communication should happen where. The model Art posted is very much like what TRLN's system does.
MarkH comments, but I'm not sure about what: I would guess they might be compatible but I think we need a much more detailed analysis of the differences and similarities.
