










Background Reading
ILL Protocol Interoperability Testing
Test Beds
For More Information,
contact
the ILL Application Standards Maintenance Agency,
Library and Archives Canada
ill_asma @lac-bac.gc.ca
Last Update: 2002/09/13
|
Interlibrary Loan Application Standards Maintenance Agency
ILL Protocol Implementation Interoperability Test Suite
The BASIC group of test cases have been reproduced at the end of this document to illustrate the introductory text. However, the bulk of the actual test suite of 69 tests and related constraint declaration tables is not available electronically. For Implementors planning to interoperability tests, NLC is willing to make a printed copy available upon request.
For more details on how to obtain a copy of the NLC Interoperability Test Suite,
Introduction
The objective of the interoperability testing is to verify that the implementations can interoperate, that services supported by the implementations can be invoked when communicating with other implementations.
This ILL Interoperability Test Suite, designed by the National Library of Canada in the early 1990's for implementations which had successfully completed ILL Conformance Testing for conformance to the ISO ILL Protocol Standard (ISO 10160/10161) and the ILL Interim Canadian Standardized Profile, will verify interoperability between implementations at a basic level.
Coupled with conformance testing, interoperability testing can provide a good indication of interworking between implementations. Note that neither conformance testing nor interoperability testing can guarantee that implementations will interwork under all conditions. Implementors should also consider some type of Beta testing, where users exercise the implementations under normal operating conditions for a certain period.
This document provides a description of the test suite structure and the use of Tree and Tablular Combined Notation (TTCN) for specifying the test cases. For details on the ILL Interoperability Testing process and the execution of the interoperability test cases, consult the document Ill Protocol Implementation Interoperability Testing
Test Suite Structure
The test suite structure is illustrated in the tables on the following pages.
|
Group |
Group |
Group |
Cases |
|
|
INTEROP |
REQUEST |
BASIC |
CASE100
CASE101
CASE102 |
|
NONRET |
CASE110
CASE111
CASE112
CASE113
CASE114
CASE115
CASE116
CASE117
CASE118
CASE119
CASE120
CASE121
CASE122
CASE123 |
|
OPTAPDU |
CASE130
CASE131
CASE132
CASE133
CASE134
CASE135
CASE136
CASE137
CASE138 |
|
RETURN |
CASE150
CASE151
CASE152
CASE153
CASE154
CASE155
CASE156 |
|
FORWARD |
CASE170 |
|
INTEROP |
RESPOND |
BASIC |
CASE200
CASE201
CASE202 |
|
NONRET |
CASE210
CASE211
CASE212
CASE213
CASE214
CASE215
CASE216
CASE217
CASE218
CASE219
CASE220
CASE221
CASE222
CASE223 |
|
OPTAPDU |
CASE230
CASE231
CASE232
CASE233
CASE234
CASE235
CASE236
CASE237
CASE238
|
|
RETURN |
CASE250
CASE251
CASE252
CASE253
CASE254
CASE255
CASE256 |
|
FORWARD |
CASE270 |
|
RESPOND1 |
FORWARD |
CASE370 |
The test suite is divided into three major test groups:
REQUEST: This group is defined for the implementation playing the requester role.
RESPOND: This group is defined for the implementation playing the responder role.
RESPOND1: This group is defined for the implementation playing the final responder role during forwarding.
The REQUEST and RESPOND groups are further subdivided into the following groups:
BASIC: This group consists of three test cases for testing basic communication. All exchanged APDUs use minimal data. Executing these cases will verify that the implementations can communicate with one another.
NONRET: This group contains the test cases for testing services required to support non-returnable items. Each test focuses on one or two specific services for which the implementations must supply data for all supported parameters.
OPTAPDU: This group consists of test cases that focus on the use of optional APDUs. The cases involve transactions of non-returnable items and consider only the use of the SHI and RCV optional APDUs.
RETURN: This group contains the test cases for testing services required to support returnable items. Each test focuses on one or two specific services for which the implementations must supply data for all supported parameters.
FORWARD: This group contains one test case for testing the services involved in forwarding. Again, all parameters supported by each implementation should be filled for the services being tested.
The test group RESPOND1 is composed of only one test group, FORWARD. It has been defined for the final responder role when testing forwarding.
The test cases defined for the above test groups are related. When an implementation plays the role of the requester, another implementation plays the role of the responder. Therefore, each test case in the REQUEST test group has a corresponding test case in the RESPOND test group. In the case of the FORWARD test group, three cases (Case170, Case 270 and Case 370) are required to define the actions for three separate roles: the requester, the forwarding responder, and the final responder. Thus the FORWARD group has been defined in three major groups: REQUEST (for the requester case), RESPOND (for the forwarding responder), and RESPOND1 (for the final responder). Three testing parties are required to complete the Forward test cases
To permit association of the test cases, the numbering of the cases has been defined as follows:
The test cases for the requester (REQUEST group) start at 100
The test cases for the responder (RESPOND group) start at 200
The test cases for the final responder (RESPOND1 group) start at 300
That is, CASE100 defined for the requester is associated with CASE200 of the responder, CASE101 with CASE201, CASE 102 with CASE202, etc. When an implementation acts as the requester and executes test case CASE100, another implementation (its partner) acts as the responder and executes test case CASE200. The cases for testing forwarding involve cases CASE170 (REQUEST group), CASE270 (RESPOND group) and CASE370 (RESPOND1 group).
For a complete list of the test case associations, see the ILL Interoperability Test Plan.
Thirty-four (34) test cases have been defined for the requester role and responder role each. Adding the single case defined for the final responder role, the test suite contains a total of 69 test cases. An implementation should take approximately 3 to 5 days to complete testing, taking into account some minor problems with the use of store and forward communications.
TTCN for Specifying Test Cases
The Tree and Tabular Combined Notation (TTCN), defined by ISO 9649, is used for specifying the test cases. This notation consists of Dynamic Behaviour Tables for Specifying the dynamic Behaviour of the Implementation Under Test (IUT); and Constraint Tables for specifying constraints placed on the data used within each event.
Dynamic Behaviour Tables
The main purpose of a Behaviour Table is to specify the test events occurring during a test case. It is composed of the header that identifies a test case and specifies its purpose and the behaviour description that specifies the test case events.
Dynamic Behaviour Table Header:
The header of the Dynamic Behaviour Table provides a test case reference and a test case purpose. The test case reference is constructed from test group names and the case name for the test case. It reflects the position of the test case within the test suite.
For example: test case CASE100 has the reference INTEROP/REQUEST/BASIC/CASE100. This reference show that the case belongs to group BASIC (for basic testing), which part of the group REQUEST (Groups for the requester role), which in turn is part of the group INTEROP (root of the interoperability test suite.)
The test case purpose identifies the purpose of the test case and also provides special instructions for executing the test case. In the CASE100 example, the purpose indicates that the case focuses on the simplest transaction for a non-returnable item. It also instructs the IUT operator to use only mandatory data that is to send the smallest possible APDU.
Test Case Events: A test event consists of:
- Invoking a request service such as the ILL-REQUEST request;
- Transmitting an APDU, such as transmitting the ILL-REQUEST APDU, as the result of invoking a request primitive;
- Receiving an APDU from a partner implementation such as receiving the SHIPPED APDU;
- Getting an indication primitive, such as the SHIPPED indication, as the result of the reception of an APDU;
- A local action such as the time-out expiry.
Abbreviations used to specify the possible test events in the DYNAMIC Behaviour Tables are as follows:
|
Requester
Request and Indication Service Test Events |
|
Abbreviation |
Test Event |
ILLreq
FWDind
SHIind
ANSind
C-REPreq
RCVreq
RCLind
RETreq
CHKind
DUEind
RENreq
REAind
LSTreq
LSTind
DAMreq
DAMind
MSGreq
MSGind
STQreq
STQind
STRreq
STRind
EXPind |
ILL-REQUEST request
FORWARD-NOTIFICATION indication
SHIPPED indication
ILL-ANSWER indication
CONDITIONAL-REPLY request
RECEIVED request
RECALL indication
RETURNED request
CHECKED-IN indication
OVERDUE indication
RENEW request
RENEW-ANSWER indication
LOST request
LOST indication
DAMAGE request
DAMAGE indication
MESSAGE request
MESSAGE indication
STATUS-QUERY request
STATUS-QUERY indication
STATUS-OR-ERROR-REPORT request
STATUS-OR-ERROR-REPORT indication
EXPIRED indication |
|
Responder
Request and Indication Service Test Events |
|
Abbreviation |
Test Event |
ILLind
FWDreq
SHIreq
ANSreq
C-REPind
RCVind
RCLreq
RETind
CHKreq
DUEreq
RENind
REAreq
LSTreq
LSTind
DAMreq
DAMind
MSGreq
MSGind
STQreq
STQind
STRreq
STRind
EXPind
expiry |
ILL-REQUEST indication
FORWARD-NOTIFICATION request
SHIPPED request
ILL-ANSWER request
CONDITIONAL-REPLY indication
RECEIVED indication
RECALL request
RETURNED indication
CHECKED-IN request
OVERDUE request
RENEW indication
RENEW-ANSWER request
LOST request
LOST indication
DAMAGE request
DAMAGE indication
MESSAGE request
MESSAGE indication
STATUS-QUERY request
STATUS-QUERY indication
STATUS-OR-ERROR-REPORT request
STATUS-OR-ERROR-REPORT indication
EXPIRED indication
Time-out expiry (local action) |
|
Abbreviations for ILL APDUs |
|
Abbreviation |
APDU Type |
ILL
FWD
SHI
ANS
C-REP
RCV
RCL
RET
CHK
DUE
REN
REA
LST
DAM
MSG
STQ
STR
EXP |
ILL-REQUEST APDU
FORWARD-NOTIFICATION APDU
SHIPPED APDU
ILL-ANSWER APDU
CONDITIONAL-REPLY APDU
RECEIVED APDU
RECALL APDU
RETURNED APDU
CHECKED-IN APDU
OVERDUE APDU
RENEW APDU
RENEW-ANSWER APDU
LOST APDU
DAMAGE APDU
MESSAGE APDU
STATUS-QUERY APDU
STATUS-OR-ERROR-REPORT APDU
EXPIRED APDU |
Points of Control and observation (PCO)
Each of the test events defined in the previous section occurs conceptually at a point of control and observation (PCO). A PCO refers to software or communication interfaces where the events take place. Three such points have been defined for the test suite:
IUT: This PCO represents the interface between the ILL Application Service Element (ASE) and the service user. It is at this point that the request and indication test events occur.
L: This PCO represents the interface between the ILL ASE and the communications service. It is at this point that the transmission and reception of APDUs occur.
LF: This PCO also represents the interface between the ILL ADE and the communications service. But with the LF PCO, APDUs are sent to and received from a different mail address. It is used when testing forwarding, that is, when the test case involves more than one partner (a different PCO refers to a different partner.)
Specifying a TTCN Test Event:
The complete description of a test event consists of identifying a PCO, one of the abbreviations for either a service or APDU, and the direction in which the data is flowing.
"!" specifies that data is flowing from the implementation, that is, when the test event involves the transmission of an APDU or a request service primitive.
"?" specifies that data is flowing to the implementation, that is, when the event involves the reception of an APDU or an indication service.
The possible formats for test event specification are:
<IUT!service> for requester service
<IUT?service> for indication services
<IUT!local action> for local actions
PCO!APDU for transmitting an APDU
PCO?APDU for receiving an APDU
Following are examples for TTCN test events:
<IUT!ILLreq>: requester test event in which the ILL-REQUEST request service is invoked
<IUT?RCVind>: responder test event in which the RECEIVED indication occurs
L!CHK: responder test event in which the CHECK-IN APDU is transmitted
L?RET: requester test event in which the RETURNED APDU is received
These events are arranged in the Behaviour Description column of the TTCN Dynamic Tables. Notice that the test event specifications are indented from one another. This indentation specifies events that occur sequentially, that is, follow one another in time.
In the test CASE133, an number of events have the same level of indentation starting with the events L?SHI and L?ANS. Arranging the test events in this fashion creates alternative branches that can be selected. In this case, the responder can either send the SHI APDU or if it cannot send this optional APDU, must send the ANS APDU (see the corresponding test case CASE233). These alternatives provide two ways of requiring the RCV APDU from the requester. Once a branch has been chosen, the case terminates at the end of the branch.
Events with Optional APDUs: The ILL protocol standard defines optional APDUs for the services SHIreq, RCVreq RETreq, and CHKreq. The initiator of these services need not transmit the associated APDU unless it is required by the remote site. Such a request can be specified in the ILLreq service for the SHI and CHK APDUs, and in the SHIreq and/or ANSreq services for the RCV and RET APDUs.
The test cases have been specified as if the optional APDUs are transmitted. This will not necessarily be the case during interoperability testing. Implementations may only transmit the optional APDU under certain conditions. For example, the optional APDU may only be transmitted when required by the remote site. An implementation may not even be capable of transmitting optional APDUs.
To allow for the various possible scenarios, when executing an event where the optional APDU is not transmitted, the IUT operator should invoke the STRreq service. The remote site receives an STRind indicating that the service has been invoked and the optional APDU has not been transmitted.
For example, the test event specifications:
|
RESPONDER SITE |
REQUESTER SITE |
<IUT!SHIreq>
L!SHI
|
L?SHI
<IUT?SHIind>
|
becomes
|
RESPONDER SITE |
REQUESTER SITE |
<IUT!SHIreq>
<IUT!STRreq>
L!STR
|
L?STR
<IUT?STRind>
|
when the SHI optional APDU is not provided by the responder. The alternatives could have been included in the test specifications. This would have rendered the case specifications lengthy and complex. To keep them simple, the cases were specified as if the optional APDUs were transmitted.
It is highly recommended that the implementations require that optional APDUs be transmitted. This ensures that the execution of cases will closely resemble the test case specification. Of course, requiring an optional APDU from an implementation that does not support it is not recommended.
Running the APTAPDU Test Group:
Since optional APDUs are defined for commonly used services, this test group was defined to verify the interoperability of implementations when different requirements were placed on the transmission of these optional APDUs. An implementation should always be capable of receiving the optional APDUs. If the implementation supports the transmission of optional APDUs, it should provide them when required by the remote site.
The test cases specified in the OPTAPDU test group focus on the SHI and RCV optional APDUs. In the purpose of each case, a note indicates if the requester should require, desire, or place no requirements for the SHI APDU; and similarly if the responder should require, desire, or place no requirements for the RCV APDU. This is done by setting the value for requester-SHIPPED parameter in the ILL APDU to require, desire, or neither for the requester; and setting the value for responder-RECEIVED parameter in either the SHI or ANS APDU to require, desire or neither for the responder. Notice that constraint formal parameters are used in the dynamic tables to set values for these parameters.
Test cases are selected from this test group based on the support of the optional APDUs and the capability for making requirements for optional APDUs by all partners in a test party.
Constraint tables
For the test events involving request or indication services, a constraint reference is specified. This reference can be used as an index into the constraint tables. A constraint table is a means of constraining parameter values used when invoking services. Much flexibility is given to the operator to specify test data. An exact parameter value is specified only when it is required to achieve the test purpose. These tables consist of ASN.1 value definitions that specify the values expected for the APDU parameters. Values can consist of actual values as defined in the ILL Protocol Standard or one of the following wildcards:
? A valid value must be present
* A value need not be present, but if one is present, it must be valid
OMIT No value must be present
Formal parameters are used to define values for constraints in the dynamic tables. For example, the constraint reference BSANSind000(unfilled) in test case CASE102 is used to specify the value "unfilled" for the transaction-results parameter of the ANS APDU. Formal parameters are used in the test suite to reduce the number of constraint tables (the same table can be used for different test events) and to bring attention to important parameter values in the dynamic tables.
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/REQUEST/BASIC/CASE100
Identifier: T0
Purpose: To check basic interconnection
Simplest transaction (non-returnable item) If possible, use only mandatory data in the APDUs.
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
<IUT!ILLreq>
L!ILL
L?SHI
<IUT?SHIind>
<IUT!RCVreq>
L!RCV
|
BSILLreq000
BSSHIind000
BSRCVreq000
|
|
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/REQUEST/BASIC/CASE101
Identifier: T1
Purpose: To check basic interconnection
Simplest transaction (returnable item) If possible, use only mandatory data in the APDUs.
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
<IUT!ILLreq>
L!ILL
L?SHI
<IUT?SHIind>
<IUT!RCVreq>
L!RCV
<IUT!RETreq L!RET>
L?CHK
<IUT?CHKind>
|
BSILLreq001
BSSHIind001
BSRCVreq001
BSRETreq000
BSCHKind000
|
|
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/REQUEST/BASIC/CASE102
Identifier: T2
Purpose: To check basic interconnection
An ILL-Answer unfilled.
If possible, use only mandatory data in the APDUs.
If possible, requester should request a service not supported by the responder.
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
<IUT!ILLreq>
L!ILL
L?ANS
<IUT?ANSind>
|
BSILLreq001
BSANSind000(unfilled)
|
|
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/REQUEST/OPTAPDU/CASE133
Identifier: T20
Purpose: Testing use of optional APDUs.
The formal parameters in the constraints indicate the parameter value for placing requirements on the optional APDUs, that is,
- value for requester-optional-messages.requester-SHIPPED in the ILL-REQUEST
- value for responder-optional-messages.responder-RECEIVED in the SHIPPED and ILL-ANSWER.
Requester desires the SHI APDU and the responder requires the RCV APDU.
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
<IUT!ILLreq>
L!ILL
L?SHI
<IUT?SHIind>
<IUT!RCVreq>
L!RCV
L?ANS
<IUT?ANSind>
<IUT!RCVreq>
L!RCV
|
BSILLreq004(desires)
BSSHIind004(requires)
BSRCVreq000
BSANSind006(will-supply, requires)
BSRCVreq000
|
|
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/REQUEST/FORWARD/CASE170
Identifier: T33
Purpose: Testing forwarding transactions.
Forward ILL request (non-returnable item).
Formal parameters in the constraints indicate to where the APDU must be sent or from where it should be received.
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
<IUT!ILLreq>
L!ILL
L?FWD
<IUT?FWDind>
L?SHI
<IUT?SHIind>
<IUT!RCVreq>
L!RCV
|
BSILLreq006(RESP1)
BSFWDind000(RESP2)
BSSHIind006(RESP2)
BSRCVreq003(RESP2)
|
|
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/RESPOND/BASIC/CASE200
Identifier: T34
Purpose: To check basic interconnection
Simplest transaction (non-returnable item) If possible, use only mandatory data in the APDUs.
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
L?ILL
<IUT?ILLind>
<IUT!SHIreq>
L!SHI
L?RCV
<IUT?RCVind>
|
BSILLind000
BSSHIreq007
BSRCVind000
|
|
Test Case 201
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/RESPOND/BASIC/CASE201
Identifier: T35
Purpose: To check basic interconnection
Simplest transaction (returnable item) If possible, use only mandatory data in the APDUs.
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
L?ILL
<IUT?ILLind>
<IUT!SHIreq>
L!SHI
L?RCV
<IUT?RCVind>
L?RET
<IUT?RETind>
<IUT!CHKreq>
L!CHK
|
BSILLind001
BSSHIreq008
BSRCVind001
BSRETind000
BSCHKreq002
|
|
case202
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/RESPOND/BASIC/CASE202
Identifier: T36
Purpose: To check basic interconnection
An ILL-Answer unfilled. If possible, use only mandatory data in the APDUs.
If possible, the requester should request a service not supported by the responder.
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
L?ILL
<IUT?ILLind>
<IUT!ANSreq>
L!ANS
|
BSILLind001
BSSHIreq007(unfilled)
|
|
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/RESPOND/OPTAPDU/CASE233
Identifier: T54
Purpose: Testing use of optional APDUs.
The formal parameters in the constraints indicate the parameter value for placing requirements on the optional APDUs, that is,
- value for requester-optional-messages.requester-SHIPPED in the ILL-REQUEST,
- value for responder-optional-messages.responder-RECEIVED in the SHIPPED and ILL-ANSWER.
Requester desires the SHI APDU and the responder requires the RCV APDU.
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
L?ILL
<IUT?ILLind>
<IUT!SHIreq>
L!SHI
L?RCV
<IUT?RCVind>
<IUT!ANSreq>
L!ANS
<IUT!SHIreq>
L?RCV
<IUT?RCVind>
|
BSILLind004(desires)
BSSHIreq011(requires)
BSRCVind001
BSANSreq012(will-supply, requires)
BSSHIreq011(?)
BSRCVind001
|
|
|
Test Case Dynamic Behaviour |
|
Reference: INTEROP/REQUEST/FORWARD/CASE370
Identifier: T68
Purpose: Testing forwarding transactions.
Forward ILL request (non-returnable item). Final Responder
Defaults Reference:
|
|
Behaviour Description |
Constraints Reference |
Verdict |
L?ILL
<IUT!ILLind>
<IUT!SHIreq>
L?SHI
L!RCV
<IUT!RCVind>
|
BSILLind007
BSSHIreq008
BSRCVind001
|
|
|
ASN.1 ASP Constraint Declaration |
|
ASP Name: ILLreq
Constraint Name: BSILLreq000
|
|
ASN.1 Value |
Constraint |
|
|
|