Z39.50/SRU gateway to DBC opensearch
====================================

The system is based on [simpleserver](http://www.indexdata.com/simpleserver),
which, is a Perl module for implementing a YAZ
[Generic Frontend Server](http://www.indexdata.com/yaz/doc/server.html) .

A Debian package, dbc-openserch-gw, is available. But it may
also be installed from source, of course.

Queries
-------

The gateway always makes a conversion from RPN (Type-1) to
DBC's CQL variant. This conversion can be performed in a table-driven
mode or a simple mode (without a conversion table).

### simple mode

In the simple mode, the use attribute, type 1, is used verbatim as
CQL field. The relation attribute, type 2, is mapped to CQL relation
operators, 1 is mapped to <, 2 is mapped to <=, .. , 5 is mapped to >.
However, if structure is phrase (4=2), then the CQL relation, might be
mapped to "adj". See ISSUES later.

Position attribute, type 3, with values  1, 2, 3 are accepted and do not
affect the resulting CQL at this time.

Structure attribute, type 4, with values 1 (phrase), 2 (phrase), 4 (year)
are accepted. Other values of structure result in a diagnostic. As already
mentioned, if structure is 2 (phrase), that alters the CQL relation.

Truncation attribute, type 5, with values 1 (right), 2 (left), 3 (left&right),
and 100 (none) are accepted. Other values result in a diagnostic.

Completeness attribute, type 6, is not checked.

### table driven mode

In the table driven mode, the RPN attributes are matched against a table
which resembles that of the CCL table of YAZ. It maps a combination
of RPN attributes to a CQL field.  It is only enabled
if the use attribute is of numeric type. If the use attribute is of
type string, the simple mode, takes effect.

The CCL table is line based and 1st colum is name of CQL field of OpenSearch.
Column 2 and later are consulted in the query matching . These columns
are separated by white space. Only items contains an equal sign (=)
are used in the attribute check. Other items are ignored.

Example:

dkcclterm.ti            ti   BIB1,u=4     r=3   p=3   s=pw   t=l,r   c=1

dkcclphrase.lti         lti  BIB1,u=4     r=3   p=1   s=1    t=r     c=3

We have two specifications. The use attribute is the same for both CQL
fields. But the selection can be distinguished only from the
completeness attribute. (pw is valid for both s=1, s=3).

See file, bibliotek.dk.fields.txt, for the complete table

### CQL

The gateway also supports CQL for SRU access. The CQL is translated
to RPN in the gateway. The translation is specified in file pqf.properties.

### Issues

1. The structure=phrase is not supported by DBC OpenSearch. The CQL
operator =, seems to perform a boolean and of the terms that follows. The
CQL operator adj have been suggested as an alternative, but that returns a diagnostic from DBC opensearch (Error: No query found in request). Example
[using adj](http://openbibdk.addi.dk/0.8/?action=search&agency=100200&profile=test&collectionType=work&start=1&stepValue=0&objectFormat=marcxchange&query=term.default%20adj%20the%20art) . Example
[using =](http://openbibdk.addi.dk/0.8/?action=search&agency=100200&profile=test&collectionType=work&start=1&stepValue=0&objectFormat=marcxchange&query=term.default%20%3D%20the%20art)

2. Is a CCL field file appropriate for future use?


Searching
---------

The gateway can be accessed via Z39.50 or SRU.

For SRU, PQF or CQL may be specified. As mentioned earlier the CQL
is translated internally to RPN, then to the CQL offered by DBC OpenSearch.

SRU Versions 1.1, 1.2 and 2.0 are supported.

Example search:
[water, 4 marcxchange records](http://pz2.dbc.dk:9994/bibliotek.work?query=water&maximumRecords=4&recordSchema=marcxchange) .
[water, 4 dkbm records](http://pz2.dbc.dk:9994/bibliotek.work?query=water&maximumRecords=4&recordSchema=dkabm) .

### Issues

1. For the work based collection, collection cound is less than the hit count.
This is a problem for client software that fetches all records in a result set.
However, the gateway performs two hints to help client software. Firstly,
the hit count itself, is clearly marked "estimate". Secondly, the
gateway will use present status "Partial" to indicate that the request
could not be fully fullfilled.

2. In other cases and the collection count is greater than the hit count.
In that case, most likely, client software will not get all records in a
result set. Example search: [hygie]( http://openbibdk.addi.dk/0.8/?action=search&agency=100200&profile=test&collectionType=work&start=1&stepValue=10&objectFormat=dkabm&query=hygie) with hit count 4, collection count 5.

Records
-------

The gateway offers records in formats XML and Danmarc. For XML, schemas
dkabm and marcxhange are supported. For XML records are UTF-8 encoded.

For Danmarc, schema/element-set F or none returns records in Danmarc
encoding. Element-set U, returns Danmarc in UTF-8 encoding.

In all cases, for marcxhcange/danmarc first record in marc collection in
first OpenSearch collection is returned.

Record conversions are customized via YAZ' record
[retrieval module](http://www.indexdata.com/yaz/doc/tools.retrieval.html) .
This allows XML conversions via XSLT.

### Issues

1. Some records are not in marcxchange format. Only dkabm. A this
point we return a surrogate diagnostic. But we could make a simple dkabm
to marcxchange conversion if necessary.

2. The old Z39.50 server offered records in USMARC/MARC21. This gateway
do not.

Facets
------

Clients may use DBC OpenSearch facets from both Z39.50 and SRU 2.0.
SRU versions 1.1, 1.2 do not support facets. For Z39.50, facets are
requested and returned in the additionalSearchInfo of the Search
APDU. This is, thus, an extension to Z39.50 and is not generally
supported by Z39.50 client software.

SRU 2.0 example requesting
[facets](http://pz2.dbc.dk:9994/bibliotek.work?query=water&maximumRecords=0&facetLimit=5:subject,5:creator,5:type,5:date)

Equivalent yaz-client example

Z> facets @attr 3=5 @attr 1=subject, @attr 1=creator, @attr 1=type, @attr 1=date

Z> f water

### Issues

1. sort order for facets is not handled yet.

Scan
----

Version 1.15 of this gateway also supports scan. This uses the
opensuggest service..  Currently at
[link](http://opensuggestion.addi.dk/b3.5_2.0/)

The scan mapping is using same table driven structure as is in use for
search.
