A scratchpad for database design, record relationship discovery scripting, and XML design for meta-records to index all related records of any type within a single meta-record.
Provide a functional ILS system to serve as a platform for experimenting with discovering, indexing, and using any record relationships. Provide an efficient design for basic ILS functions when most non-explicit record relationships have not yet been discovered or which may never be specified by choice or limitations of relationship discovery.
Please use the wiki to post your designs, corrections, reasons why it may not work or may not work efficiently.
Use the login button at the bottom of the document to login. If you have not yet registered a login name for this wiki, use the same login button to register.
Please post quickly with a posting prepared in advance so that the wiki editor is not open too long and the page is not locked preventing other users from adding content.
Please include your name and contact information in posting if you want others to know who it was who posted.
In this wiki you can change anything, so please do so in a constructive manner if needed.
Koha is in transition to a series of new major versions with significant design changes to correct many major previous design mistakes and solve all the world's problems. Can an ILS really solve all the world's problems, even if it is free software? Perhaps not but it can come a little closer if you have something to share.
The original design from Katipo Communications and Horowhenua Library Trust in New Zealand used an FRBR like model implemented in SQL. The original Koha model was developed independently of FRBR but had a similar inspiration even if the Koha model muddled and flattened proper bibliographic relationships.
Paul Poulain adapted that design to support MARC. Paul added many fantastic innovations to put Koha ahead of other systems in some respects but attempting to use an SQL database store and index bibliographic records in an SQL database was a major limitation on development and performance.
Currently Koha stores bibliographic and holdings information together in MARC bibliographic records. Indexed bibliographic fields are stored according to an FRBR-like hierarchy. Holdings currently uses an adaptation of the French Recommandation 995 local use field for holdings in both UNIMARC and MARC 21.
Koha has some sophisticated support for searching using indexed reference and tracing fields in authority records contributed by Paul Poulain. A new feature has just been added for browsing broader, narrower, and parallel terms in a subject authority records contributed by Henri-Damien Laurent. However, Koha has historically under utilised authority records because there had been no support for matching authority records with the bibliographic records unless the authority records were originally created in the local Koha records database.
Zebra from Index Data is a textual database being used to store records for the current development version of Koha already implemented at some libraries. Zebra is optimised for storing and indexing structured textual records such as MARC and XML. Zebra functions as a Z39.50/SRU server with queries and indexing limited to a large subset of Z39.50/SRU with sophisticated support for the indexing difficulties presented by some languages.
Koha needs to store holdings and bibliographic information in separate records to most easily transition from the all holdings in one local use field model of Recommandation 995 to holdings using multiple fields such as standard MARC 21 holdings, SUDOC holdings, the recently created UNIMARC holdings format, IBERMARC holdings, etc. Koha needs to index bibliographic and holdings records together with a common key.
Koha also needs to index authority records and bibliographic records together with a common key.
Zebra has no support for combining different indexes based on a common key either most efficiently at the time the indexes are built or at query time in the inefficient SQL model of joining indexes. Additional development of Zebra to support a more flexible indexing design is outside the capacity of the current Koha community to support at this time. Index Data could develop such a feature if funding were available.
Related records can be retrieved from common keys contained in the records after the result set for a query is parsed to find the keys. If a search is needed to constrain the related records, then subsequent searches are required against every related record. Additional searches against each related record takes much too long for large result sets with a large number of related records.
Koha could continue to use the legacy indexing system while an improved indexing system independent of Zebra was developed. That option would loose much of the advantage of Zebra if Zebra was storing the records without providing the indexing.
Fields which need common indexing could be added to all record types. A possible but undesirable solution. That would require the creation of redundant local use fields with any data needing indexing in common for every record type such that every record would contain every record.
Tümer Garip, who has found excellent solutions for much of the Koha implementation of Zebra already, proposed creating meta-records in XML containing both bibliographic and holdings records. The MARC equivalent to this solution has long been used by union catalogues such as MELVYL. Joshua Ferraro extended Tümer's proposal by suggesting adding FRBR model support in the meta-record. Thomas Dukleth extended Tümer's proposal further by suggesting solving all the world's library record management problems with a super meta-record.
All traditional union catalogue record matching work is relevant.
Thomas wrote this and fell asleep before continuing later …