GPO Shipping List Load

Prebulk Configuration File | Duplication Detection Profile | Bulk Import Rules

Each week, usually on Thursday, we receive a file of GPO Shipping List records from MARCIVE. These preliminary records are loaded into our database as suppressed bibs and mfhds. Once the items are received, the Government Documents staff adds items and unsupresses both the bibs and mfhds.

The GPO Shipping List records are run through three preprocessing programs before being loaded to production via Bulk Import. The first program is a Java program written by one of our programmers to replace the prefix tmp which precedes the source's system id number in the 035 |a field with a (GPO)tmp prefix.

The Java program is stored on deleon: /m1/voyager/yaledb/local/GPO/tagchangernew.jar. The script for running this program is as follows:

From deleon:/m1/voyager/yaledb/local/GPO:

$ java -jar tagchangernew.jar (input) (output)

(input) - where the input file resides, typically, /m1/bulk/GPO/(file name)

(output) - where the output should be stored - ../local/GPO/gpoship071204.tag

Next the file is run through Prebulk in order to create an interleaved file of bibs and mfhds for loading. The Prebulk script is run from deleon: /m1/voyager/yaledb/sbin as follows:

$ Pprebulk -i ../local/GPO/gpoship071204.tag -o ../local/GPO/gpoship071204.pre -c ../local/SCRIPTS/gpoship.cfg.

The -i flag signifies where the records are stored before Prebulk (having already been run through the Java program).
The -o flag signifies where the records will be stored after Prebulk (add the .pre extension to avoid confusion)..
The -c flag signifies where to find the correct configuration file for processing the GPO Shipping List records.

The last preprocessing program, addmfhdindicators.jar, is another Java program developed by our Programming group to add a second indicator to the mfhd. To run this program, the AddMFHDIndicators.ini file found in deleon:/m1/voyager/yaledb/local/GPO must be changed to reflect the input path and output path for the file of records we want changed.

Input_File=/m1/voyager/yaledb/local/GPO/gpoship071204.pre Output_File=/m1/voyager/yaledb/local/GPO/gpoship071204.mfhdind

The program is then run from the same directory on the server (m1/voyager/yaledb/local/GPO) with the following command: runaddmfhdind

Once the preprocessing is done, the files are ready to be loaded to production via Endeavor's Bulk Import program. IPS sends an e-mail to ITS letting them know that the files have been preprocessed and are ready to be copied to degama and loaded using the following Bulk Import script:

From degama: /m1/voyager/yaledb/sbin:

$ Pbulkimport -f ../local/GPO/gpoship071204.mfhdind -i GPOSHIP -o GPOSHIP -m

The -f flag signifies where the preprocessed records are stored.
The -i flag signifies which Bulk Import rule will be used.
The -o flag signifies the name of the load under which the record was added. This will be the "operator" found in the bibliographic record history tab when viewing the record in the cataloging module.
The -m flag signifies that both bibs and mfhds will be loaded.

Once the files have been loaded, ITS will notify ISP that the job has run and will include the names of the log files that contain the job output. ISP then checks the output stats for the load to see that it has loaded as expected and checks some records via the ProdOrbis cataloging module to assure the records have loaded correctly.

Once ISP is satisfied that the records have loaded correctly, the output stats are sent to Government Documents so items can be attached and records unsuppressed.

Return to Database Loads Page