Vendor Bulk Loads

VENDOR BULK LOADS

Vendor Configuration Files | Vendor Duplicate Detection Profile | Vendor Bulk Import Rules

Sporadically, the Acquisitions Department sends ISP e-mail notifications and the concatenated MARC record file of records from various vendors for a single miscellaneous Bulk Import load. The e-mail notifications verify the original file names and the number of records in the original file sent by the vendors.

After ISP records the file name and record count in the log book, the file is uploaded to deleon in order to run it through Prebulk to strip tags and create holdings records. The Prebulk script is as follows:

From clark: /m1/voyager/yaledb/sbin:

$ Pprebulk -i /export/home/mgarman/SMLBULK/smlbulk071204.txt -o ../local/SMLBULK/smlbulk071204.pre -c ../local/SCRIPTS/smlbulk.cfg

The -i flag signifies where the records are stored before Prebulk.
The -o flag signifies where the records will be stored after Prebulk.
The -c flag signifies where to find the correct configuration file for processing the records.

Once the file has been run through Prebulk, ISP sends an e-mail to ITS letting them know the records have been preprocessed and are ready to be copied to degama and loaded to production using the following Bulk Import script:

From degama: /m1/voyager/yaledb/sbin:

$ Pbulkimport -f /m1/voyager/yaledb/local/(director)/(file name) -i VENBULK -o -ACQRECORD -m

The -f flag signifies where the preprocessed records are stored.
The -i flag signifies which Bulk Import rule is to be used.
The -o flag signifies the name of the load under which the record was added. This will be the "operator" found in the bibliographic record history tab when viewing the record in the cataloging module.
The -m flag signifies that both bibs and mfhds will be loaded.

Once the files have been loaded, ITS will notify ISP that the job has run and will include the names of the log files that contain the job output. ISP then checks the output stats for each load to see that the file has loaded as expected and checks some records via the ProdOrbis cataloging module to assure the records have loaded correctly.

Once ISP is satisfied that the records have loaded correctly, the output stats and list of bib id replaces* are sent to Acquisitions for resolution.

* Although a Duplication Dectection profile is used during Bulk Import, these vendor records are loaded to production unconditionally. Therefore, even if a duplicate record is detected in the database, the incoming record is loaded anyway. The ouput log stats record how many records were detected as "replaces" (even though they were not replaced) and gives the bib id for the record that would have been replaced as well as the bib id for the record that was loaded unconditionally. DPA runs an AWK program against the log file to pull out these ids to send to Acquisitions to resolve.

From degama: /m1/voyager/yaledb/rpt:

$ awk -f /m1/voyager/yaledb/local/AWK/VENDORREPLACE log.imp.20040712.1015 (log name)

Return to Database Loads Page