Updated by MG July 15, 2004
|
Integrated Library Technology Services Web, Workstation & Digital Consulting Services Database Loads Schedule |
CJK LOADS CJK Configuration File | CJK Duplication Detection Profile | CJK Bulk Import Rules Once a month we receive an e-mail notification from OCLC letting us know that our CJK recon Bib records are ready to be picked up from their server. After the file is retrieved and recorded in our log book, it must be run through a series of six preprocessing programs before we ask ITS to load it into production via Bulk Import. The first preprocessing program is for the reformatting of the RLIN 035. The program we use to perform this job is a local Java program, cjkloadernew.jar, developed by one of our programmers. The Java preprocessing programs are stored on clark: /m1/voyager/yaledb/local/MARCHandlers. The DPA person runs the following script from the MARCHandlers directory: java -jar cjkloader.jar (input file) (output file). The input file is the directory on clark where the CJK file is stored. The output file takes the date from the CJK file followed by .cjk and .035 to represent it is a CJK file that has gone through the program to reformat the 035. java -jar cjkloadernew.jar /m1/bulk/CJK/v02m08.ccr.bib.cjk /m1/voyager/yaledb/local/CJK/v02m08.cjk.035 The second preprocessing program is another Java program called tag949buildernew.jar. It is also stored in the MARCHandlers directory on clark. This is another local program developed to move the incoming 049 tag information (needed for building the mfhds) and reformat it so that the call number stamps are put in separate subfields in order to be able to map them to the appropriate subfields in the mfhd 852. The input file for running this script is the output from the previous job (cjkloadernew.jar). The ouput file takes the date from the original file followed by .cjk and .949. From the MARCHandlers directory: java -jar tag949buildernew.jar /m1/voyager/yaledb/local/CJK/v02m08.cjk.035 /m1/voyager/yaledb/local/CJK/v02m08.cjk.949 The third preprocessing program is Endeavor's Prebulk program. We use Prebulk to strip some tags and build an interleaved file of bibs and mfhds. Prebulk is run from clark: /m1/voyager/yaledb/sbin using the following script: $ Pprebulk -i ../local/CJK/v02m08.cjk.949 -o ../local/CJK/v02m08.cjk.pre -c ../local/SCRIPTS/cjk.cfg
Once we have created an interleaved file of bibs and mfhds, a fourth program for adding |2 localyale based on a first indicator 7 in the mfhd, is run. This is another locally developed Java program, addlocalyale.jar, but it is executed differently. This program is also located in the MARCHandlers directory on clark. For this program to run, the AddlocalyaleIn852.ini file needs to have the correct file name and path inserted for both the input and output files using the VI editor. The input file is the output file from the Prebulk job (/m1/voyager/yaledb/local/CJK/v02m08.cjk.pre). The output file name replaces the .pre in the input file with .localyale (/m1/voyager/yaledb/local/CJK/v02m08.cjk.localyale). Then at the $ prompt simply type runlocalyale. When the program is finished, it will produce output stats for how many bibs and mfhds were in the file and how many were updated. The next preprocessing job that needs to be run for the CJK loads is the addmfhdindicators.jar. This file is run in the same manner as the preceeding program. It will add the correct second indicator to the 852 tag in the mfhd, and the correct first and second indicators if there is an 866 tag present in the mfhd. The AddMfhdIndicators.ini file is in the MARCHandlers directory on clark and needs to have the correct file name and path inserted for both the input and output files using the VI editor. The input file is the output file from the previous addlocalyale.jar program /m1/voyager/yaledb/local/CJK/v02m08.cjk.localyale). The output file path and name will also need to be updated, changing the .localyale to .mfhdind. Then at the $ prompt simply type runaddmfhdind. The last preprocessing program to run for this load is the addmultiple866.jar. This is also a Java program. It will add additional 866's tags to the mfhd for holdings that have gaps as long as the incoming record indicates those gaps with a comma and |v. The AddMultiple886.ini file is in the MARCHandlers directory on calrk and needs to have the correct file name and path inserted for both the input and output files using the VI editor. The input file is the output file formt he previous addmfhdindicators.jar program /m1/voyager/yaledb/local/CJK/v02m08.cjk.mfhdind. The outpu path and file name will also need to be updated, changin the .mfhdind to 866. Once the preprocessing jobs are done, the files are ready to loaded to production via Endeavor's Bulk Import program. DPA sends an e-mail to ITS letting them know that the files have been preprocessed and are ready to be loaded using the following Bulk Import script: $ Pbulkimport -f /m1/voyager/yaledb/local/CJK/(file name) -i > YLCJK -o CJKRECON -m After ITS notifies DPA that the job is complete, DPA checks the log stats and looks at some records in ProdOrbis to assure everything has loaded correctly. An e-mail is then sent to Dajin Sun informing him that the records have been loaded. Include a copy of the output stats in this e-mail and bib ids for any records that were replaced.
|