Criteria Spreadsheet (revised)

Category of Requirement Components  
Product Implementation Services and Resources Conversion services and other Libraries experience with them:  
  Are conversion services available ?  
  What are the vendor-specific measures-of-success  
  How many NOTIS conversions ?  
  Project planning and resource scheduling  
  Implementation timeframes  
  Provide examples of timetables for conversion/implementation  
  What factors have been most responsible for causing timetable disruptions ?  
  Implementation personnel  
  How many are librarians ?  
  For the following categories, how many staff do you have and how are they deployed during a typical implementation?  
  Software Consultants  
  Hardware Consultants  
  Database Consultants  
  How does the vendor measure the success of implementation ?  
Product Support Services and Resources Availability and variety  
  Provide a general description of the levels and degrees of support you offer.  
  What support services are available during normal business hours (Eastern time)?  
  What support services are available 24 hours a day, 7 days a week year around?  
  What quality assurance measures ensure that support services are available as promised?  
  Delivery Modes  
  How is support available ? (Phone , WWW , Email , On-Site)  
  How are these modes monitored to detect and avoid saturation ?  
  How does the vendor measure success of these modes ?  
  Flavors (intended for end-users vs. intended for tech staff)  
  How does the vendor triage incoming requests for help and info ?  
  How does the vendor measure success of these practices ?  
  Depth and breadth of expertise  
  How many product support staff does the vendor employ?  
  What professional background and training do they have?  
  How are they deployed to be most responsive in diverse environments?  
  Is any one (or more) staff assigned to oversee and ensure the quality of support services for individual libraries? If so, who might be assigned to the Yale library, and to what other libraries has that person been assigned? Can you provide a resume for this person ?  
  Problem tracking & problem escalation mechanisms  
  What problem tracking and resolution mechanisms does the vendor employ?  
  What measures of success does the vendor use for its problem resolution services?  
  User groups  
  What user groups does the vendor sponsor; how long have they been in existence; are there bylaws, etc?  
  Are there user groups not sponsored by the vendor with which the vendor works collaboratively?  
  What range of product support and development activities is conducted through user groups?  
  Vendor staff assigned to user groups  
  What staff are assigned to these user groups?  
  What professional background and training do they have?  
  Are they deployed in a way that is particularly responsive to the needs of large research libraries?  
  Customer roles (if any) in the following user group management:  
  Setting user group agendas  
  Conducting meetings  
  Reporting to the vendorís senior management  
  What measures of success does the vendor use for its user group activity?  
  What is the vendorís record of success by these measures?  
  Quality assurances and warranties  
  What warranties does the vendor make to its customers?  
  Have customers requested particular warranties that the vendor has not provided?  
  What procedures does the vendor use for interpreting its warranties, when some provision of the warranty is disputed?  
  What means have been employed by either customers or the vendor to resolve disagreements?  
  Is there any history of litigation between vendor and customer regarding product performance?  
  Is there any history of litigation between vendor and customer regarding contract compliance?  
  Have vendor and customer ever mutually agreed to cancel a contract? Why?  
Product Development Services and Resources User groups (see above under Product Support Services and Resources)  
  New releases  
  What kinds of assistance does the vendor provide during implementation of new releases ?  
  Testing and QA of new releases  
  What testing procedures does the vendor employ in developing new functionality?  
  How fully are the results of these procedures shared with customers?  
  What specific quality standards does the vendor use for deciding that new software is ready for release?  
  What are the common causes for failures, or the common ingredients for success?  
  Pace and Timing  
  What considerations inform the vendorís decisions about the pace and timing of new software releases?  
  How frequently does the vendor issue new software releases?  
  Has this frequency varied significantly over the last three years?  
  Announced vs. Actual dates of delivery  
  How does the vendor measure its success in meeting its goals for on-time delivery of new releases?  
  History of patches and fixes following releases (three-year snapshot)  
  How many patches/fixes were required to correct or improve the operation of new releases?  
  What software design measures ease upgrades and minimize disruption to library services?  
  How much down time have libraries experienced in bringing up each new releases over the last three years?  
  Did the vendor provide estimates of the required down time?  
  How did customer experience compare with vendor estimations for down time?  
  What relationships exist with companies selling/developing information technology products (or any sort) and/or information content of likely interest to ARLs?  
  What strategic purposes inform these vendor-to-vendor partnerships?  
  What partnerships does the vendor have with individual customers?  
  What strategic purposes inform these vendor-to-customer partnerships?  
  How does the vendor measure the success of its partnerships with customers?  
  Have any (vendor/vendor or vendor/customer) partnerships been strikingly successful or unsuccessful?  
Vendors Business Complexion Staffing
  How many staff does the vendor now employ?†  
  How many are librarians ?  
  How many of these staff are assigned respectively to:†(as a number, a percent of total, or a ratio)  
  Product Implementation  
  Product Support  
  Product Development  
  Business Management  
  Other tasks  
  How (if at all) has the vendorís staffing patterns varied significantly in the last three years?  
  What was the average gross pre-tax income of the vendor over the last three years?  
  Are there circumstances making this average a misleading statement of the companyís ability to generate income over the last three years?  
  Does the vendor price software maintenance and product development separately?  
  If so, on what principles does the vendor distinguished between the two?  
  How much of the vendorís income over the last three years was attributable respectively to:  
  Initial System Sales  
  Product Implementation fees  
  Product Support Services fees  
  Product Development fees (i.e., charging for new releases)  
  Other activities  

Last Updated on 7/18/00
By Brian Kupiec