Systems Criteria

Number Category QUESTION
  Product Implementation Services and Resources  
1   Are conversion services available ?
2   Describe Quality Control measures for conversion services
3   How many NOTIS conversions ?
4   Provide examples of implementation timetables for conversion/implementation
5   What factors have been most responsible for causing implementation timetable disruptions ?
6   How many implementation personnel are librarians ?
7   For the following categories, how many staff do you have and how are they deployed during a typical implementation: Trainers, Software Consultants, Hardware Consultants, Database Consultants?
8   How does the vendor measure the success of implementation ?
9   What varieties of instructional support (instructor-led, online help, AV modules, etc.) does the vendor provide?
10   Will the vendor provide a record of the actual implementation particulars of this implementation for the customer?
  Product Support Services and Resources  
11   Provide a general description of the levels and degrees of support you offer.
12   What support services are available during normal business hours (Eastern time)?
13   What support services are available 24 hours a day, 7 days a week year around?
14   What quality assurance measures ensure that support services are available as promised?
15   How is support available ? (Phone , WWW , Email , On-Site)
16   How are these delivery modes monitored to detect and avoid saturation ?
17   How does the vendor measure success of these delivery modes ?
18   How does the vendor triage incoming requests for help and info ?
19   How does the vendor measure success of these triage practices ?
20   How many product support staff does the vendor employ?
21   What professional background and training do the product support staff have?
22   How are product support staff deployed to be most responsive in diverse environments?
23   Is any one (or more) staff assigned to oversee and ensure the quality of support services for individual libraries? If so, who might be assigned to the Yale library, and to what other libraries has that person been assigned? Can you provide a resume for this person ?
24   What problem tracking and resolution mechanisms does the vendor employ?
25   What measures of success does the vendor use for its problem resolution services?
26   What user groups does the vendor sponsor; how long have they been in existence; are there bylaws, etc?
27   Are there user groups not sponsored by the vendor with which the vendor works collaboratively?
28   What range of product support and development activities is conducted through user groups?
29   What vendor staff are assigned to these user groups?
30   What professional background and training do the vendor staff assigned to Users Groups have?
31   How are vendor staff assigned to User Groups deployed to meet the needs of the different kinds of member libraries?
32   What roles do customers play (if any) in setting user group agendas, conducting meetings and reporting to the vendor's senior management?
33   What warranties does the vendor make to its customers?
34   What procedures does the vendor use for interpreting its warranties, when some provision of the warranty is disputed?
35   What means have been employed by either customers or the vendor to resolve disagreements?
36   Is there any history of litigation between vendor and customer regarding product performance?
37   Is there any history of litigation between vendor and customer regarding contract compliance?
38   Have vendor and customer ever mutually agreed to cancel a contract? Why?
  Product Development Services and Resources  
39   What kinds of assistance does the vendor provide during implementation of new releases ?
40   What testing procedures does the vendor employ in developing new functionality?
41   How are new release testing results shared?
42   What specific quality standards does the vendor use for deciding that new software is ready for release?
43   What are the common causes for failures, or the common ingredients for success in new releases?
44   What considerations inform the vendorís decisions about the pace and timing of new software releases?
45   How frequently does the vendor issue new software releases?
46   Has this frequency varied significantly over the last three years?
47   How does the vendor measure its success in meeting its goals for on-time delivery of new releases?
48   How many patches/fixes were required to correct or improve the operation of new releases over the last 3 releases?
49   What software design measures ease upgrades and minimize disruption to library services?
50   How much down time have libraries experienced in bringing up each new releases over the last three years?
51   Did the vendor provide estimates of the required down time?
52   How did customer experience compare with vendor estimations for down time?
53   What relationships exist with companies selling/developing information technology products (or any sort) and/or information content of likely interest to ARLs?
54   What strategic purposes inform these vendor-to-vendor partnerships?
55   What partnerships does the vendor have with individual customers?
56   What strategic purposes inform these vendor-to-customer partnerships?
57   How does the vendor measure the success of its partnerships with customers?
58   Have any (vendor/vendor or vendor/customer) partnerships been strikingly successful or unsuccessful? Describe.
  Vendors Business Complexion  
59   How many staff does the vendor now employ?†
60   How many staff are librarians ?
61   How many of these staff are assigned respectively to†(as a number, a percent of total, or a ratio): Sales, Production Implementation, Product Support, Product Development, Business Management, Other tasks.
62   Sales
63   How (if at all) has the vendorís staffing patterns varied significantly in the last three years?
64   What was the average gross pre-tax income of the vendor over the last three years?
65   Does the vendor price software maintenance and product development separately? If so, on what principles does the vendor distinguish between the two?
66   How much of the vendorís income over the last three years was attributable respectively to: Initial System Sales, Product Implementation fees, Product Support Service fees, Product Development fees (I.e., charging for new releases), Other activities.
67   What measures does the vendor take to control the cost to customers of project implementation and product support and development?

Last Updated on 9/6/00
By Brian Kupiec