Software Evaluation

Many steps are involved in selecting a software system from identifying the need to upgrade an existing system versus purchasing a new one, to narrowing the list of potential vendors, and to making the final selection. What could be considered as the most vital step in the process is the onsite vendor demos. At this stage, vendors have the opportunity to present their wares in a face-to-face setting showing all the fancy bells and whistles their system offers. But are all those bells and whistles important to your particular needs? Given a free form platform, vendors tend to give the exact same presentation to each potential client.

A few steps to consider prior to scheduling demos include: narrowing the list of vendors to 3-4, preparing scripts for vendors to follow during the demo, preparing an expected results document to compare with results presented during the demo, and preparing a scoring matrix for evaluation purposes.

Preparing a script for vendors to follow and defining expected results will help put each vendor on a more level playing field by presenting what you need to see and not just what each vendor wants you to see. It also gives you a more structured presentation tailored to your needs, as well as consistency among the presentations. With that said, there are still a number of factors to consider during the demos. Vendors tend to focus on their strong points and attempt to steer away from weaker areas. Keeping the following topics in mind will help attendees ensure they are presented with the appropriate level of software functionality:

1. Don’t just tell me, show me - If a vendor isn’t able to show a software feature to their audience’s satisfaction, they may just say that the software can handle the function without actually showing how it works. Hold them to task and ask them again to show it.

2. Compatibility with existing processes - Providing a “day in the life” script for presenters to follow during the demo may flesh out instances where the software handles a process somewhat differently than existing requirements.

3. Presentation Flow - If the presenter is following the scripts provided and encounters a topic where the software is weak, he may try to speed through that topic or avoid it altogether.

4. Version creep - Make sure you are aware if the vendor is presenting the demo on the same version offered for installation at your firm. Vendors often present functionality in a beta version of an upcoming release not yet available to its customers.

5. System navigation - It may be a good idea to ask presenters to walk through the screen navigation process before jumping into system functionality. The audience may become distracted or confused during the demo if the presenter is jumping between screens without the audience understanding how.

6. Look and feel versus information access - Some systems have a great look and feel but may not provide viable options for loading information into it or may provide limited options for retrieving data. Ease of access to information should be addressed in the scripts including presenting multiple options for retrieving data.