During the evaluation I've extracted the following methodology which may be useful for future evaluations:
- Community support - how much support is provided by the community
- Access to the latest code - whether the up to date code is available to the community
- Documentation - how extensive is the documentation if any
- Coding Standards - any well established open source software should have coding standards and guidelines for development.
- Development team - it is important to determine the size of the development team and the number of contributors to determine the adoption
- User interface - how intuitive is the user interface to enable the adoption and eventually the success of the solution
- Functionality - does it cover the requirements and level of sophistication (simple is better)
- Security - how secure is the solution according to the current standards. In case of web application solution how much it covers the OWASP (Open Web Application Security Project) and WASC(Web Application Security Consortium) guidelines to cover all the latest security aspects and to be able to pass ISO certification if requiredImplementation programming language - to determine the skills required, security level, software robustness (strongly typed language are in general more robust) etc.
- Technologies - analyze used technologies to determine their quality
- Contemporary methodologies and technologies - does the solution uses the latest methodologies and technologies.
- Adoption - how many success stories from well known organizations
- Build methodology - how good is the documentation and how easy is to perform a build
- Debug - how easy is to debug this software
- Learning curve - how easy is to learn existing implementation
- Scalability - how scalable is the solution
- Testing coverage - how much testing coverage has the solution
- Responsiveness - how performant is the solution using performance tools
- Architecture - determine the architectural quality of the software, how many tiers has the application and how decoupled are various components
- Open issues - determine the amount of open issues (critical and high priority) and how contemporary are the issues (to determine if there is real support for the software). It is important to determine if there is an issue tracking system.
- Versions/Releases - how many versions per year and how many versions in the last year (to make sure the software is in an active state), latest stable release
- Installation - analyze installation process to determine how easy is to install
- Operating System - platform independence
- Browser compatibility - in case of web application whether or not all the popular browsers are supported
- Licensing - this is a very important aspect in case this will be used as a commercial solution
- Pricing - some of the open sources solutions offer services, software modules for a certain price in addition to the open source solution.
- Maturity - how mature is the software, for how log has been released (first release date)
- API/SDK - does the software provide means to extend the existing functionality without touching the existing code.
- Forum - is there any forum for this software to address existing questions
- Roadmap - is there any roadmap for the software
- Version control system - which version control system is used if any
- Software maintenance utilities - are there any utilities to simplify maintenance
- Visible problems - how many issues discovered during the software trial
- Language - determine the extent of language support if this is necessary
- Code quality:
- Error handling - level of sophistication, detail and how well is done
- Comments - how extensive is the code commented if any
- Class/function size
- General Code Smoke Test - does the code build correctly? Execute as expected? Is it understandable?
- Resource Leaks - is allocated memory freed? Are objects released more than once
- Control Structures - are loop ending conditions accurate? No unintended infinite loops?
- Performance - do recursive functions run within a reasonable amount of stack space? Is blocking system calls used?
- Reinvents the Wheel -does the code recreate some function that exists in a library included in the code base (or perhaps something from a utility library)
- Certification program - is there any certification program
- Commercial manuals - whether or not there are commercial manuals available
- Online help - whether or not it provides help online
- Users conference - whether or not community organizes conferences for user
- Reliance on non-open source software - determine if it requires to function with other software which is not open source (can be a database).
No comments:
Post a Comment