Evaluating, selecting and implementing a SaaS or a Cloud based application goes beyond just the application functionality. This is just 50% of the equation or value proposition. Looking into the other technology aspects of the SaaS vendor is very very important such as Security, Compliance, Data, Identity Management, Integration, Standards, Support, Hosting facility, SLAs etc.
Business has a lot of urgency and temptation to select and implement SaaS applications. As IT we need to help them in choosing the right solution that meets both business and technology requirements. This is where IT can partner with the Business to provide additional value in evaluating and selecting the right SaaS vendors that meets both business and technology objectives. By doing this you can avoid the pitfalls of business selecting and implementing SaaS applications outside of IT.
Following are the Top 12 questions & requirements you will need to ask SaaS and Cloud vendors. This is based on my experience in evaluating, selecting and implementing 12+ SaaS applications and 3 cloud environments over the past 4 plus years.
**See related detailed post on SaaS Vendor Evaluation and Selection Process – Framework, Reference architectures, SaaS identity Management)**
**See Cloud ROI framework from Forrester**
1. Hosting Provider & Data Location
- Who is the hosting provider?
- Where is the hosting location? Country, State?
- What type of infrastructure is used? Hardware, software, operating system, technology platform?
- Ask for the architecture diagrams for all layers? Business, Application, Integration, Data & infrastructure layer diagrams?
- Where is the primary data being stored? In order to comply with local jurisdiction, privacy and regulation requirements
- Where is the backup data being stored?
- What type of virtualization software is used? VM Ware, Hyper-V?
- What type of network bandwidth is available (min 100 Mbps) ? What options are available for dedicated bandwidth?
- What type of scalability is provided for additional computing power – CPU, RAM, Storage? Costs? Time to implement?
2. Data Access, Security, Segregation & Encryption
- Is it a dedicated or a shared environment?
- If it a shared environment, how is the data segregated from other shared environments?
- What type of data architecture is implemented? Diagrams?
- How is security managed in the shared environment? What controls are in place?
- Who has access to the infrastructure, hardware, software, data? Ask for specific info on the roles & responsibilities of administrators, profiles, hiring practices etc
- What application & data access audit logs are available? How often can you get this?
- How is the primary data encrypted? What encryption schemes are used? Who has access to the decryption keys? How often is this tested?
- How is the backup data stored? Is the data in raw files or encrypted format? What locations are the backup data stored? Who has access to this backup data?
- What type of investigative support is provided in cases of breach?
- Is the vendor is acquired, sold or dissolved? What options are available to get the data? Costs? How is the data wiped out of the environment?
3. Regulatory Compliance
Business is responsible for the data security, integrity and privacy even if it is a SaaS app that is management by the vendor.
- What types of regulations are being followed complied?
- PCI & HIPPA compliance? What options are available? How is this managed?
- How often is this audited?
- How is this enforced?
- Ask for availability & access to the audit reports on a regular basis
4. Hosting Facility Security & Compliance
- Is the hosting facility SAS 70 II (Statement of Auditing Standards) compliant? This is an important requirement as this encompasses all security and regulations compliance?
- How often is this compliance audited?
- Auditing and compliance is just 50% of the requirement. It is important to find out how the hosting vendor actively enforcing SAS 70 II controls is & requirements in to their work processes. Ask for this info in detail.
5. Business Continuity & Disaster Recovery
- What type of business continuity & disaster recovery options are available? Is this part of the standard services?
- Where are the DR (disaster recovery) data centers locations located?
- What type of infrastructure exists to replicate and synchronize data between the primary and DR data centers? Is this available in real-time, daily?
- If the primary environment is down? How quickly can the DR environment be made active either in the primary or the DR data center?
6. Identity Management, Security & Single Sign-On
- What type of identity management solution is provided? (See related post on Identity management)
- Is Single Sign-On (SSO) provided? What types of SSO options are available? SAML, HTTP-Fed, Open Auth etc?
- Can the SaaS app be integrated with an existing Identity Management system?
- What type of user store is available? Can this user store be integrated with Active Directory or any other user store database?
- What type of user security, authentication and authorization options are available?
7. Standards, Policies, Procedures & Frameworks
- What architecture and technology standards, policies and procedures do you follow and comply?
- What architecture frameworks do you follow? TOGAF?
- How do you manage the projects internally? Agile, PMP?
- What type of professional services do you offer to implement and support the SaaS application? What type of PM resources do you have? Skills, experience, certifications etc?
8. Integration, APIs & Reports
- What type of APIs and web-services are available to pull and push data?
- Are the APIs secured and encrypted?
- Is there an option to access the data directly from the database?
- What type of reports can be generated or created?
9. Support & Maintenance
- What type of support is provided? Self-service, email, phone?
- What are the support times? 24×7, 5 days a week?
- What are the support response times? Critical, Urgent, High & Low issues/requests?
- Who provides the support desk and where are they located? How many employees is part of the support desk? Dedicated or shared with projects?
- Is there a premium support model?
- What type of monitoring and alerting does the vendor provide?
- What type of migration and integration support does the vendor provide?
- Is there a dedicated support manager and account rep?
- How do you support and manage integration with the customer’s existing SaaS apps?
- How are upgrades, patches and other maintenance performed?
- What type of change management & risk management procedures do you follow? How often is this communicated to the customers?
- Does the customer have any control on applying patches, upgrades and changes to the SaaS app? (this is very important to know especially if the SaaS app is integrated with other SaaS apps).
10 Service Level Agreements (SLAs)
- What SLAs are available – reliability, availability, performance, issues, requests etc? Penalties?
- What types of credits are available if SLAs are not met?
- Are the terms & conditions of the contract tied to the SLAs?
- Is the exit strategy tied to the SLAs?
- Is there a regular meeting (monthly/quarterly) to review the SLAs, issues, requests?
- Who will be part of the SLAs meetings?
- How are the issues escalated if the SLAs are not complied? Who can we escalate to in the management team?
11. Vendor Management, Product Road-map & Viability
- Who is the management? What is their experience?
- Is it funded by a VC firm? Who is it?
- What are their financials? How many customers do they have?
- What is the organization structure? How many employees? Where are they located?
- How many employees are there in the product development & support team?
- What is their product road-map and strategy?
- How are they managing their product strategy? Competition, Market, Positioning, Customer requirements? How is this communicated and how often?
- How do they accommodate customer requirements into their product strategy? Is there a customer advisory council?
- Do you provide a trial or proof of concept for your product including new features?
12. Pricing & Contract
- After reviewing the above 11 items, the prices may vary from your initial analysis/requirements (this always happened in my case!!). Understanding the true pricing is very important.
- What is included and excluded in the pricing? Will you charge for new product features?
- Are you open to contract negotiations that meet the company legal needs & requirements?
- What is the minimum contract period? Are there any discounts for long-term contract? Is there an option to exit during the contract and what are the terms & conditions, penalties?
- See sample SaaS legal addendum
By doing the above, IT can play a valuable partner with business to evaluate, select and implement SaaS or Cloud based applications.