Due Diligence and Ongoing Monitoring

 View Only
  • 1.  Performance Monitoring for Cloud Service Providers

    This message was posted by a user wishing to remain anonymous
    Posted 12-13-2023 05:06 PM
    This message was posted by a user wishing to remain anonymous

    What sorts of KPIs do folks have for cloud service providers?  We have a large number of CSPs as vendors and have been relying upon the service levels listed on their websites. These largely are limited to uptime metrics.  Are there other measurements folks use?



  • 2.  RE: Performance Monitoring for Cloud Service Providers

    Posted 12-19-2023 02:50 PM

    Hi there,

    Because there are so many potential KPIs that can be used, it's important to select KPIs that are most relevant to the specific cloud services you are using. For a more detailed look at KPIs and metrics for cloud services, you might refer to The NIST Special Publication 500-307 Cloud Computing Service Metrics Description. However, to promote discussion on this topic, I have included 10 basic Cloud services KPIs that can be generally used in most organizations.

    1. Uptime percentage: This measures the amount of time that a cloud service is available to users. It is important to track this KPI to ensure that the service is reliable and accessible to users.

    2. Average response time: This measures the amount of time it takes for the cloud service to respond to a user request. A low average response time indicates that the service is performing well and is responsive to user needs.

    3. Cost per user: This measures the cost of providing the cloud service to each user. This KPI helps to track the efficiency of the cloud service and identify areas where costs can be reduced.

    4. User adoption rate: This measures the percentage of users actively using the cloud service. This KPI helps track the cloud service's success and identify areas where user engagement can be improved.

    5. Security incidents: This measures the number of security incidents that occur within the cloud service. It is essential to track this KPI to ensure that the cloud service is secure, and that user data is protected.

    6. Downtime duration: This measures the duration of time that the cloud service is unavailable to users. This KPI is important to track to ensure that the cloud service is stable and reliable.

    7. Backup and recovery time: This measures the time it takes to backup and recover data in the event of a disaster or system failure. This KPI helps to track the effectiveness of the cloud service's backup and recovery procedures.

    8. Capacity utilization: This measures the percentage of the cloud service's capacity that is being used by users. This KPI helps to track the efficiency of cloud services and identify areas where capacity can be increased or reduced.

    9. Service level agreement (SLA) compliance: This measures the cloud service's compliance with the SLA, which outlines the level of service that the provider is expected to deliver. This KPI helps to track the provider's performance and identify areas where improvements can be made.

    10. Customer satisfaction: This measures the satisfaction of users with the cloud service. This KPI helps to track the success of the cloud service and identify areas where user satisfaction can be improved

    I hope this is helpful, but I would love to hear from other members with experience developing and using cloud services KPIs.




  • 3.  RE: Performance Monitoring for Cloud Service Providers

    Posted 12-20-2023 12:19 PM

    Hi Hilary,

     

    Thanks. I'm still in the learning stage about vetting vendors that utilize Cloud. Where would we find the information on these 10 items. In the vendor SOC report?

     

    Sincerely,

     

    Cheryl

     






  • 4.  RE: Performance Monitoring for Cloud Service Providers

    Posted 12-20-2023 12:46 PM

    Hi Cheryl,

    Thanks for the question. It's important to understand that a SOC report helps to validate a vendor's control environment. Although you might find specific details related to your vendor's uptime or response time, keep in mind that these reports are conducted by an independent third party and represent a point-in-time assessment. On the other hand, KPIs are ongoing and are meant to validate the vendor's performance. Some of the KPIs I have listed would be dependent on reports or other data provided to you by the vendor. Others require your organization to compile data both internally and from the vendor.

    ·         These KPIs would typically be validated through vendor-provided reporting.

    ·         Uptime percentage

    ·         Average response time

    ·         Security incidents

    ·         Downtime duration

    ·         Backup and recovery time

    ·         Capacity utilization

    While these KPIS would require your organization to look at internal data such as financial data, users in your organization, contract management reports, etc.

    ·         Cost per user

    ·         User adoption rate

    ·         Service level agreement (SLA) compliance

    ·         Customer satisfaction

    I hope that this information is useful. I would love to hear from other members if they have anything to add here.