Posted inUS

Protecting Against Cyber Threats: A Guide for Managed Service Providers and Their Customers

No Comments

Tactical Actions for Immediate Security Enhancements

For Managed Service Providers (MSPs) and their clientele, immediate cybersecurity actions are paramount. These critical steps should be implemented today to bolster defenses:

  • Account Audits and Deactivation: Identify and promptly disable any user accounts that are no longer active or in use. This reduces potential entry points for malicious actors.
  • Multi-Factor Authentication (MFA) Enforcement: Mandate MFA for all MSP accounts that require access to customer environments. Rigorously monitor these accounts for any unusual or failed authentication attempts.
  • Clear Contractual Security Responsibilities: Ensure that all MSP-customer contracts explicitly and transparently define the ownership of Information and Communications Technology (ICT) security roles and responsibilities. Clarity in contracts is crucial for effective security management.

These actions are vital in the face of escalating cyber threats targeting managed service providers, as highlighted by cybersecurity authorities across the globe.

The Rising Tide of Cyber Threats Targeting Managed Service Providers

Cybersecurity agencies from the United Kingdom (NCSC-UK), Australia (ACSC), Canada (CCCS), New Zealand (NCSC-NZ), and the United States (CISA, NSA, FBI) have jointly recognized and issued warnings regarding a significant surge in malicious cyber activities aimed at managed service providers (MSPs). This concerning trend is expected to persist and intensify, necessitating urgent and comprehensive action from both MSPs and their customers. [1]

This joint Cybersecurity Advisory (CSA) is designed to provide actionable strategies for MSPs and their customers to effectively mitigate the escalating risk of cyber intrusions. It outlines essential cybersecurity best practices specifically tailored for ICT services and operations. The guidance emphasizes facilitating transparent and crucial dialogues between MSPs and their clients concerning the protection of sensitive data. Organizations are urged to adapt and implement these guidelines to suit their unique operational landscapes, specific security requirements, and in adherence to all relevant regulatory frameworks. Critically, customers of MSPs must ensure that their contractual agreements with service providers incorporate robust cybersecurity measures that align with their individual security needs.

This advisory is the product of collaborative expertise from leading international cybersecurity bodies, including the UK’s National Cyber Security Centre (NCSC-UK), the Australian Cyber Security Centre (ACSC), the Canadian Centre for Cyber Security (CCCS), the New Zealand National Cyber Security Centre (NCSC-NZ), and the U.S. Cybersecurity and Infrastructure Security Agency (CISA), National Security Agency (NSA), and Federal Bureau of Investigation (FBI), with valuable insights from industry partners within the Joint Cyber Defense Collaborative (JCDC). MSPs and their customers should consider this advisory in conjunction with other critical cybersecurity resources, such as NCSC-UK guidance on heightened cyber threat actions, CCCS guidance on cyber security for managed services consumers, and CISA’s Shields Up and Shields Up Technical Guidance web pages. These resources collectively provide a comprehensive framework for enhancing cybersecurity posture in the face of evolving threats.

Understanding Managed Service Providers (MSPs) in the Cyber Landscape

For the purposes of this advisory, Managed Service Providers (MSPs) are defined as entities that are contracted to deliver, operate, and manage Information and Communications Technology (ICT) services and functions for their clients. These services are typically governed by a contractual agreement, such as a Service Level Agreement (SLA). MSPs often integrate their service offerings with those of other providers, creating a comprehensive suite of solutions. These offerings can span across various domains, including platform, software, and IT infrastructure services, business process and support functions, and increasingly critical cybersecurity services.

MSPs commonly manage these services within their customer’s network environment, whether on the customer’s physical premises or hosted in the MSP’s own data centers. It’s important to note that while this advisory focuses on MSPs, it does not specifically address guidance for Cloud Service Providers (CSPs). CSPs cater to customer ICT needs through cloud services like Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), and Infrastructure-as-a-Service (IaaS). However, it is worth noting that MSPs may also offer these cloud-based services as part of their broader portfolio. (For a more detailed understanding, refer to the Appendix for additional definitions.)

The services provided by MSPs inherently require trusted network connectivity and privileged access to and from customer systems. Organizations of all sizes, from large critical infrastructure entities to small and medium-sized businesses (SMBs), rely on MSPs to manage intricate ICT systems, securely store data, and maintain essential business processes. The utilization of MSPs enables organizations to efficiently scale and support their network environments and operational processes without the need for extensive internal staff expansion or in-house capability development. This reliance underscores the critical role MSPs play in the modern digital ecosystem and, consequently, the importance of securing these providers against cyber threats.

Why Threat Actors Target MSPs: A Gateway to Extensive Networks

Whether a customer’s network environment is physically located on their premises or externally hosted, a compromised MSP can serve as a critical initial access point for threat actors to infiltrate numerous victim networks. This vulnerability has the potential for globally cascading cyber incidents. Cybersecurity authorities in the UK, Australia, Canada, New Zealand, and the U.S. anticipate a continued and intensified focus on MSPs by malicious cyber actors, including state-sponsored advanced persistent threat (APT) groups. These actors aim to exploit the inherent trust relationship between providers and customers.

Successful compromise of an MSP can enable a range of damaging follow-on activities. These include ransomware attacks, where critical systems are encrypted and held hostage, and cyber espionage, aimed at stealing sensitive data and intellectual property. These attacks can be directed not only at the MSP itself but also across its entire customer base, potentially affecting a wide array of organizations simultaneously.

Recognizing this escalating threat landscape, cybersecurity agencies from the UK, Australia, Canada, New Zealand, and the U.S. have previously issued general guidance for both MSPs and their customers. [2], [3], [4], [[5](), [6](), [7](), 8] This advisory builds upon this prior work by providing specific, actionable guidance designed to foster transparent and informed discussions between MSPs and their customers. The central focus of these discussions should be on robustly securing sensitive information and data assets.

These dialogues should lead to a critical re-evaluation of existing security processes and contractual obligations, ensuring they are aligned with the customer’s risk tolerance and current threat landscape. A shared, proactive commitment to security is essential to mitigate risks for both MSPs and their customers, and to strengthen the overall resilience of the global ICT community against evolving cyber threats.

Download the Joint Cybersecurity Advisory: Protecting Against Cyber Threats to Managed Service Providers and their Customers (pdf, 697kb).

Key Recommendations for MSPs and Their Customers

Cybersecurity authorities in the UK, Australia, Canada, New Zealand, and the U.S. strongly recommend that both Managed Service Providers (MSPs) and their customers adopt and implement the following baseline security measures and operational controls. Furthermore, it is crucial that customers ensure their contractual arrangements with MSPs explicitly stipulate the implementation of these essential measures and controls. These recommendations are designed to create a robust security framework that protects both MSPs and their clients from evolving cyber threats.

1. Prevent Initial Compromise: Fortifying Entry Points

Malicious cyber actors commonly target MSPs by exploiting vulnerabilities in devices and internet-facing services. They also employ brute force attacks to gain unauthorized access and utilize sophisticated phishing techniques to deceive users. To counter these threats, MSPs and their customers must prioritize and implement robust mitigation strategies. Valuable resources for understanding and mitigating initial compromise attack methods include:

  • CISA’s Shields Up Guidance: Provides comprehensive advice on reducing the risk of cyberattacks.
  • NCSC-UK’s guidance on mitigating malware and ransomware attacks: Offers specific strategies for preventing initial infections.

By proactively addressing these common entry points, organizations can significantly reduce their vulnerability to cyberattacks.

2. Enhance Monitoring and Logging Processes: Early Threat Detection

The time between initial network intrusion and incident detection can often extend to months. To address this critical delay, cybersecurity authorities from the UK, Australia, Canada, New Zealand, and the U.S. advise all organizations to retain their most critical logs for a minimum of six months. Implementing a robust and segregated logging regime is essential for detecting threats early. This can be achieved through comprehensive Security Information and Event Management (SIEM) solutions or by deploying discrete logging tools. For guidance on selecting appropriate data for security logging and its effective utilization, organizations can refer to NCSC-UK’s resource: “What exactly should we be logging?“.

Furthermore, all organizations, whether managing security internally or through MSP contracts, should deploy endpoint detection and network defense monitoring capabilities. These should be complemented by application allowlisting and denylisting strategies to enhance security posture.

Specific Actions:

  • MSPs: Implement comprehensive logging of delivery infrastructure activities used to provide customer services. Log both internal and customer network activity as contractually agreed and appropriately.
  • Customers: Ensure effective monitoring and logging of their systems. When engaging an MSP for these services, contractual arrangements should mandate:
    • Implementation of comprehensive security event management for monitoring and logging of provider-managed customer systems.
    • Provision of visibility to customers regarding logging activities, including MSP presence, activities, and connections to customer networks, as defined in the contract. (Crucially, customer contracts should ensure MSP account monitoring and auditing.)
    • Prompt notification to customers of any confirmed or suspected security events and incidents on the provider’s infrastructure and administrative networks. These notifications should be directed to a Security Operations Center (SOC) for immediate analysis and triage.

3. Enforce Multi-Factor Authentication (MFA): Strengthening Access Security

Securing remote access applications and enforcing Multi-Factor Authentication (MFA) wherever feasible is crucial for hardening the infrastructure that provides access to networks and systems. MFA adds a critical layer of security beyond passwords, significantly reducing the risk of unauthorized access. Refer to these resources for guidance on MFA implementation:

Important Note on MFA Protocols: It is critical to be aware that Russian state-sponsored APT actors have demonstrated the capability to exploit default MFA protocols. Organizations must therefore review their MFA configuration policies to proactively protect against “fail open” scenarios and re-enrollment exploits. Refer to CISA Alert “Joint CISA-FBI CSA: Russian State-Sponsored Cyber Actors Gain Network Access by Exploiting Default MFA Protocols and “PrintNightmare” Vulnerability” for detailed information.

Specific Actions:

  • MSPs: Actively recommend the adoption of MFA across all customer services and products. Critically, MSPs must implement MFA on all accounts that access customer environments and treat these accounts as privileged.
  • Customers: Ensure that contractual agreements with MSPs mandate the use of MFA for all provided services and products. Contracts should also stipulate that MFA is enforced on all MSP accounts used to access customer environments.

4. Manage Internal Architecture Risks and Segregate Networks: Limiting Breach Impact

A thorough understanding of the network environment and the implementation of network segmentation are vital security measures. Organizations should identify, categorize, and isolate critical business systems. Applying appropriate network security controls to these segments minimizes the potential impact of a security breach across the entire organization. For resources on network segmentation and architecture security, consult:

Specific Actions:

  • MSPs: Conduct comprehensive reviews and verification of all connections between internal systems, customer systems, and external networks. Segregate customer datasets (and services, where applicable) from each other and from internal company networks. This limits the spread of attacks originating from a single point of compromise. Crucially, avoid reusing administrative credentials across multiple customers.
  • Customers: Review and verify all connections between internal systems, MSP systems, and other networks. Ensure robust management of identity providers and trust relationships across different environments. Utilize dedicated Virtual Private Networks (VPNs) or alternative secure access methods to connect to MSP infrastructure. Restrict all network traffic to and from the MSP to these secure, dedicated connections. Verify that networks involved in trust relationships with MSPs are adequately segmented from the rest of their internal networks. Ensure contractual agreements explicitly prohibit MSPs from reusing administrative credentials across multiple customers.

5. Apply the Principle of Least Privilege: Restricting Unnecessary Access

Implementing the principle of least privilege across the entire network environment is a fundamental security best practice. This involves granting users and accounts only the minimum levels of access necessary to perform their required tasks. Privileges should be updated immediately upon any changes in administrative roles. Employ a tiered model for administrative accounts to prevent unnecessary access or privileges. Enterprise-wide full privilege accounts should only be used when absolutely essential, and consider time-based privileges to further limit their usage. Identify high-risk devices, services, and users to minimize their access permissions. Further information on least privilege can be found in:

Specific Actions:

  • MSPs: Apply the principle of least privilege to both internal and customer environments, consistently avoiding default administrative privileges.
  • Customers: Ensure that MSPs apply the principle of least privilege to both provider and customer network environments. Customers who have contractual arrangements granting them administration of MSP accounts within their environment must ensure that these MSP accounts are strictly limited to accessing only the services and resources managed by the MSP.

6. Deprecate Obsolete Accounts and Infrastructure: Reducing Attack Surface

Regularly reviewing the internet attack surface and taking proactive steps to minimize it is essential for both MSPs and customers. This includes disabling user accounts promptly when personnel transitions occur. Refer to NCSC-UK guidance on managing obsolete products: “Device Security Guidance: Obsolete products“. (While account sharing is discouraged, if it is necessary, ensure passwords for shared accounts are reset immediately when personnel changes occur.)

Organizations should also conduct regular audits of their network infrastructure, paying particular attention to systems at the MSP-customer boundary. Identify and disable any unused systems and services. Port scanning tools and automated system inventories can aid in verifying the roles and responsibilities of systems and identifying redundancies.

Specific Actions:

  • Customers: Ensure the prompt disabling of MSP accounts that are no longer actively managing infrastructure. Disabling MSP accounts is often overlooked when contracts are terminated, creating a potential security vulnerability.

7. Apply Updates and Patches: Addressing Known Vulnerabilities

Maintaining updated software across all systems is a critical security practice. This includes applying the latest updates to operating systems, applications, and firmware. Prioritize applying security updates for software with known exploited vulnerabilities. It is crucial to prioritize patching vulnerabilities listed in CISA’s Known Exploited Vulnerabilities (KEV) catalog over simply focusing on vulnerabilities with high Common Vulnerability Scoring System (CVSS) scores that may not have been actively exploited. Resources on patching and vulnerability management include:

Specific Actions:

  • MSPs: Implement updates on internal networks as rapidly as possible to mitigate risks.
  • Customers: Understand their MSP’s software update policy and request comprehensive and timely updates as a standard and ongoing service.

8. Backup Systems and Data: Ensuring Data Recovery and Resilience

Regularly updating and rigorously testing backups is a cornerstone of data protection and disaster recovery. This includes creating and maintaining “gold images” of critical systems for rapid rebuilding if necessary. (The frequency of backups should be based on the organization’s Recovery Point Objective (RPO) [20]). Backups should be stored separately and isolated from network connections to prevent ransomware from spreading to backup data. Many ransomware variants actively target accessible backups for encryption or deletion. Isolating backups is crucial for restoring systems and data to a pre-attack state in the event of a ransomware incident. Best practices include storing backups on offline or external media. Relevant resources include:

Specific Actions:

  • MSPs: Regularly back up internal data and customer data (where contractually required). Maintain offline backups encrypted with separate, offline encryption keys. Encourage customers to create secure, offsite backups and regularly test recovery capabilities.
  • Customers: Ensure contractual arrangements include backup services that meet resilience and disaster recovery needs. Require MSPs to implement backup solutions that automatically and continuously back up critical data and system configurations, storing backups in easily retrievable and secure locations, such as cloud-based solutions or air-gapped environments.

9. Develop and Exercise Incident Response and Recovery Plans: Preparedness is Key

Comprehensive Incident Response and Recovery plans are essential for effective cybersecurity. These plans should clearly define roles and responsibilities for all organizational stakeholders, including executives, technical leads, and procurement officers. Maintain up-to-date hard copies of these plans to ensure access in the event of network inaccessibility, such as during a ransomware attack. NCSC-UK offers guidance on creating effective cyber exercises: “Effective steps to cyber exercise creation“.

Specific Actions:

  • MSPs: Develop and regularly exercise internal incident response and recovery plans. Encourage customers to develop and exercise their own plans as well.
  • Customers: Ensure contractual arrangements include incident response and recovery plans that meet their resilience and disaster recovery requirements. Mandate regular testing of these plans to ensure effectiveness.

10. Understand and Proactively Manage Supply Chain Risk: Securing the Ecosystem

All organizations must proactively manage ICT supply chain risk across security, legal, and procurement functions. Conduct thorough risk assessments to identify and prioritize resource allocation for supply chain security. Resources on supply chain security include:

  • NCSC-UK Supply Chain Security Guidance: “Supply chain security guidance“
  • CISA ICT Supply Chain Resource Library: “ICT Supply Chain Resource Library”

Specific Actions:

  • MSPs: Understand their own supply chain risks and manage the cascading risks they pose to their customers.
  • Customers: Understand the supply chain risk associated with their MSP, including risks from third-party vendors and subcontractors. Establish clear network security expectations with MSPs. Understand the level of access MSPs have to their network and data. Ensure contractual agreements meet specific security requirements and clearly define responsibility ownership for areas such as hardening, detection, and incident response. CISA Insights provides further guidance on risk considerations for MSP customers: “Risk Considerations for Managed Service Provider Customers“.

11. Promote Transparency in Contracts and Services: Clear Communication

Transparency in contractual arrangements between MSPs and customers is beneficial for both parties. Clear definitions of responsibilities and service offerings are essential for effective security management.

Specific Actions:

  • MSPs: When negotiating contracts, provide clear explanations of services included, services excluded, and all incident response and recovery contingencies.
  • Customers: Ensure a thorough understanding of the security services provided by the MSP under contract. Address any security requirements that fall outside the contract’s scope. Contracts should detail how and when MSPs will notify customers of incidents affecting their environment.

12. Manage Account Authentication and Authorization: Strong Access Controls

Adhering to best practices for password and permission management is crucial for all organizations. Resources for strong authentication include:

Regularly review logs for unexplained failed authentication attempts. A surge in failed attempts immediately following a password change could indicate account compromise. Network defenders can proactively monitor for such “intrusion canaries” by reviewing logs after password changes across sensitive accounts, ensuring users are notified of changes via off-network communications. For further guidance, consult the ACSC publication “Windows Event Logging and Forwarding” and Microsoft’s documentation “4625(F): An account failed to log on“.

Specific Actions:

  • MSPs: Verify that customers restrict MSP account access to only the systems managed by the MSP.
  • Customers: Ensure MSP accounts are not assigned to internal administrator groups. Restrict MSP account access to systems specifically managed by the MSP. Grant access and administrative permissions on a need-to-know basis, adhering to the principle of least privilege. Conduct audits to verify MSP account usage is appropriate and that accounts are disabled when not actively in use.

Purpose of this Advisory

This advisory is a collaborative effort by cybersecurity authorities in the UK, Australia, Canada, New Zealand, and the U.S. to fulfill their respective cybersecurity missions. This includes developing and disseminating cybersecurity specifications and mitigations to protect organizations from evolving cyber threats.

Acknowledgements

The cybersecurity authorities of the UK, Australia, Canada, New Zealand, and the U.S. extend their gratitude to Secureworks for their valuable contributions to this Cybersecurity Advisory.

Disclaimer

The information provided in this report is for informational purposes only and is provided “as is.” NCSC-UK, ACSC, CCCS, NCSC-NZ, CISA, NSA, and FBI do not endorse any commercial products or services mentioned herein. References to specific commercial products, processes, or services do not constitute or imply endorsement, recommendation, or favoring by these agencies.

Contact Information

United Kingdom organizations: Report significant cyber security incidents via ncsc.gov.uk/report-an-incident (24/7 monitoring) or call 03000 200 973 for urgent assistance.
Australian organizations: Visit cyber.gov.au or call 1300 292 371 (1300 CYBER 1) for incident reporting, alerts, and advisories.
Canadian organizations: Report incidents via email to CCCS at [email protected].
New Zealand organizations: Report incidents to [email protected] or call 04 498 7654.
U.S. organizations: Report anomalous cyber activity and incidents 24/7 to [email protected] or call 1-844-Say-CISA (1-844-729-2472) and/or to the FBI via your local FBI field office or the FBI’s 24/7 CyWatch at (855) 292-3937 or [email protected]. Include incident date, time, location, activity type, affected individuals, equipment, company/organization name, and a point of contact. For NSA client requirements or general cybersecurity inquiries, contact [email protected].

Resources

In addition to the references below, consult the resources hyperlinked throughout this advisory for detailed guidance.

References

[1] State of the Market: The New Threat Landscape, Pushing MSP security to the next level (N-able)
[2] Global targeting of enterprises via managed service providers (NCSC-UK)
[3] Guidance for MSPs and Small- and Mid-sized Businesses (CISA)
[4] Kaseya Ransomware Attack: Guidance for Affected MSPs and their Customers (CISA)
[5] APTs Targeting IT Service Provider Customers (CISA)
[6] MSP Investigation Report (ACSC)
[7] How to Manage Your Security When Engaging a Managed Service Provider
[8] Supply Chain Cyber Security: In Safe Hands (NCSC-NZ)
[9] Multi-factor authentication for online services (NCSC-UK)
[10] Zero trust architecture design principles: MFA (NCSC-UK)
[11] Joint CISA-FBI CSA: Russian State-Sponsored Cyber Actors Gain Network Access by Exploiting Default MFA Protocols and “PrintNightmare” Vulnerability
[12] Security architecture anti-patterns (NCSC-UK)
[13] Preventing Lateral Movement (NCSC-UK)
[14] Preventing Lateral Movement: Apply the principle of least privilege (NCSC-UK)
[15] Device Security Guidance: Obsolete products (NCSC-UK)
[16] Known Exploited Vulnerabilities Catalog (CISA)
[17] The problems with patching (NCSC-UK)
[18] Security principles for cross domain solutions: Patching (NCSC-UK)
[19] Joint CSA: 2021 Top Routinely Exploited Vulnerabilities
[20] Protecting Data from Ransomware and Other Data Loss Events: A Guide for Managed Service Providers to Conduct, Maintain, and Test Backup Files (NIST)
[21] Stop Ransomware website (CISA)
[22] Offline backups in an online world (NCSC-UK)
[23] Mitigating malware and ransomware attacks (NCSC-UK)
[24] Effective steps to cyber exercise creation (NCSC-UK)
[25] Supply chain security guidance (NCSC-UK)
[26] ICT Supply Chain Resource Library (CISA)
[27] Risk Considerations for Managed Service Provider Customers (CISA)
[28] Device Security Guidance: Enterprise authentication policy (NCSC-UK)
[29] Preventing Lateral Movement: Apply the principle of least privilege (NCSC-UK)
[30] Implementing Strong Authentication (CISA)

Appendix: Definitions

This advisory adopts definitions of MSPs consistent with established industry and government sources.

Gartner’s IT Glossary Definition of MSP (Referenced by NIST):

“A managed service provider (MSP) delivers services, such as network, application, infrastructure and security, via ongoing and regular support and active administration on customers’ premises, in their MSP’s data center (hosting), or in a third-party data center.” [Improving Cybersecurity of Managed Service Providers]

MSPs may offer their own services in conjunction with other providers’ services. While some MSPs specialize (pure-play), many integrate services from diverse providers. The term MSP has evolved from infrastructure-focused services to encompass a broader range of continuous management, maintenance, and support functions.

UK Department for Digital, Culture, Media & Sport (DCMS) Definition:

“Managed Service Provider – A supplier that delivers a portfolio of IT services to business customers via ongoing support and active administration, all of which are typically underpinned by a Service Level Agreement. A Managed Service Provider may provide their own Managed Services or offer their own services in conjunction with other IT providers’ services. The Managed Services might include:

  • Cloud computing services
  • Workplace services
  • Managed Network
  • Consulting
  • Security services
  • Outsourcing
  • Service Integration and Management
  • Software Resale
  • Software Engineering
  • Analytics and Artificial Intelligence (AI)
  • Business Continuity and Disaster Recovery services” [Call for views on supply chain cyber security]

These services can be delivered from customer premises, MSP data centers, or third-party facilities, including public cloud data centers.

Revisions

May 11, 2022: Initial version

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply

Your email address will not be published. Required fields are marked *