Scaling your infrastructure in the cloud: How to handle huge traffic spikes

As the coronavirus places increasing pressure on the infrastructure, companies such as telecoms, streaming services, and other companies are struggling to ensure that the lights stay on, and they can meet increased traffic demands. This involves adapting to changing usage patterns and removing scalability bottlenecks.

A scalable system can help keep your application running during peak times and not end up losing your money or damaging your reputation. 

As a company that has been delivering DevOps services for many years (25+ successfully completed projects), we understand how challenging it may be for a company to scale up in the cloud. Creating a fully scalable system and infrastructure can be a tedious task that requires assessment, planning, and testing. If you already have an application in place, redesigning that system can be a challenging process that may require code changes, software updates, and a lot of monitoring.

Our experts are sharing their knowledge on how to do it right and cost-efficiently depending on your specific business needs. 

The success of scaling your infrastructure in the cloud is predetermined by rigorous assessment and effective planning.

There are 5 key stages of infrastructure scaling in the cloud

1. Infrastructure audit 

Monitoring and assessment of the infrastructure. It is important to identify the current load on the system, and how the load is expected to grow. Analysis of the historical data and collecting metrics helps to understand what problems a business faced at certain periods of time and forecast issues it will have in the future. It will also help to spot blockers to effective scaling and identify the most viable solutions to these problems. The audit is performed by a team of network, system, and DevOps engineers.  

2. Design 

When there is a clear understanding of the infrastructure and the traffic patterns at specific periods of time, infrastructure specialists design at least three different pricing plans for scaling. And you can choose the one that fits your specific business needs and your budget.  

The plans cover choosing different cloud vendors, hosting in different regions, choosing different availability zones, taking advantage of autoscaling, using different types of instances (e.g. spot instances, reserved instances, etc.), and more.  

Also, when blending public and private clouds with on-premise assets to create a hybrid environment, you need to redesign your in-house IT infrastructure to minimize inconsistencies and interoperability problems between different systems. 

3. Pilot stage. 

That’s when the system is put to load tests. The system is adjusted if needed and goes to production.

4. Production stage. 

The system goes live and is used by real users. 

5. Support stage. 

Support of the infrastructure can be performed either on the client's side or delegated to your outsourcing vendor. 

3 most common scalability bottlenecks companies face 

Problem: The architecture of the application does not allow for scaling in the cloud 

Solution:

First of all, in order to make your software take advantage of multiple computers (or even multiple CPUs within the same computer), your software needs to be able to parallelize or distribute its tasks. In fact, it is easier to migrate to the cloud for enterprises that already have microservices architecture and orchestrate their containers with tools like Kubernetes or Docker engine. AWS and Azure offer specific assistance for Kubernetes as well as other engines that further helps in the migration process. To prepare the architecture for migration, you should partner with a team of IT specialists that will audit the legacy architecture, resolve tech debt, figure out interdependent parts, and create profound documentation. 

Also, the application should be tolerant of instances going up or down at any moment For instance, your application needs to use a database service that allows dynamic scaling. Furthermore, the application should be designed to be capable of working with data centers in different regions. 

If the very software architecture is a single-point-failure, you can solve it either by redesigning the application or adding specific services to your existing application. Both strategies have their advantages and shortcomings, and the choice of the best solution will depend on a specific business case. 

Problem: Network bottlenecks 

Solution:

You can have very powerful servers but because a server’s bandwidth is low, the data transmission lags. Something may be wrong with load balancers, routers, VPNs, etc. They must be either replaced or multiplied, depending on a specific type of issue.  Also, the problem may be caused by a low level of network bandwidth of a specific instance. In this case, you need to choose a different type of instance.

Problem: Servers you use may lack capacity (e.g., disc subsystem, memory, processors)

Solution:

The problem can be solved by adding resources to a specific instance (vertical scaling) or adding a few more instances ( horizontal scaling).

Vertical scaling in the cloud - adding more CPU, memory, or I/O resources to an existing instance. Amazon Web Services (AWS) vertical scaling and Microsoft Azure vertical scaling can be performed when you change instance sizes. Scaling vertically in the cloud is possible for everything from EC2 instances to RDS databases. AWS and Azure cloud services offer many different instance sizes. The key advantage of vertical scaling is that it’s fast. The key disadvantage of it is that it offers limited scalability Therefore, the vertical scaling strategy is widely used by small and mid-sized companies. 

Horizontal scaling in the cloud - adding more instances instead of moving to a larger instance size. That often means splitting workloads between instances to limit the number of requests any individual instance is getting, and it is good for performance, no matter how large the instance.

Scaling horizontally is usually the best practice if your business deals with very high traffic spikes. Also, It’s much easier to accomplish without downtime as scaling vertically (even in the cloud) usually calls for making the application unavailable for a certain period of time. And there even may be cases that that particular instance may be unavailable after the server is up, and you need to find another one to replace it. Therefore, horizontal scaling is the best choice when a high availability of services is required. Also, horizontal scaling is easier to manage automatically.

Many global tech giants such as Google, Yahoo, Facebook, eBay, Amazon, etc. use horizontal scaling to meet high load demands.

What is the best choice for the majority of big companies? A hybrid approach.

Large companies usually prefer horizontal scaling, but they also take advantage of vertical scaling by adding very powerful machines when scaling horizontally. This way, you can benefit from the speed of vertical scaling, combined with the infinite scalability and the resilience of horizontal scaling. 

Scaling cloud infrastructure can be manual, scheduled, or automated

• Manual scaling. Both vertical and horizontal scaling can be accomplished manually by an individual. However, manual scaling is not very efficient as it can’t take into account all the minute-by-minute changes in demand and traffic. This also can bring about human errors as an individual might forget to scale back down when the demand goes down. And that will lead to extra charges. 

• Scheduled scaling. Based on your typical demand curve during the day or certain periods of time, you can scale out to, for example, 5instances from 5 pm to 10 pm and then back into two instances from 10 pm to 7 am, then back out to five instances at 5 pm. This makes it easier to tailor your provisioning to your actual usage without requiring a team member to make the changes manually every day.

• Automatic scaling (also known as autoscaling) is when your compute, database, and storage resources scale automatically based on predefined rules. For example, when metrics like CPU, memory, and network utilization rates go above or below a certain threshold, you can scale up, down, out, or in. The key advantage of autoscaling is that your application is always available and has enough resources provisioned to prevent performance problems or outages—without paying for far more resources than you are actually using. 

Infrastructure as code (IaC) is a key enabler for efficient migration of legacy systems to the cloud. Thanks to it, you can automatically manage and provision computers and networks (physical and/or virtual) through scripts instead of manually configuring them.

Optimizing costs when scaling cloud infrastructure

There are many best practices for cloud cost optimization. Here are the most common ones.

How to choose specialists for effective infrastructure scaling 

• Specialists need to have a solid understanding of servers, networking, and protocols. 

• They must be proficient in concurrency, performance, and resource constraints.

• They must be able to anticipate future issues and potential risks, offer and implement solutions.

• They must have long-standing experience in cloud migration and have the corresponding certifications. 

• The company needs to have a large portfolio of cases that show solving different architecture and infrastructure challenges for various types of businesses.

• The company can undertake a Discovery Phase, conduct assessment of the present state and devise the road map that fits a specific business case.

• Infrastructure specialists need to have a high level of communication and problem-solving skills. 

Why choose N-X as your partner for scaling cloud infrastructure

• Our expertise in cloud computing includes on-premise-to-cloud migration, cloud-to-cloud migration as well as multicloud and hybrid cloud management;

• We offer professional DevOps outsourcing services, including Cloud adoption (architecture, migration, optimization), building and streamlining CI/CD processes, security issues detection/prevention (DDOS & intrusion), firewall-as-a-service, and more;

• N-iX is a certified AWS partner, a Microsoft gold certified partner, an Opentext Services silver partner, and a SAP partner;

• N-iX is recognized as a trusted vendor by IAOP, GSA, Inc. 5000, Software 500, Clutch.co, and others;

• N-iX is compliant with PCI DSS, ISO 9001, ISO 27001, and GDPR standards.