How to deal with sudden traffic surge and server overload

Time:2022-11-24

Author: Frances Fedoriska of F5
Original link:How to deal with sudden traffic surge and server overload
Reproduced source: NGINX official website


Under the impact of the new crown epidemic, many retailers’ customers have changed the channels and methods of their holiday shopping. Since people can’t go to physical stores to shop, online shopping has ushered in an unprecedented new upsurge —U.S. e-commerce sales soar to $189 billion in 2020, up 33% from 2019. With the shopping festival less than six months away, here are a few ways you can use NGINX to help you prepare for unexpected traffic spikes.

Step 1: Know yourself and your enemy

Before tuning your stack, it’s a good idea to understand how much you need to optimize your site’s performance to perform optimally (rather than barely) during the shopping spree. Start by collecting performance data on your competitor’s sites. Page load time is probably the most important metric because many users these days are impatient and justPages take longer than three seconds to load and they leave. There are many tools for measuring load times, and most of them are free. You may wish to refer to11 Testing Tools Recently Reviewed by Geekflare

You might consider measuring the load time of the following pages:

  • home page
  • Product Search Page
  • Product Details Page
  • Confirmation page after clicking the “Buy” button

It only takes a few hours to test key pages of several competitors and your own website. (Before measuring download times, be sure to useShift+RefreshClear your browser cache. ) with the blessing of the measurement results, what you need to do next is:

  • Develop overall performance metrics. How long does the entire process (visit, search, and purchase) take from start to finish for different sites?
  • identify strengths and weaknesses. Find out where your site is strong, competitive, or lagging.
  • Note functional differences. How are competitors enhancing the shopping experience with additional features? What features does your website offer to create a unique user experience?
  • making plans. If your site is way behind your competitors in one or more areas, work to match their average response times; if you already have a strong competition, work to be number one.

Step 2: Use NGINX Like the World’s Busiest Websites

Early in 2021,NGINXexistRanked No. 1 in Internet Web Server Ranking. We are honored that so many websites trust us to use our products to deliver websites and applications, and we welcome you to join us. However, NGINX is more than just a web server, it is an all-in-one softwarereverse proxyload balancercacheandAPI Gateway

One of the most important advantages of NGINX is that it can greatly optimize the traffic to your website. You can think of NGINX as the gatekeeper managing the flow of people at the front of your store. Its ability to queue and authorize shopper actions (HTTP requests) is like turning a chaotic crowd on a sidewalk into a smooth and orderly checkout queue in a store. It directs shoppers to a specific spot on an item on a shopping list, ensuring that traffic is evenly distributed and all resources are used evenly.

NGINX primarily uses two out-of-the-box technologies to achieve this:

  • HTTP offload by using keepalive connections, the purpose of which is to buffer slowly arriving HTTP requests and wait until they are ready before forwarding them to the backend server. Transactions initiated from NGINX (on a fast local network) complete much faster than remote clients.
  • Mature with multiple algorithmsload balancing, with the goal of optimizing traffic distribution and using server resources as efficiently as possible.

How to deal with sudden traffic surge and server overload

Read more:

Improve web page response time through caching

Many functions will increase the success of online transactions, such as clicking to save, ordering online and picking up offline, and even customer payment options. NGINX’s content caching capabilities have a similar effect to web traffic. NGINX automatically stores every file it sends to a client and leverages the cache directly for subsequent requests for a given file. Not only does caching speed up the response to users, but it also reduces the load on upstream servers because they don’t have to always process the same request over and over again from the beginning.Content caching can reduce internal traffic by 100x, thereby reducing the processing power of the hardware required to serve the application (depending on the application).

For more details on caching and sample configurations, see our blog postCaching and Microcaching Site ContentandThe NGINX and NGINX Plus Caching Guide

Manage guest traffic with connection, rate and bandwidth controls

During the busiest times, store janitors may need to limit the number of people entering the store for safety reasons (to avoid overcrowding) or to ensure that important customers receive preferential treatment (VIP hours, invitations to promotions, etc.). Web applications also need to take similar measures. You can prevent server overload by limiting the traffic to your website and ensure that clients get access to the resources they need in a timely manner. NGINX (in container environments isNGINX Ingress Controller) provides a series of methods to restrict incoming traffic, including:

  • concurrency limit– Limit the number of concurrent requests forwarded to each server to match the limited number of worker threads or processes in each server
  • request rate limit– Limit each request from the client by the second or by the minute to prevent server overload for services such as payment gateways or complex searches (see our blog post [“NGINX and NGINX Plus Rate Limiting”] for details).
  • bandwidth limit—— Control the amount of data that the client can download in each connection

You can distinguish between different types of clients if necessary. Your store’s delivery zones may not extend to the other side of the world, or you want to prioritize users with items in their shopping carts, you can leverage cookies, geolocation data, and other parameters to control how NGINX applies traffic throttling.

Avoid server overload with elastic scaling

If you don’t have the proper infrastructure in place, traffic surges can easily overload your servers.NGINX’s lightweight event-driven architectureCan maximize the application delivery performance of the current infrastructure. We provide forNGINX Ingress Controllerand NGINX Plus inbare metalandvirtual container environmentThe selection guide below can help you understand exactly what operating costs you need for the performance and scale you intend to achieve.

DevOps teamYou can also take advantage of the following NGINX features to efficiently handle traffic spikes:

Protect customer data with built-in security safeguards

Securing transactions is essential for any website, and especially for online retailers that handle credit card information. Cyberattacks can cost brands reputation damage and lawsuits, something retailer Target knows all too well.Target Addresses 2014 Data Breach with Enhanced Visibility and Security. To differentiate yourself from the competition by creating one of the safest shopping experiences on the market, you can implement:


more resources

Want to get timely and comprehensive access to NGINX-related technical dry goods, interactive questions and answers, series of courses, and event resources?

Please go to the NGINX open source community: