Squeezing Microseconds: High-Performance API Gateway Tuning for Nordic Traffic

I distinctly remember the silence in the Slack channel. It was Black Friday, 2016. Our primary API gateway—a bloated Java stack sitting on shared legacy hosting—choked. Not because of CPU load, but because we exhausted ephemeral ports connecting to backend microservices. The latency spiked from 45ms to 4,000ms. We lost thousands of kroner per minute. That outage taught me a brutal lesson: default configurations are production suicide.

If you are serving the Norwegian market, you aren't just fighting bad code; you are fighting physics and the default settings of your Linux distribution. Whether you are running Kong (currently v0.10) or raw NGINX as your gateway, the bottleneck is rarely the software itself. It is the plumbing underneath.

1. Stop Ignoring the Kernel

Most VPS providers deliver instances tuned for generic web serving, not high-throughput API routing. When your gateway proxies thousands of requests to backend services (like a Magento store or a Node.js app), the TCP connection overhead becomes your enemy.

You need to modify /etc/sysctl.conf. By default, Linux is too conservative with reclaiming TCP connections. We need to enable tcp_tw_reuse to allow the kernel to reuse sockets in the TIME_WAIT state for new connections.

Here is the baseline configuration we apply to all high-performance CoolVDS instances intended for gateway duties:

# /etc/sysctl.conf

# Maximize the number of open file descriptors
fs.file-max = 2097152

# Increase range of ephemeral ports
net.ipv4.ip_local_port_range = 1024 65535

# Reuse connections in TIME_WAIT state
net.ipv4.tcp_tw_reuse = 1

# Decrease time to keep sockets in FIN-WAIT-2
net.ipv4.tcp_fin_timeout = 15

# Increase the maximum backlog of connection requests
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535

Apply these with sysctl -p. If you skip this, your API gateway will hit a hard ceiling regardless of how much RAM you throw at it.

2. The NGINX Upstream Keepalive Trap

If you are using NGINX (or OpenResty/Kong) as a reverse proxy, you are likely making a classic mistake: opening a new connection to your backend for every single request. This adds the full TCP handshake overhead to every API call. In a microservices architecture, this internal latency compounds rapidly.

You must configure the upstream block to use keepalive connections. This keeps the pipe open between the gateway and your application.

upstream backend_api {
    server 10.0.0.5:8080;
    
    # Keep 64 idle connections open to the backend
    keepalive 64;
}

server {
    location /api/ {
        proxy_pass http://backend_api;
        
        # Required for HTTP/1.1 keepalive to backends
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        
        # Pass the Host header correctly
        proxy_set_header Host $host;
    }
}

Pro Tip: Do not set the keepalive value too high if your backend is single-threaded (like Node.js). You might starve the event loop. A value between 32 and 64 is usually the sweet spot for standard deployments.

3. SSL/TLS: Speed vs. Security

With Chrome pushing hard against non-secure sites and HTTP/2 requiring TLS, encryption is mandatory. However, the handshake is expensive. In 2017, we finally have widespread support for Elliptic Curve Cryptography (ECC) and ChaCha20/Poly1305 (thanks to OpenSSL 1.0.2).

To reduce the time-to-first-byte (TTFB), you must enable OCSP Stapling. This allows your server to present a valid certificate revocation status to the client, saving the client a separate DNS lookup and connection to the CA.

ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;

# Optimize the cache
ssl_session_cache shared:SSL:50m;
ssl_session_timeout 1d;
ssl_session_tickets off;

4. The Hardware Reality: Why IOPS Matter

Software tuning only gets you so far. Eventually, your API gateway needs to write logs, cache data to disk, or buffer requests. If you are running on standard SATA SSDs—or worse, spinning rust—your request queue will block while waiting for the disk controller.

We benchmarked a standard SSD VPS against a CoolVDS NVMe instance using ioping. The difference is not subtle.

When your access logs are writing thousands of lines per second, that 370-microsecond difference per operation accumulates into measurable lag for the end user.

5. The Norwegian Context: Latency and Jurisdiction

If your users are in Oslo, Bergen, or Trondheim, hosting in a US-East data center is a disservice. The speed of light dictates a minimum round-trip time (RTT) of ~90-100ms across the Atlantic. Add SSL negotiation and server processing, and you are staring at 300ms before the user sees data.

Hosting in Norway or Northern Europe (via the NIX exchange) drops that RTT to under 10ms. Furthermore, with the GDPR enforcement date looming next year (May 2018), data residency is becoming a board-level discussion. Keeping personal data on servers within the EEA isn't just about performance anymore; it's about compliance with upcoming Datatilsynet audits.

Final Thoughts

Performance isn't an accident. It is the result of stripping away bloat, tuning the kernel to handle the load, and choosing infrastructure that doesn't steal your CPU cycles.

At CoolVDS, we don't oversell our host nodes. We use KVM virtualization to ensure your memory and CPU are actually yours, and we run exclusively on NVMe storage because we know that I/O wait is the silent killer of API performance.

Ready to drop your API latency? Deploy a high-performance NVMe instance on CoolVDS today and test your throughput against the competition.