Console Login

Stop Watching Progress Bars: Optimizing CI/CD Pipelines in 2015

The "Coffee Break" Build is Killing Your Velocity

I walked into a development shop in Oslo last week. The lead developer was staring at a terminal, sipping coffee, looking bored. "Compiling?" I asked. "Deploying," he corrected. "The Jenkins queue is backed up again. It takes 20 minutes to run the integration tests."

In 2015, this is unacceptable. We have tools like Docker (version 1.6 just dropped, and it is promising) and configuration management like Ansible. Yet, teams still run their build pipelines on underpowered, oversold shared hosting environments. They treat the Continuous Integration server as an afterthought.

Here is the brutal truth: Your CI/CD pipeline is the single most I/O-intensive workload in your infrastructure. If you throw it on cheap hardware, you are paying for it in developer hours.

The Silent Killer: Disk I/O and Small Files

Most people blame the CPU for slow builds. They are usually wrong. Look at a standard web application build process today:

  • npm install (Thousands of tiny writes)
  • composer install or pip install (Network fetch + disk write)
  • Database migration tests (Intense read/write bursts)
  • Artifact archiving (Large sequential writes)

I recently audited a Magento deployment where the build took 45 minutes. We ran iostat -x 1 during the build. The %util on the disk was pinned at 100% while the CPU sat idle at 15%.

The culprit? Spinning rust. Standard HDDs, or even cheap SATA SSDs in a crowded VPS node, cannot handle the random I/O required by dependency managers. This is why at CoolVDS, we are aggressively rolling out NVMe storage (PCIe-based flash). The difference isn't just percentage points; it is orders of magnitude.

Pro Tip: If you are running Jenkins on a Linux VPS, check your dirty page ratio. If your disk can't keep up, the kernel blocks processes while flushing RAM to disk.
vm.dirty_ratio = 10
vm.dirty_background_ratio = 5
Tuning these in /etc/sysctl.conf can prevent the "system freeze" effect during heavy writes.

Virtualization: OpenVZ vs. KVM

In the hosting world, not all "Clouds" are created equal. Many budget providers use OpenVZ (container-based virtualization). It is efficient for them, but terrible for you if you need kernel-level control.

For a robust CI pipeline, specifically if you are experimenting with Docker containers inside your builds, you need KVM (Kernel-based Virtual Machine). KVM provides full hardware virtualization. No noisy neighbors stealing your CPU cycles because the host scheduler got overwhelmed. Managed hosting isn't just about support; it's about guaranteeing that the resources you pay for are actually yours.

Optimizing the Jenkins JVM

Out of the box, Jenkins is conservative. If you are renting a VPS Norway instance with 8GB or 16GB of RAM, you need to tell Java to use it. Don't let the garbage collector pause your build every 30 seconds.

Edit your Jenkins configuration (usually in /etc/default/jenkins on Debian/Ubuntu systems):

JAVA_ARGS="-Xmx4096m -XX:MaxPermSize=512m -XX:+UseConcMarkSweepGC -Djava.awt.headless=true"

Note: If you have updated to Java 8, MaxPermSize is ignored (replaced by Metaspace), but many of us are still stuck supporting legacy Java 7 apps, where this flag prevents the dreaded OOM crash.

The Nordic Advantage: Latency and Sovereignty

Why host your build server in Norway? Two reasons: Speed and Law.

1. Low Latency: If your dev team is in Oslo, Bergen, or Trondheim, pushing gigabytes of code to a server in Virginia (US-East) is painful. Physics is undefeated. Hosting on a CoolVDS instance peered at the NIX (Norwegian Internet Exchange) ensures your git pushes and artifact downloads happen in milliseconds, not seconds.

2. Data Sovereignty: With the ongoing debates in the EU regarding data privacy and the scrutiny on Safe Harbor, knowing exactly where your code lives is paramount. The Norwegian Data Protection Authority (Datatilsynet) is rigorous. Hosting locally simplifies compliance. While the proposed EU data regulations (GDPR) are still being debated in Brussels, positioning your infrastructure within strong jurisdictions now is a smart "Pragmatic CTO" move.

DDoS and The Build Server

We often think ddos protection is just for the public-facing website. But what happens if your build server exposes a webhook for GitHub or Bitbucket? If that IP gets hammered, your developers can't deploy hotfixes.

I've seen it happen. A botnet scans for open port 8080 (default Jenkins), floods it, and suddenly the entire company is paralyzed. Ensure your hosting provider offers upstream mitigation that cleans traffic before it hits your NIC.

Conclusion

Stop accepting slow builds as a fact of life. It is usually just a symptom of bad I/O and network latency. Move your pipeline to a KVM-based, NVMe storage backed instance close to your team.

Time is the one resource you can't scale horizontally.

Ready to cut your build time in half? Spin up a high-performance instance on CoolVDS today. Our infrastructure is ready for the Docker revolution.