VPS or Dedicated Server: Diagnosing OpenVZ Memory Limits

Symptoms

A common reason to move a site from a dedicated server to a cheaper VPS is cost. A typical LAMP stack (Linux, Apache, MySQL, PHP) is copied over, and at first it works. Under load, or after a few days of uptime, odd failures start to appear:

• Apache children fail to start, and the error log shows Cannot allocate memory or fork failures.
• MySQL refuses to start, or is stopped, even though the memory report inside the VPS shows free RAM.
• Shell commands fail with fork: Cannot allocate memory while the machine is otherwise idle.
• Behaviour differs from the old dedicated server with the same amount of RAM, and no obvious out-of-memory message appears in the system log.

These symptoms are specific to container-based VPS platforms, of which OpenVZ is the common example. The decision between a VPS and a dedicated server is therefore not only about price. It is also about how the platform enforces memory, and where it reports a limit being hit.

Why it happens

Two points in the usual VPS-versus-dedicated comparison are often blurred.

Containers are not hypervisors

Xen and VMware run a separate kernel per guest. OpenVZ does not. It is operating-system-level virtualization: every VPS shares the single kernel of the host node, and the kernel enforces per-container limits. Those limits are called User Beancounters (UBC). Each limit has a held value, a maximum held value, a barrier, a limit and a failure counter.

Burstable memory is a limit on allocation, not on use

An OpenVZ plan advertising a RAM figure usually maps to several counters:

• privvmpages limits the memory address space a container may allocate. A program can fail to allocate memory long before the physical RAM is exhausted.
• vmguarpages and oomguarpages express the guaranteed share. This is the figure to read as the real RAM, as opposed to the burst figure.
• Other counters such as numproc (process count) and kmemsize (kernel memory) can also fail independently.

Software that reserves a lot of address space can hit privvmpages early. Examples are Apache in prefork mode with many children, PHP with a generous memory_limit, and MySQL with large buffers. The kernel then returns an allocation error to the application. The failure is recorded only in the failure counter of that resource, not in the application's own memory statistics.

A dedicated server has none of this. Allocation failures happen only when RAM and swap really run out. That difference is the real trade-off, alongside CPU sharing and disk contention on the node.

Source of truth

Two official OpenVZ documents support the diagnosis and the fix:

• OpenVZ wiki: UBC failcnt reset. It confirms that a container has a set of resource limits (User Beancounters). When one is hit, the matching fail counter increases. That counter is the last column of /proc/user_beancounters. It also confirms that failure counts are kept for the uptime of the container, so a non-zero value can be an old event.
• OpenVZ manual pages (vzctl). They confirm that vzctl set CTID --privvmpages barrier:limit sets both values of that counter, with an optional unit suffix (the example is 5M:6M), and that --save writes the change to the container configuration. Without it, the change is lost at the next container start.

These two sources establish how to read the counters and how to change them. The counters can only be changed on the host node, by whoever administers it.

Fix options

The first two options are not exclusive. The sensible order is to measure first and then decide whether the limits or the application is wrong.

Why this fix

The chosen method is to measure the counters, then reduce the application footprint, then ask for a limit increase only if the counters still fail. Reasons:

• It keeps the cheap VPS. A migration to Xen or a dedicated server is justified only when the workload really needs more resident memory than the plan guarantees.
• The beancounter file shows which resource failed. Without that, a limit change is guesswork. A container can fail on numproc while its memory is fine.
• Footprint reduction works on any platform, so it still pays off after a later migration.
• Limit changes require the node administrator, and a request with specific failing counters is easy to act on.

Step by step

1. Read the counters inside the VPS

Take a snapshot and list only the rows with a non-zero failure count. The last field of each row is the fail counter. The first two lines are the version line and the column header.

cat /proc/user_beancounters > /tmp/ubc.before
awk 'NR>2 && $NF+0 > 0' /proc/user_beancounters

A row for privvmpages with a non-zero last column confirms the diagnosis. Rows for numproc, kmemsize or the socket buffers point to other limits.

2. Find which processes hold the memory

free -m
ps -eo pid,rss,vsz,args --sort=-rss | head -15

The vsz column (virtual size) is what counts against privvmpages. A process with a large vsz and a small rss still consumes the allocation limit.

3. Reduce the Apache footprint

On a CentOS 5 style layout, check the prefork settings:

grep -A8 'prefork.c' /etc/httpd/conf/httpd.conf

Lower the number of children to an amount that fits the guaranteed memory. Illustrative starting values for a small VPS:

<IfModule prefork.c>
StartServers 4
MinSpareServers 4
MaxSpareServers 8
ServerLimit 40
MaxClients 40
MaxRequestsPerChild 2000
</IfModule>

Test the syntax and reload without dropping connections:

apachectl configtest
service httpd graceful

Also lower memory_limit in /etc/php.ini if scripts do not need much, and disable unused Apache modules.

4. Shrink the MySQL buffers

In /etc/my.cnf, under [mysqld]:

key_buffer = 32M
query_cache_size = 16M
max_connections = 60
thread_cache_size = 8
table_cache = 128

service mysqld restart

5. Compare the counters after a load period

diff /tmp/ubc.before /proc/user_beancounters

Only the held values and maximums should move. The fail counter column should stay unchanged. Because failure counts persist for the container uptime, an unchanged value is the success signal, not a zero.

6. If the counters still fail, raise the limit on the node

This step runs on the host node as root, not inside the VPS. The container ID 101 is an example. Keep --save so the change survives a restart:

vzctl set 101 --privvmpages 256M:272M --save
vzctl exec 101 cat /proc/user_beancounters

Keep the barrier and limit values close together. Do not raise them above what the node can back with physical memory and swap, because other containers rely on the same pool.

Looking back

What worked

• Reading /proc/user_beancounters turns vague out-of-memory symptoms into a named resource, and it is the right first step on any OpenVZ VPS.
• Trimming Apache and MySQL reduces both the allocation footprint and the real memory use, with no change of platform.

Limits

• Tenants cannot change the beancounters themselves. The limit increase depends on the node administrator and on spare node capacity.
• The fail counter records that a limit was hit at some point in the uptime, not how often or how recently. The snapshot-and-diff method is needed to tell old events from new ones.
• Container platforms share a kernel and a disk array with other tenants. Heavy writes by a neighbour can add latency, and a kernel problem affects every container on the node.

When to choose another method

• Pick a Xen-style VPS when a fixed, kernel-visible RAM figure and a private swap matter more than elastic limits.
• Pick a dedicated server for large databases, sustained heavy I/O, specific RAID layouts or special hardware. Also pick it when an auditor requires physically isolated hardware, for example in payment-card environments.
• Pick a VPS for small sites, test environments and one-VPS-per-client setups, where isolation between clients and fast re-imaging outweigh raw power.
• For personal data, the physical location of the hardware matters under the EEA privacy rules. This applies equally to a VPS and a dedicated server.