The problem
A team that keeps proprietary code on a public hosting service, or on a server far from its developers, tends to run into three things:
- Slow pushes and fetches. Every
git pushorgit fetchover SSH starts with a connection handshake and several protocol round trips. A distant server makes each of them noticeably slower. - Unclear control over the data. Source code, test fixtures and database dumps checked into a repository can contain personal data. Where the server sits, and who operates it, then becomes a compliance question. In the EU/EEA of 2012 the relevant frame is Directive 95/46/EC and, in Norway, the Personopplysningsloven.
- Clumsy access management. Plain Git over SSH has no notion of per-repository permissions. Each developer needs a shell account, or everyone shares one account and one set of rights.
This article builds a small, key-only Git server on CentOS 6 using Gitolite 2.x. It also covers what to check when it is slow, and how to back it up.
Why it happens
Latency
Network round-trip time grows with distance and the number of hops, and no server software can remove it. Git over SSH needs a TCP connection, an SSH key exchange, authentication and then a few request/response exchanges. A server on the same continent, or in the same country, as the developers shortens each of those steps.
Check the real figure from a workstation instead of assuming it:
ping -c 10 git.example.com
ssh -v [email protected] true
Disk I/O
Server-side Git work (packing objects for a clone, git gc, repacking after large pushes) reads and writes many files. When several developers fetch a large repository at once, a slow or shared disk shows up as high I/O wait. In that state even the SSH daemon responds slowly. Look at it with iostat -x 5 from the sysstat package, or at the wa column in top. Faster storage and a scheduled git gc both help. No fixed speed-up figure applies to every setup.
Access control
Git itself delegates authentication and authorisation to the transport. Over SSH that means operating-system users and file permissions. A tool such as Gitolite sits in front of Git. Every developer connects as the same operating-system user, and the SSH key decides who they are and what they may do.
Source of truth
The Gitolite project documents its requirements and its setup model in its own repository and manual:
- The Gitolite repository and README confirms the server requirements: a Unix system with
sh, Git 1.6.6 or later, Perl 5.8.8 or later and OpenSSH 5.0 or later. It also confirms that the hosting user is dedicated and has no existing pubkey access. Users, repositories and rules are changed by pushing to the specialgitolite-adminrepository, never by hand on the server. That repository contains theconfandkeydirdirectories. - The install and setup page confirms that the administrator's public key is copied to the server under the admin's own name. A key file called
alice.pubcreates the useralice. It also confirms the split between the install step (put the code in place) and the setup step (create the files, directories and the admin repository). - The step-by-step setup page confirms two things. The administrator's key must not be added to the hosting user's
authorized_keysby hand. A working setup can be checked from the workstation withgit ls-remoteagainstgitolite-admin.
Those pages describe the current Gitolite generation, which uses gitolite setup. The Gitolite 2.x series that was current in early 2012 follows the same model but uses the gl-system-install and gl-setup commands. Check the README of the release tag you check out. CentOS 6 ships Git 1.7.1, Perl 5.10.1 and OpenSSH 5.3, which satisfy the minimum versions above.
Fix options
| Option | Strengths | Weaknesses |
|---|---|---|
| Bare repositories with one shell account per developer | Nothing extra to install; standard OpenSSH and file permissions | Accounts to manage per person; per-repository rights depend on Unix groups; developers get a shell |
One shared git account with many keys in authorized_keys | Simple; no extra software | No per-user or per-repository rules; every key can reach every repository; removing a person means editing the file by hand |
| Gitolite on a dedicated user | Per-repository and per-user rules in a text file; no shell for developers; small Perl-only footprint; configuration lives in a Git repository | Command-line administration only; learning the rule syntax |
| Web-based suites (Gitorious, GitLab) | Web UI, merge requests, user self-service | A Ruby application stack to install, patch and back up; much more to secure |
| A hosted service | No server to run | Code leaves your control; latency and jurisdiction depend on the provider |
Why this fix
Gitolite is the best fit when the goal is a private server with real access rules and a small attack surface. Developers never receive a shell. The forced-command mechanism in authorized_keys limits each key to Git operations. Rules are plain text, kept in a Git repository, so every permission change is committed, reviewable and reversible. The software needs only Git, Perl and OpenSSH, all of which are in the CentOS 6 base repositories. The server still holds ordinary bare repositories, so migrating away later means copying directories.
The shared-account option is the weakest for more than a handful of people, because it cannot separate repositories or users. A web suite is the right choice only when the team needs a browser interface and can maintain the extra stack.
Step by step
These commands are given as instructions. This article does not reproduce program output. Success is described in words after each step.
1. Prepare the server (as root)
yum -y update
yum -y install git perl openssh-clients
useradd -m git
passwd -l git
The git account is created without a usable password, so the passwd -l step is mostly a safeguard. The account is reachable only through su from root until Gitolite installs the first key. If key logins are later refused for the locked account, check that UsePAM yes is set in /etc/ssh/sshd_config. That is the CentOS default.
2. Create the admin key (on the workstation)
ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa_git_admin
scp ~/.ssh/id_rsa_git_admin.pub root@<server-ip>:/tmp/admin.pub
The file name decides the Gitolite user name, so admin.pub creates the user admin. The key must be readable by the git user. The default file mode of scp is usually enough.
3. Install and set up Gitolite 2.x (on the server)
su - git
mkdir -p $HOME/bin
git clone https://github.com/sitaramc/gitolite
cd gitolite
git tag -l
git checkout <latest-v2-tag>
src/gl-system-install $HOME/bin $HOME/share/gitolite/conf $HOME/share/gitolite/hooks
export PATH=$HOME/bin:$PATH
export EDITOR=vi
gl-setup /tmp/admin.pub
Replace <latest-v2-tag> with the newest 2.x tag listed by git tag -l. gl-setup opens ~/.gitolite.rc in the editor. The defaults are acceptable, so save and quit. A successful setup creates ~/repositories/gitolite-admin.git and writes the forced-command entry for the admin key into ~/.ssh/authorized_keys. Add export PATH=$HOME/bin:$PATH to ~/.bash_profile if the shell is used again later.
4. Point the workstation at the server
Host git-server
HostName <server-ip>
User git
IdentityFile ~/.ssh/id_rsa_git_admin
IdentitiesOnly yes
Put this block in ~/.ssh/config. Then check access and clone the admin repository:
git ls-remote git-server:gitolite-admin
git clone git-server:gitolite-admin
If the key is accepted, the first command lists references and the second creates a gitolite-admin directory containing conf and keydir.
5. Add a developer and a repository
cp /path/to/lars.pub gitolite-admin/keydir/lars.pub
Add this to conf/gitolite.conf:
repo project-viking
RW+ = admin
RW = lars
Then commit and push:
cd gitolite-admin
git add .
git commit -m "Add repo project-viking and user lars"
git push
The server-side hook compiles the rules and creates the empty bare repository. Lars can then run git clone git@<server-ip>:project-viking with his own key. RW+ permits non-fast-forward pushes and ref deletion. RW does not, which protects history from accidental rewriting.
6. Back up
Save the following as /root/backup_git.sh:
#!/bin/bash
set -e
DATE=$(date +%Y-%m-%d)
TAR_FILE="/backup/git-backup-$DATE.tar.gz"
tar -czf "$TAR_FILE" -C /home/git .
scp "$TAR_FILE" user@backup-server:/storage/
Archiving the whole home directory captures repositories, .gitolite.rc and the SSH key entries. Make the script executable, create the target directory, and schedule it:
mkdir -p /backup
chmod 700 /root/backup_git.sh
crontab -e
Add this line to the crontab:
30 2 * * * /root/backup_git.sh
The scp step needs a passwordless key for the root user on the backup host. Test a restore on another machine before relying on the archive.
Limits and alternatives
- Consistency. A plain
tarof live repositories can capture a repository mid-push or mid-git gc. For busy servers, take a filesystem or LVM snapshot first, or schedule the backup in a quiet window. Agit clone --mirrorof each repository is a slower alternative that always produces a valid copy. - Version drift. The setup shown here belongs to the Gitolite 2.x generation. A later major version changes the install and setup commands, and the migration path should be read from the project documentation before upgrading.
- No web interface. There is no browser, no issue tracker and no merge request workflow. Pair it with a read-only web viewer such as gitweb if one is needed, or choose a web suite.
- Latency. Hosting near the developers shortens round trips. It does not speed up the work Git does on the developer's own machine.
- Compliance. Server location is only one input. Retention, access logging, encryption of backups and the legal basis for storing personal data still have to be handled. A repository should not contain personal data in the first place.
- Passwords and keys. Keep password logins disabled for the whole server with
PasswordAuthentication noinsshd_configonce the admin key works. RSA keys of 2048 bits were the accepted baseline in 2012, and 4096 bits is a reasonable option for new keys.