June 20, 2014 •

Back when DVCSes were all new and yet unexplored, one of the touted benefits was the ability to work offline, be it because you are on the plane or because code hosting site went down. While true, you still need to communicate with the outside world, and when your Internet connection gets all cranky, this can become an ordeal of its own.

If you ever used Mercurial over an unreliable connection, you might be wondering why can’t it resume broken clones and why, oh why can’t it do partial pulls and pushes. The reason for the former is purely technical and will be explained shortly, and the latter is simply not the case.

Why Resume is Impossible?

Mercurial communicates changesets by streaming changelog followed by manifest data followed by file data. This minimizes seeking both for client and server, and maximizes on-the-wire compression.

This design has two implications for the viability of “resume”.

First, a clone or pull operation gets interrupted in the middle, odds are that not a single valid changeset was ever transferred (since you might be missing the associated manifest and filelogs) so there’s no consistent point to resume an interrupted operation from.

Second, if a repository gets a new commit between starting a pull and resuming a pull, the contents of a new stream will change at various places, with new changesets, manifests and filelogs appearing in the middle of the stream. This makes it impossible to implement something similar to byte serving, where client can request a portion of a response. Nor is it possible to resume transfer at an arbitrary changeset.

With that in mind, here are all the ways that can help you work around broken connections.

Cloning

Depending on how flaky your connection is, there are two options for performing initial clones.

First, you can try so-called “streaming clones”. These minimize TTFB, but do generally require a bit more data to be transferred.

Here’s how to do a streaming clone:

$ hg clone --uncompressed http://live.hglabhq.com/hg/hgsharp/hgsharp

Note that HgLab can enforce this behavior for all compatible clients.

Your second option will be a hg clone –-rev operation, followed by a number of incremental pulls. This behaves similarly to cloning a repository in some distant past and doing occasional updates.

$ hg clone --rev 5 http://live.hglabhq.com/hg/hgsharp/hgsharp

Pulling and Pushing

With incremental pulls you can get the entire repository on a changeset-by-changeset basis, thus reducing the amount of data transferred during each operation to an absolute minimum:

$ cd hgsharp
$ hg pull --rev 10 http://live.hglabhq.com/hg/hgsharp/hgsharp
$ hg pull --rev 20 http://live.hglabhq.com/hg/hgsharp/hgsharp
$ hg pull --rev 21 http://live.hglabhq.com/hg/hgsharp/hgsharp

In true Mercurial spirit (which means, among other things, being consistent throughout the entire system), pushes work very similarly. To get all fancy, here’s a bit of revset-fu to do incremental pushes:

$ hg push --rev "limit(sort(draft(), rev), 2)"

What this does is sends first two draft revisions to a remote repository. Of course, this works just as good:

$ hg push --rev 42

Watch Out

There’s one command to watch out for when you are struggling to keep connected. This command is hg incoming and here’s why you should care.

What hg incoming does is it shows which changesets from the remote repository are not yet in your local one. The way it works is it pulls all the missing parts of history locally, displays the log and then disposes of the bytes it had received. This means that, in case of broken connections, it can very well fail or waste precious bandwidth.

Happy Coding

Hopefully, the next time you find yourself in a situation like this you’ll know exactly what to do to get the work done.

And if you need a reliable and featured Mercurial server and collaboration platform, give HgLab a try.



comments powered by Disqus

HgLab is a behind-the-firewall self-hosted Mercurial server and source control management system which gives you:

  • Sophisticated Mercurial-based version control system
  • Straightforward setup on your servers
  • Powerful code collaboration platform
  • Seamless integration with your infrastructure - including ActiveDirectory

Get Started with Full-Featured 45-Day Evaluation →


Interested in HgLab and Mercurial? Want to know when new releases are out? Join the HgLab HQ Mailing List for to get notified when something interesting happens.

Subscribe to the HgLab HQ Newsletter →


Take HgLab for a Spin

Try HgLab now. Full-featured 45-day evaluation, no credit card required.