July 07, 2014 • • ∞
It’s been well over a year since HgLab 0.1 was first released. Back then, it barely had any functionality at all, let alone any kind of logo.
Now that HgLab has become a full-featured commercial product, one of the things it needed is a nice logo. So, without further ado, please welcome:
June 20, 2014 • • ∞
Back when DVCSes were all new and yet unexplored, one of the touted benefits was the ability to work offline, be it because you are on the plane or because code hosting site went down. While true, you still need to communicate with the outside world, and when your Internet connection gets all cranky, this can become an ordeal of its own.
If you ever used Mercurial over an unreliable connection, you might be wondering why can’t it resume broken clones and why, oh why can’t it do partial pulls and pushes. The reason for the former is purely technical and will be explained shortly, and the latter is simply not the case.
Mercurial communicates changesets by streaming changelog followed by manifest data followed by file data. This minimizes seeking both for client and server, and maximizes on-the-wire compression.
This design has two implications for the viability of “resume”.
First, a clone or pull operation gets interrupted in the middle, odds are that not a single valid changeset was ever transferred (since you might be missing the associated manifest and filelogs) so there’s no consistent point to resume an interrupted operation from.
Second, if a repository gets a new commit between starting a pull and resuming a pull, the contents of a new stream will change at various places, with new changesets, manifests and filelogs appearing in the middle of the stream. This makes it impossible to implement something similar to byte serving, where client can request a portion of a response. Nor is it possible to resume transfer at an arbitrary changeset.
With that in mind, here are all the ways that can help you work around broken connections.
Depending on how flaky your connection is, there are two options for performing initial clones.
First, you can try so-called “streaming clones”. These minimize TTFB, but do generally require a bit more data to be transferred.
Here’s how to do a streaming clone:
$ hg clone --uncompressed http://live.hglabhq.com/hg/hgsharp/hgsharp
Note that HgLab can enforce this behavior for all compatible clients.
Your second option will be a
hg clone –-rev operation, followed by a number of incremental pulls. This behaves similarly to cloning a repository in some distant past and doing occasional updates.
$ hg clone --rev 5 http://live.hglabhq.com/hg/hgsharp/hgsharp
With incremental pulls you can get the entire repository on a changeset-by-changeset basis, thus reducing the amount of data transferred during each operation to an absolute minimum:
$ cd hgsharp $ hg pull --rev 10 http://live.hglabhq.com/hg/hgsharp/hgsharp $ hg pull --rev 20 http://live.hglabhq.com/hg/hgsharp/hgsharp $ hg pull --rev 21 http://live.hglabhq.com/hg/hgsharp/hgsharp
In true Mercurial spirit (which means, among other things, being consistent throughout the entire system), pushes work very similarly. To get all fancy, here’s a bit of revset-fu to do incremental pushes:
$ hg push --rev "limit(sort(draft(), rev), 2)"
What this does is sends first two draft revisions to a remote repository. Of course, this works just as good:
$ hg push --rev 42
There’s one command to watch out for when you are struggling to keep connected. This command is
hg incoming and here’s why you should care.
hg incoming does is it shows which changesets from the remote repository are not yet in your local one. The way it works is it pulls all the missing parts of history locally, displays the log and then disposes of the bytes it had received. This means that, in case of broken connections, it can very well fail or waste precious bandwidth.
Hopefully, the next time you find yourself in a situation like this you’ll know exactly what to do to get the work done.
June 11, 2014 • • ∞
There are a lot of hard things when it comes to Computer Science: cache invalidation, naming things, off-by-one errors and, as it turns out, agreeing on the actual number of said hard things.
Good names capture the role and function of the concept being named, which, in turn, affects how you or others can understand and reason about stuff. Bad names complicate perception, muddy waters and can often lead one to incorrect conclusions.
One of the illustrations of the phenomenon of a “poorly chosen name” is the concept of “editing history” in the context of a distributed version control systems.
In any nontrivial software project, code is read much more often than it’s written (that is, unless you’re writing in Java). Good code and good comments alike make it simpler to understand what the code does and how it does what it does. Version control systems complement actual code and make it simpler to understand why it was written.
Now, just as code, history can be good or bad. Bad history is easy to spot, and we’ve all seen it. Empty commit messages or messages similar to “i dunno, maybe this works” or “Oops, forgot the comma” or “!!!DO NOT CHECKOUT – DOES NOT COMPILE!!!”. Elephant-sized commits affecting dozens of files with no rhyme or reason.
History in a version control system should be about logical changes that lead to introduction of a new feature, bugfix, performance tweak. It should not be about “raw” and “unprocessed” changes. It should not be about editing physical bytes. Rather, it should be about why the code was written this particular way.
DVCSes allow us commit early and often, but with great power comes great responsibility. Nobody wants to see how you fix a typo in a variable name or backtrack after programming yourself into a corner. It’s a sign of a good software engineer to pretty up your commit history before unleashing it in public.
First of all, let me show what real history rewriting looks like:
Dictatorship aside, here are the reasons why I think “editing history” is a misnomer.
The first one is technical. “Editing” or “rewriting” means taking something and changing it in-place. Yet if you
commit -–amend or
rebase (either in "stock" Git or with Changeset Evolution enabled in Mercurial) you are not changing history. You create an “alternate” history, in which your commits unfold differently than they originally did. There’s nothing destructive about it, contrary to what the name “editing history” implies.
The second is social. Every so often you come across an article postulating that “Pushing rebases is evil!”, where authors tell their tales of grave horror about what happens when one exploits the superpowers of editing history. This mindset has its roots in limitations of tools; we just need to version control our version control.
I don’t expect to turn the world upside down with the revelation of mine, but I think I’m onto something here.
When you’re cleaning up history before letting the genie out of the bottle and pushing it upstream, you’re not “editing” or “rewriting” history.
You are deciding history.
May 24, 2014 • • ∞
HgLab, a behind-the-firewall self-hosted Mercurial server, source control management system and code collaboration platform, has been updated to version 1.6.
Since introducing an updated installer for HgLab 1.5, there was a number of complaints about how cumbersome it was to launch an MSI from the Elevated command prompt. HgLab 1.6 fixes this by wrapping the MSI package in a what is called a “bootstrapper”. A bootstrapper is essentially an EXE files with an embedded manifest which forces UAC dialog to prompt for Elevation from the get-go. This means that HgLab no longer requires any secret incantations to start the installer: it can now be launched just by double-clicking the setup application.
With this release HgLab gets proper Teams. Previously, you could effectively have only one Team per Project. From now on, HgLab supports multiple Teams, each with varying Roles and Members. This allows for greater flexibility and lays the foundation for a much more sophisticated functionality down the road.
If you are using HgFlow in your development, HgLab now detects that and does something nice. The “Branches” tab gets replaced with a brand-new “Flow” tab, where you can see all the development streams going on.
The “Flow” tab has all kinds of smarts: it groups substreams according to where they originated from, correctly displays divergence information and has appropriate comparison behavior.
ProjRc enables seamless configuration sharing across the organization. With this release, HgLab becomes a ProjRc-compatible server and can be configured to send your environment-specific settings to all the clients. Enable extensions, remap subrepositories, configure defaults and aliases – all within the HgLab web interface.
There’s also a number of bug fixes and improvements, as usual.
Previously, being a Project Lead had little to no difference as to what you could do in the project. From now on, Project Lead receives unlimited rights within the project: add repositories, manage Teams, add and remove Team Members and Roles. Be careful!
A couple of most prominent bugs were squashed:
April 29, 2014 • • ∞
Mercurial is reaching version 3.0. Code freeze has begun on April 17, which means that Mercurial 3.0 will ship in early May. Here’s an glimpse into what’s going on with Mercurial development, as well as a short overview of upcoming new features, experimental and otherwise.
This feature has been available as an Evolve extension since Mercurial 2.1, but is still under active development and is not enabled by default.
While Mercurial enables distributed version control, Changeset Evolution is a set of features that enables distributed history rewriting. “Distributed” is key here. Currently, there are only partial solutions to the problem of history rewriting.
hg rebase and
hg histedit are limited in what they can do, have trouble communicating changes with the rest of the team and make it hard (or nearly impossible, depending on which side you’re on) to get previous versions of changesets.
Mercurial itself tracks changes to files. Changeset Evolution enables tracking changes to changesets, recording which of them got deleted or superseded by a new version of the changeset. Obsolescence markers is what enables this functionality. Note that nothing is ever rewritten; obsolete changesets simply are hidden and can always be recovered.
Now, to enable collaborative history rewriting, this information has to be somehow communicated between parties. This is where Bundle2 comes in.
Bundle2 implementation is still experimental in Mercurial 3.0 and should be enabled around 3.2 timeframe.
When cloning, pushing and pulling changesets (and running
hg bundle), Mercurial exchanges changesets in the format called Bundle. Although battle-tested and extremely reliable, this format has a number of shortcomings:
Bundle2 will be the replacement for current Bundle format, featuring better compression, extensibility and bidirectional communication. Additionally, it will significantly decrease the number of HTTP requests sent to the Mercurial server when performing pushes and pulls, because what currently requires separate trips to the server (get bundle, get bookmarks, get phase markers) will be shipped to the client in one go.
Previously, Mercurial was not very smart about HTTP authentication, very much resembling a goldfish: it would try to connect to the server, receive 401, get authentication information (either from the user or from Keyring) and retry the request, authenticated, only to forget to authenticate the next request. This behavior was particularly bothersome for pushes, when all the effort of transmitting a potentially large bundle was wasted because the request was unauthenticated in the first place:
GET /hg/server?cmd=capabilities HTTP/1.1 401 260 - mercurial/proto-1.0 GET /hg/server?cmd=capabilities HTTP/1.1 200 147 - mercurial/proto-1.0 POST /hg/server?cmd=unbundle HTTP/1.1 401 260 - mercurial/proto-1.0 POST /hg/server?cmd=unbundle HTTP/1.1 200 42 - mercurial/proto-1.0
This is going to change in 3.0: Mercurial will remember that it needs to be authenticating and will not be trying to break into a password-protected door:
GET /hg/server?cmd=capabilities HTTP/1.1 401 260 - mercurial/proto-1.0 GET /hg/server?cmd=capabilities HTTP/1.1 200 147 - mercurial/proto-1.0 POST /hg/server?cmd=unbundle HTTP/1.1 200 42 - mercurial/proto-1.0
hg push on a newly created repository, Mercurial errored out with a rather cryptic message:
$ hg init $ hg ci -m "initial commit" –A # assuming there were changes $ hg push pushing to default-push abort: repository default-push
Those familiar with Mercurial will immediately know what’s wrong here. Those who aren’t, however, are left to their own Google-foo. In 3.0, Mercurial will get a bit more helpful and offer to consult an exact section of a super-helpful built-in help.
More detailed help comes from yet another angle, future-proofing existing Mercurial.
Mercurial will get a new command, called
hg config. It can be used in two flavors: running
hg config will print names and values for all configuration items. Running
hg config --edit will launch a configured editor to tweak a user-level configuration file. With
--local switch you’ll be editing a repository-level configuration file, and with
--global you’ll be editing a system-level configuration file.
April 29, 2014 • • ∞
HgLab, a behind-the-firewall self-hosted Mercurial server, source control management system and code collaboration platform, has been updated to version 1.5.
There were several reasons for dropping support for Microsoft Windows Server 2003. First, it is already on the Extended Support which is rapidly nearing its end in 2015. Second, very few HgLab customers are actually using it (if you are one of them, please do contact HgLab Support – there’s no need to worry).
The third, and the most important reason, is that I just could not deliver the desired installation experience. MSI and WiX make it nearly impossible to achieve what I was envisioning on Windows Server 2003, so something had to give.
From now on, HgLab Installer will be the one responsible for creating and managing AppPools, Web Sites and Web Applications in the Microsoft Internet Information Services. This means that you won’t have to manually configure anything in the Internet Information Services Manager.
HgLab Documentation has been given the attention it deserved. Head over to see the Getting Started Guide, check out Prerequisites and follow the detailed Installation Guide. User Guide contains a lot of useful bits on how to use each and every feature of HgLab, and Administrator Guide provides all the information required for managing the HgLab installation.
With HgLab 1.5, the only users who will be able to use HgLab are the ones who have Collaboration permission and System Administrators.
Refer to Security Model to see how Users, Groups, Roles and Permissions play together.
This release also features:
March 25, 2014 • • ∞
Welcome HgLab 1.4. This release features an exciting new feature called “LAN Optimizations”
Previously, cloning an entire Mercurial repository (55 Mb, almost 21,000 changesets) took a bit over a minute (1:16, to be exact):
Now watch this:
Fifteen seconds. That’s almost 5x speed increase for initial clones. Impressive, isn’t it? What’s even better is that no configuration tweaking is required on the client side. HgLab itself enforces this behavior, with (compatible) Mercurial clients silently obeying.
Of course, nothing in the computing world comes for free, and this is no exception. When you are enabling LAN Optimizations, you’re trading speed for bandwidth. In corporate scenarios bandwidth is practically unlimited, so enabling LAN Optimizations for, well, LANs is a no-brainer. Additionally, it saves a bit of CPU cycles since HgLab doesn’t have to do all the complex computations involved in building a bundle.
For as long as I can remember (since 2006, actually, but that’s like 50 years Internet-time), Mercurial had an unfortunately named
--uncompressed switch for
hg clone http://firstname.lastname@example.org/hg/mercurial/hg --uncompressed
With this flag set, Mercurial client asks server to perform for a what is called a “streamed clone”, when server just takes all the revlogs from the repository, glues them all together and streams the entire thing back to the client. With LAN Optimizations enabled, HgLab effectively enforces this streaming behavior even when client doesn’t explicitly asks for it.
Streaming clones, however, are not available if you’re doing any of the fancy-pants cloning, like
hg clone --rev,
hg clone -–branch or
hg clone http://...#revision-or-branch-or-tag. This is by design, because when you’re cloning only a subset of history, there’s no reason to transfer the repository in its entirety, so Mercurial has to revert to a traditional cloning behavior.
Just check the checkbox. Go to Administration, then to Repositories and enable “LAN Optimizations”. No reboot, no restart. Changes are effective immediately:
This release also has a number of minor User Interface tweaks, and discovering them is left as an exercise to the reader.
March 11, 2014 • • ∞
HgLab 1.3 has just been shipped. With this release HgLab gets revamped security model as well as several new features, with most prominent being Commit Approvals and Source Code Archve Downloads.
Below are the highlights of HgLab 1.3.
There’s now yet another way to track of All Things HgLab: Atom feeds allow you to get updates directly in your favorite RSS reader. Click the Feed link and you’ll get either a full feed for all your watched projects or for a specific project.
Of course, feeds support Auto Discovery.
Previously, the only way to get source code from a repository was to clone it with Mercurial. While being appropriate for traditional collaboration models, this was a big roadblock in certain scenarios.
See this tiny button?
It allows you to abandon that version-controlled distributed universe and download a plain old ZIP file with all the source code for a particular changeset, branch or tag.
HgLab now has a light-weight approval process that allows teammates to Approve a commit, essentially green-lighting changes.
Later, when browsing commits, HgLab displays the total number approvals for a particular commit as either a gray or a green circle. Gray badge shows how many reviewers have approved a particular change. The badge turns green for the commits that were reviewed and approved by you.
There is a number of changes to the Security Model in this release.
Previously, “Anonymous” user was used to grant public access to a project. This is now replaced by an independent checkbox in Project Settings.
HgLab now splits permissions into two silos: System Permissions and Role Permissions.
System Permissions are assigned to Groups and are (and will be) used to restrict access to system-wide areas of functionality, such as ability to manage Projects.
Role Permissions are, as the name implies, assigned to Roles (which are in turn granted via Team Membership) and are used to restrict narrower areas of functionality, such as ability to manage Repositories and Teams.
Neverending quest for great UI and UX continues. This release features expandable commit messages (see the little ellipsis in Message column in Commits tab), slightly redesigned Compare page and Diffstats and improved language detection for code highlighting.
That’s it for this release, but there’s a lot in the works.
January 17, 2014 • • ∞
Yesterday it was announced that Microsoft will not be adding support for Mercurial to Team Foundation Server.
I see two reasons why this might have happened.
There is a way of comparing and contrasting Git and Mercurial I came up with recently. Git is simple on the inside (with its ingeniously simple and elegant storage format) but overly complex on the outside (with its cumbersome and often illogical CLI). Mercurial, on the other hand, is quite complex on the inside, but has a refreshingly simple CLI.
This observation has far-reaching consequences.
Git, with its simple and-well documented storage format, makes it easy to do a clean room implementation of Git Core in any programming language. Clean room implementation effectively frees authors of GPL restrictions and allows this code to be used in any closed-source product.
Even if not for a possibility of a clean room implementation, libgit2 (which is what Microsoft uses internally) is licensed under GPLv2 with Linking Exception, so incorporating this library into Visual Studio and Team Foundation Server is a no-brainer technology-wise and licensing-wise.
When it comes to Mercurial, matters are significantly more complex here. Low-level Mercurial internals are documented only to a degree, which makes it impossible to implement Mercurial Core using clean room approach. This means that any reimplementation will automatically be treated as derivative work, which forces it to be licensed under GPL, which in turn prohibits usage of the library in a closed-source commercial project.
It’s unclear to me if this issue can be resolved at all. If Microsoft eventually decides to add support for Mercurial to Team Foundation Server, there are several options for them here:
Comparing absolute numbers of developers using either VCS, Git is indeed more popular than Mercurial, so with Team Foundation Server integrating with Git Microsoft caters for a larger target audience.
Or it does not?
In absolute numbers Git is more popular, but Mercurial is arguably more popular in a corporate world and even more so in companies that are building on Microsoft stack and are using Microsoft products, client and server alike. It’s unlikely that the OSS community is the target market for either Visual Studio or Team Foundation System.
This is a minor point that is unlikely to have influence Microsoft’s decision, but it’s worth mentioning nevertheless.
Mercurial plugin ecosystem is richer than that of Git and, thanks to a nicer API of Mercurial, its extensions seamlessly integrate into Mercurial proper. Most notably, hg-git is significantly simpler to install than its’ Git counterpart, git-remote-hg, with latter being less feature-rich.
Again, this alone is unlikely to have had any significant weight in a decision to decline the UserVoice request, but it might have left Microsoft thinking that this will somehow satisfy us Mercurial aficionados.
All in all, this is an unfortunate piece of news for several thousand developers and several hundred companies. Let’s see what the future holds.
January 14, 2014 • • ∞
It’s been over a month since HgLab 1.0 and it’s high time for an update, so here goes: HgLab 1.1.
Up until now HgLab only had commit-level discussions. Welcome inline commit discussions:
Discuss with style: inline discussions support Markdown, Emojis and @mentions.
Some teams prefer their mainline branch to be something other than
default. Therein lies a catch: newly cloned repositories gets automatically updated to
default branch and commits often end up in the wrong place.
To solve that, Mercurial 2.4 introduced a special bookmark called
@ to which all fresh clones get updated to. Where does HgLab come into play here? Well, if you change your Mainline Branch to something other than
default (and you don’t happen to have
@ bookmark already), HgLab will manage this oh-so-special marker by itself, allowing you to always be on the correct path.
Your favorite repositories are now even more accessible. Clicking that tiny arrow opens up a handy list of all your Starred Repositories.
Each Repository page now comes with a nice language statistics graph which shows the language breakdown.
Speed is a feature, and I continue to work on improving HgLab performance. This release is no exception: HgLab became much smarter about caching and cache invalidation.
These are the big features in this release, but there’s also a lot of small fixes that contribute to the overall experience: updated icons and
.hgignore templates, improved language detection, numerous UI and UX fixes, tweaks and improvements.
These 19 branches you saw on the screenshots above – they are there for a reason. This is just the beginning, and will be many new and exciting features. Stay tuned!
November 26, 2013 • • ∞
Today I’m officially launching HgLab 1.0, a self-hosted, behind-the-firewall Mercurial server and source control management system for Windows.
For many companies, hosting an internal source control server and maintaining access to repositories can be a difficult and time consuming process. Now, you can focus on building great software and let HgLab take care of your repository management.
Mercurial has always had excellent Windows support on the client-side. With HgLab, server-side gets equally great treatment. HgLab runs on Microsoft Windows Server 2003 and above and requires only Microsoft .NET Framework, Internet Information Services and Microsoft SQL Server: everything you are already familiar with.
What’s more, HgLab integrates seamlessly with your LDAP server and provides all the benefits of centralized user and group management.
HgLab delivers you a central, secure solution to create and manage distributed repositories, on your own servers. Developers can trust that they will find the latest official version of a project. Managers can trust that users’ access to code is appropriate for their role, and assigned with minimal administrative overhead.
This is really big: HgLab has now grown out of its beta pants and turned into a mature, real-world application. Over the past 18 months HgLab has come a long way, but there’s an even longer road ahead. My goal with HgLab is to build state-of-the art code collaboration platform for Mercurial and Windows, and version 1.0 is just the beginning.
November 21, 2013 • • ∞
About 18 months ago I made a first commit to the HgLab repository. From the get-go I had several major goals in mind:
After 1.5 years in the making, I think I have first two points covered.
HgLab runs on Microsoft Windows Server 2003 and above and only requires .NET Framework, Internet Information Services and (for now) Microsoft SQL Server 2008 (Express edition or any later version will do just fine); it’s trivial to install and upgrade.
On the management side HgLab has a lot to offer too: web-based interface for managing Mercurial repositories and users, integration with ActiveDirectory, roles, groups and teams and much much more.
All in all, it’s been a bumpy ride and I have quite a few war stories to tell about IIS, WiX and .NET Framework. Yet, fast-forward 1 300 commits — and HgLab 1.0 is here.
With this release HgLab will be turning into a commercial product. It’s here for the long haul and I believe it has a much bigger chance of of succeeding if it has people committed to doing the not-so-fun things (as opposed to doing “fun” features), producing a very useful, high-quality and reliable product that you can trust your source code with and that use to do what you do best: build awesome software.
From now on, HgLab will be distributed as an MSI package. This is a much cleaner approach compared to what was available previously. Upgrades are handled automatically, so no more hand-picking which files to delete and which to leave out.
One thing to note here. HgLab itself is compiled as AnyCPU and actually runs as 64-bit process on 64-bit systems, but since there is no such a thing as as “AnyCPU MSI” and since I didn’t want to complicate things by offering two versions of installer, there will only be an x86 MSI.
HgLab will be licensed under a perpetual licensing model. Your license key allows you to use any release of HgLab that was available when you purchased the key, as well as any future releases (bug fixes, enhancements, new features, etc.) for 12 months from the date of purchase. After that 12 months, you can continue to use the version you have forever, or you can choose to renew your license key for another 12 months to continue getting updates.
There will be four licensing tiers for HgLab (plus a full-featured 45-day evaluation license), all providing the same functionality but differing in the number of users who will be able to access the system.
HgLab has now matured into a real application, and it's time for the beta period to come to an end. The current 0.4.x series of releases will be the last beta releases while I tidy a few things up. On November 25th I'll release the final 1.0 version, and this will be a farewell to the beta label.
To everyone who participated in the beta, you have my sincere thanks for helping to make HgLab the product it is today!
October 24, 2013 • • ∞
Last time I mentioned:
…more work related to improving stability, performance and reliability
What I didn’t mention was security. How thoughtful of me.
There is a regression introduced in 0.4.4 while fixing #345, which results in all projects being available to an Anonymous user as long as there is at least one project where an Anonymous was added as a Team member.
HgLab 0.4.5 fixes this issue.
October 23, 2013 • • ∞
Please, update to HgLab 0.4.5.
HgLab is nearing its 1.0 release, which means there will be less work put into new features and significantly more work related to improving stability, performance and reliability. Still, there are a couple new features in HgLab 0.4.
As it always is, upgrading HgLab is trivial.
Email notifications go hand in hand with Watched repositories. Whenever someone pushes to any of the repositories you are watching, you’ll get a nice concise email highlighting all the commits with links to pages where you can review each commit in greater detail and discuss it with your team.
Just as mentioned before, you can now discuss commits directly in HgLab. This feature will be serving as a basis for upcoming Pull Requests and Code Reviews.
While discussing commits, you can use Emojis, Markdown formatting and @mentions:
All participants will get subscribed to the discussion and will receive email notifications whenever there is a new post.
defaultbranch. See #314.
Huge thanks to all the beta-testers. Your feedback is truly valuable!
HgLab is a behind-the-firewall self-hosted Mercurial server and source control management system which gives you:
Interested in HgLab and Mercurial? Want to know when new releases are out? Join the HgLab HQ Mailing List for to get notified when something interesting happens.