Nested (recursive) submodules in Git

Git submodules

Git submodules are the way to implement dependencies between projects.

Let's say you have a common libray 'mylib' shared between apps 'myapp1' and 'myapp2' then you will keep only 3 repositories (1 for each of them).

Both 'myapp1' and 'myapp2' will have 'mylib' declared as a submodule, which is cloned from / pushed to its own repository. And the repository of each parent app will only keep a reference to the commit to use from the submodule repository.

Nested Git submodules

Extending the same principle, 'mylib' could as well be dependant on 'somebaselib', declared as a submodule and hosted elsewhere.

There's plenty of documentation about how to clone or update such repositories (once they exist), using

   git clone ... --recursive
   git submodule update --recursive

But how to build one of those is not so intuitive.

Adding nested submodules: the problem

You may have used this procedure to add a submodule into the 'mylib' repository:

   cd ~/mylib
   git submodule add git@someurl/somebaselib baselib   

As the baselib folder does not exist, the previous command does a clone of the specified repository into that folder, and then adds it as a submodule.
You may think the same would to add the nested submodules to the superproject:

   cd ~/myapp
   git submodule add git@someurl/mylib lib

but unfortunatelly this does not work... because the cloning fails.

Adding nested submodules: the solution

The workaround is to manually clone the nested submodules into the subfolder with the --recursive option, before doing the 'submodule add'

   cd ~/myapp
   git clone git@someurl/mylib lib --recursive
   git submodule add git@someurl/mylib lib

Now, as the lib folder does exist, the 'submodule add' command does not try to clone it and just adds it as a submodule. You are all set!

Git labeller, branches and dynamic values

After some use of the Git labeller for CCNET described in my previous post, we were faced with an unexpected problem.

On a project following the basic workflow described here, we had the following scenario:

• At some point of time, release tag 'v1.1.200.0' was placed on 'qa' branch and that version went into production
• Later, while development continued on the 'master' and 'qa' branches, a fix was needed for the previous release, so we've created a branch 'release1.1.200' from the last release tag, commited the fixes into it, and placed a new tag 'v1.1.200.1' on the last commit of the release branch - pushing both the commits and the tags to the origin.
• On the CCNET automated build, when getting the code using the Git source control plugin, we specified 'release1.1.200' as the branch to use, and expected to get '1.1.200.1' as the CcNetLabel

But instead of that, we kept getting a '1.1.200.0' label  ¿?

By following the CCNET server log, we've found out that the sequence in which the Git source control plugin works is:

1. repository is cloned or updated
2. labeller is invoked
3. branch specified in CCNET configuration is checked out (!)

So when the labeller tries to get the last tag by invoking git describe, it receives something like 1.1.200.0-xxx where xxx specify the commits present after that tag.

Once the problem was clear, the solution was to add to the labeller configuration an (optional) element specifying a branch to ckeckout, and to do so within the labeler before issuing git describe.

As a final touch, we've also upgraded the labeller to support dynamic parameters, so the name of the release branch could be specified dynamically when launching the build. The updated labeller is available at its Gitub project .

Git tag labeller plugin for CCNET

When implementing the basic Git workflow described in my previous post, we needed a way to read the last version tag and other related info and pack it as the "build label" within CruiseControl.Net, the integration platform we use.


More precisely, we needed a valid version identifier to be built in two different scenarios:

• Release builds: take the release tag from Git (i.e. "v1.0.4.0"), skip the "v" prefix and return the rest as the version ID.
• QA builds: we do not tag each QA build, so we need to take into account the last tag and the number of commits ahead that, then merge them into something like "1.0.4.105" meaning "5 commits ahead of release 1.0.4.0".  Why 105 and not 5?  Well, we are reserving a block of  numbers (1.0.4.1 - 1.0.4.100) to accommodate eventual  production fixes on a dedicated release branch.

Enter Git tag labeller plugin for CCNet - we built a plugin to be used as a block in a CCNet configuration script and posted it on GitHub, you can find it here.

Besides the functionality we needed, we added a couple of features some one else may use, like a simple concatenation of the existing git tag with the commits ahead count; i.e. "ReleaseCandidate.6".

See the GitHub project for the most up-to-date documentation and code.

Get Going with a Minimalistic Git Workflow

Yeah, I know. The workflow in Git, as in any DVCS, is something to be taken seriously. Once your whole team is fluid in branching, merging and rebasing you are ready for a full blown Git workflow like git-flow, or the GitHub flow described by Scott Chacon.

But what if you project is small, and / or you've just migrated from SVN and are still trying to figure it all out, and / or you just don't want a model so complicated?

What would be a good starting point? If you need to, lets say:

1. commit or merge code into a main branch with CI
2. regularly set milestones for deployment in a QA or staging server
3. from time to time release a production version - that's what you are paid for, after all
4. do some maintenance and bug fixing on the released code.

After some research and some practice we choosed the following minimalistic workflow to solve those four needs.


1. The master branch is the place where all developers integrate their code.

This may be done:

• By committing their code directly into the master branch - when working in small teams or for small incremental changes.
• On a separate feature or develop branch that is later merged into master - when the circumstances require it.

CI tests should be running on this branch.


2. A fast-forward-only qa branch acts as a "sliding tag" following closely the master head.

This allows you to set temporary marks that will be used for deploying in a QA  or  staging environment. Some of them will later turn into releases - see the following section.


3. A series of annotated version tags signal each release that will go into production.

The version tag for a release (i.e. v1.0.2.0) is always placed on the commit at the current head of the qa branch (not from the master branch, which may be ahead of the last quality assured build).

Note that in this model, only an annotated tag is initially required for deploying a release into production. Only later, if required, a dedicated branch will be created from this tag.


4. Dedicated branches are created for each release that requires maintenance or fixes.

On early stages of an application development , releases may occur quite frequently and most of the times they will not be updated with fixes - the fix will be included in the next full release from the master / qa  branches.
In these cases, the version tag mentioned in section 3 will be enough.

However, when the app or site is in production and releases tend to be more sporadic, the requirements for fixes on the released version are quite common. On the first occasion that a fix is required on a newly created version - initially defined only by a tag - a dedicated branch for that specific release should be created (i.e. release1.0.2)

Each fix release for this major version will be created on the same dedicated branch, and will get a separate tag (i.e. v1.0.2.1, v1.0.2.2, ...)

Changes on the release branch may be merged back - as a whole or cherry picked - into the master tag; never into the qa branch that is following the master in a fast-forward way.

The next full release (i.e. v1.0.3.0) will be created as a version tag on the main qa branch, and later branched from there, if necessary.

Credits

The fast-forward qa branch and release tag concepts are well explained in the two last sections of this post from Rein Henrichs.

My friend Benjamin Eidelman was a key participant in the discussion and contributed many of the basic ideas behind this post.

Git, Nuget packages and Windows line-endings

The problem

So, you've done your homework and gathered a top set of tools for your web development on a Windows environment: Visual Studio 201x, Git for distributed source control, and a handful of best-of-breed components, i.e.:

• jQuery and jQuery.UI
• Bootstrap on 'less' as CSS base framework
• RequireJs for script loading and dependency handling.

You have added all these third-party components to your solution as Nuget packages.

When setting up your Git repository, you've followed the widely recommended practice for a Windows environment: CRLF conversion to LF when commiting to the repository, and reverse conversion when getting files back into your working area.

So far, so good... or not?

The problems arise when a new version of a package is available and you try to update it.

During the process, the existing package is uninstalled prior to installing the new version, and then a long list of warnings are shown:

Skipping 'Scripts\less.min.js' because it was modified.
Skipping 'Content\less\accordion.less' because it was modified.
Skipping 'Content\less\bootstrap.less' because it was modified.
...

Following that, the installation of the new package fails to update those files:

'Scripts\less.min.js' already exists. Skipping...
'Content\less\accordion.less' already exists. Skipping...
'Content\less\bootstrap.less' already exists. Skipping...
...

Maybe you did modify a few of those files - i.e. 'variables.less' which is meant to be customized - and that is the expected behavior for that file. But for the rest of them, which you never intended to modify, the development workflow is definitely not working.

Lets try to isolate the problem from a fresh start. Assuming you don't have a certain package in your solution (nor any of its files), if you open the Package Manager Console and do:

PM> Install-Package Twitter.Bootstrap.Less -Version 2.0.3

This will add a bunch of files, in this case .less and .js. After that, if you open a Git console and start tracking the new files you'll notice the following:

$ git add .
warning: LF will be replaced by CRLF in ... Scripts\less.min.js
warning: LF will be replaced by CRLF in ... Content\less\accordion.less
...

This smells like trouble... and it is.
If you further commit this changes and then checkout to another branch and then commit back to the current one (causing a refresh of your working folder):

$ git commit -m "Package added, v 2.0.3"
$ git checkout elsewhere
$ git checkout master

This lets your working directory with an altered copy of your original files (CRLFs instead of the original LFs). If you further try to update the package

PM> Update-Package Twitter.Bootstrap.Less -Version 2.0.4

You'll get the ugly warnings shown at the beginning of this post, and none of your files will be updated.
Obviously, this blind "safe behavior" that cannot tell your modified files from the rest is not what you want for your project.

Workaround

When the conflict has already gotten to this point, you have no choice other than:

1. Uninstall the package
2. Manually delete each of its remaining files - except for the ones you did willingly modify
3. Install the new version.

But you may prevent it from happening again on the next update. And this is where you should revisit the decision factors on the line-ending behavior of your Git repository.

 The Right Way   Then what?

In almost every current Windows developer tool, LF-ended lines are no longer a problem.

The shameful exception is Notepad - are you still using it for development tasks? Seriously? Come on...

Notepad++ is one among the many good replacements you can find out there.

In my opinion, if you decide to use certain js / css / less Nuget packages (built for Visual Studio and the Windows environment) because you trust their individual or collective authors, why wouldn't you abide to the line-ending conventions they have chosen?

At a quick glance, all of the components mentioned at the start of this post are using LF line-endings for their text files ( .js, .css, .less)

That's enough motive for me to use the same convention for those types of files. So, while keeping the CRLF line-ending for specific windows-generated text files (i.e. .sln and .csproj files), we are switching our Git repositories to not apply the CRLF to LF and reverse conversions for the extensions .js, .css and .less.

This is done with a .gitattributes file placed at the root of each repository - look at this help file for an explanation of the use of such file and its advantages. For the issue discussed in this post, this is all you need in your .gitattribute file:

# Set default behaviour, in case users don't have core.autocrlf set.
* text=auto

# Explicitly declare files we do not want to be converted 
*.js -text
*.css -text
*.less -text

And you are done! All those files will keep their original LF line-endings, both in your working folder and in the repository.

Going further, you may opt to enforce the conversion to LF line-endings for those file types even when you accidentally create a file of your own with CRLFs. This would result in better compliance with the "LF only" recommended practice for Git repositories.

To do that, change the .gitattributes file as follows:

# Set default behaviour, in case users don't have core.autocrlf set.
* text=auto

# Explicitly declare files handled as LF-ended (converting them if necessary) 
*.js text=input
*.css text=input
*.less text=input

WARNING: If you apply any of the preceding changes on an existing repository, action must be taken to avoid Git from detecting all affected files as modified on the next commit. You should follow the steps outlined on the Re-normalizing a repo section at the end of the previously mentioned help file.

Acknowledgments

This post from Tim Clem was very enlightening about the handling of line endings in Git. Thanks!