In case your are organising multiple git repositories and add them into one global, the most obvious choice is to use git submodule. It basically creates a pointer to a specific git commit hash in a remote repository allowing you to clone the repository into a sub directory as module.
Adding submodules is fairly easy, purging them can become cumbersome. When we were working on the Icinga Vagrant boxes one issue was to re-organize the used puppet modules into a central modules directory, as well as purge all local copies and instead use the official git repositories others provided.
Using git submodules turned out to be simple to add, but ugly to manage. Users normally forgot to initialise and update the submodules, and if the developers (me) decided to add/remove modules, it was always in sort of an incompatible check-out state. A fresh git clone –recursive always helped (hi Bernd) but in the end it wasn’t satisfying to work with as users struggled from a simple demo setup with Vagrant.
Looking for alternatives unveiled git subtree as originally suggested by Eric – instead of only adding a module and its commit pointer, you’ll add the repository and all of its commit history into your own git repository, as sub tree with directories and files. This also solves the problem that remote repositories might be gone, unreachable, or anything else hindering the successful clone.
There are several options like to squash the history into a single commit (like one would use git rebase) when adding a new subtree.
Add a subtree
When I was working on the Graphite/Grafana integration into the icinga2x box, I’ve just added the Grafana puppet module. The –prefix parameter defines the root directory for the cloned repository, then add the remote url, the branch and let it squash the entire commit history (–squash).
Git doesn’t like uncommitted changes so make sure to stash/commit any existing changes before adding a new subtree.
git subtree add --prefix modules/grafana https://github.com/bfraser/puppet-grafana.git master --squash
This results into two new commits:
commit 0b3e0c215e3021696fce3a37eff3274c174348a8 Merge: 482dc29 6d6fd37 Author: Michael Friedrich <email@example.com> Date: Sat Nov 14 18:47:39 2015 +0100 Merge commit '6d6fd37ec971314d820c210a50587b9d4ca2124b' as 'modules/grafana' commit 6d6fd37ec971314d820c210a50587b9d4ca2124b Author: Michael Friedrich <firstname.lastname@example.org> Date: Sat Nov 14 18:47:39 2015 +0100 Squashed 'modules/grafana/' content from commit 89fe873 git-subtree-dir: modules/grafana git-subtree-split: 89fe873720a0a4d2d3c4363538b0fa5d71542f41
Update a subtree
In case the remote repository should be updated to incorporate the latest and greatest fixes, you can just use “git subtree pull”. You’ll need the repository url (that is merely why it is documented in README.md inside the Vagrant box project).
$ git subtree pull --prefix modules/grafana https://github.com/bfraser/puppet-grafana.git master --squash From https://github.com/bfraser/puppet-grafana * branch master -> FETCH_HEAD Subtree is already at commit 89fe873720a0a4d2d3c4363538b0fa5d71542f41.
Purge a subtree
Purging a git subtree is also fairly easy – just remove the directory and commit the change. There are no additional config settings to purge unlike known from git submodules.
If you want to get more in-depth insights into Git make sure to check out the new Git training 🙂