Git strategies for Network Automation
Let’ s be honest: git is so damn complex. Whoever does not admit that is either a liar or someone who does not have a great social life.
Whenever I have to work with git, I feel like I have been dropped in the middle of a minefield and I have to find my way out thanks to a map written in cuneiform (...try to read some git documentation and then you’ ll let me know!). So many times I stepped on a mine and blew-up the repository in which I was working , or I covered my eyes and started walking randomly, hoping to get the result I wanted. This strategy was not Ok, so It was obvious that I had to change approach. Stop, step back, start to study cuneiform and understand what I was doing.
Every time I face something that is bit complicated for my brain, I try to simplify the topic and build a solid understanding around that. Later, I deep dive in more advanced features (...still waiting for that moment).
After a good reading of “Version Control with Git — O’Reilly” I could identify few git operations that fit very well for Network Automation: git merge,git merge --squash, git cherry-pick, git rebase, git reverte. So why we do not go through them and learn by examples? Hopefully you will not struggle with git as much as I had to.
Assumptions: I expect you to have a general understanding of git, especially branches, git add and git commit as I will not cover them on this post.
git merge
Git merge let you (guess what?) merge one branch and all its files/changes into another branch. If you work in a git branch and your team mate works in another branch, you can merge each other work into each other branch. Simple like that. Let’s go through a practical example:
Let’s assume that I am working to a jinja2 template for switch interface configuration. I am in charge of building the template while my team mates have just to write a bunch of values in a YAML file and save the interface configuration generated by the template rendering.
I initialize a new git repository with a master branch where I will create my amazing interface.j2 template and the python render.
olivierif$ git init
Initialized empty Git repository in /Users/git/demo/.git/By default, git drop us on default master branch.
olivierif$ git status
On branch masterNo commits yetnothing to commit (create/copy files and use "git add" to track)
I am going to add first the jinja template for interface configuration interface.j2, then the python render render.py (you can actually add both files at the same time but I want to have few commits in my git for this example).
olivierif$ git status
On branch masterNo commits yetUntracked files:
(use "git add <file>..." to include in what will be committed)
interface.j2nothing added to commit but untracked files present (use "git add" to track)olivierif$ git add interface.j2olivierif: git commit -m "Add jinja template for ifaces"
[master (root-commit) 1addf0e] Add jinja template for ifaces
1 file changed, 32 insertions(+)
create mode 100644 interface.j2
For brevity, I am not going to show you the same steps for render.py, but the repo at the end will look like this:
olivierif$ git ls-files
interface.j2
render.pyolivierif$ git log --pretty=oneline --abbrev-commit
e93c376 (HEAD -> master) Add python render for jinja template
8a538b4 Add jinja template for ifaces
If we have to graphically represent the history of our master branch, it would look like this:
A --- B(HEAD) masterWhere A is our first commit — 8a538b4 and HEAD is our last commit — e93c376.
Our team mate can now start to work on building the YAML file and render the template (ideally this would be done on a remote repository, cloning the repo). Considering master the production branch or the one NOT to be used for development, our dear friend is going to create a different branch:
someone$ git checkout -b "dev_iface"someone$ git branch
* dev_iface
mastersomeone$ git log --pretty=oneline --abbrev-commit
e93c376 (HEAD -> dev_iface, master) Add python render for jinja template
8a538b4 Add jinja template for ifaces
As we can see from the output above, we have now moved (or in git terminology — checkout ) to a different branch taking as diverging point the HEAD of master (HEAD -> dev_iface, master). That means that all the files and changes present in master at commit e93c376 (aka HEAD ) are now present in our dev_iface branch too. Let’ s also assume that our master branch carried on with some work and committed a new change. Our repo will look like the following:
D(HEAD) dev_iface
/
A --- B --- C(HEAD) masterIt is not mandatory to create a branch from HEAD, it is possible to checkout a new branch from whatever commit we want. In this case, the HEAD of our new branch would be the commit that we used to checkout.
someone$ git checkout -b "i_do_what_i_want" 8a538b4
Switched to a new branch 'i_do_what_i_want'someone$ git log --pretty=oneline --abbrev-commit
8a538b4 (HEAD -> i_do_what_i_want) Add jinja template for ifacessomeone$ git ls-files
interface.j2
After adding iface_r1.yaml and R1_Eth1.conf our development branch will look like this:
someone$ git log --pretty=oneline --abbrev-commit
15f2799 (HEAD -> dev_iface) Add render for Eth1 R1
170caca Add iface variables for R1
e93c376 Add python render for jinja template
8a538b4 Add jinja template for ifacessomeone$ git ls-files
R1_Eth1.conf
iface_r1.yaml
interface.j2
render.py
D --- E --- F(HEAD) dev_iface
/
A --- B ----- C(HEAD) master
Great! We know want to make R1_Eth1.conf available in master so everyone else can take that config file and do whatever they like. Then we need to merge dev_iface into master, however before doing so we have first to switch to master branch and then merge.
olivierif$ git checkout masterSwitched to branch 'master'olivierif$ git merge dev_iface -m "Add R1 Eth1 config"
Merge made by the 'recursive' strategy.
R1_Eth1.conf | 0
iface_r1.yaml | 0
2 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 R1_Eth1.conf
create mode 100644 iface_r1.yamlolivierif$ git log --pretty=oneline --abbrev-commit
2a9ce69 (HEAD -> master) Add R1 Eth1 config
15f2799 (dev_iface) Add render for Eth1 R1
170caca Add iface variables for R1
1794e6b Fix render.py line
e93c376 Add python render for jinja template
8a538b4 Add jinja template for ifaces
As you can see, all the files/changes that were in dev_iface are now also into master.
D --- E --- F dev_iface
/ \
A --- B ----- C ----- G(HEAD) masterLet’s have a closer look at git log output: we can now see in our master all the commits belonging to dev_iface(where 15f2799 (dev_iface) represents the HEAD of dev_iface — F in the above diagram) and 2a9ce69 is the new commit generated by the merge. 2a9ce69 is also became our new HEAD in master (G in the above diagram).
git merge squash
It has often happened to me that for whatever reason I had a very looooooong list of commits on my development branch. Most of them were just test commits, that I did not want to be shown on master when was time to merge. One way to do that is by using --squash argument. —-squash is going to zip all commits into only one (there are actually better way to remove unwanted commits if that is your main purpose). Let’s go through an example where we create a dev_snmp branch in which we add some files and edit some existing ones:
else$ git checkout -b "dev_snmp"
Switched to a new branch 'dev_snmp'[...]else$ git log --pretty=oneline --abbrev-commit -n 6
9e359ac (HEAD -> dev_snmp) fix line in smp yaml
f017fec edit Eth1 R1 render manually
37d77eb add R1 snmp render
c48dbff add snmp variables for R1
5501e1c add conditional for R2 in render.py
324063b Implement log error in render.py
We can now see a series of commits in dev_snmp. If we now merge into master, we will see all those commits in master too. What we want in this case, is to show one commit only in master after the merge.
olivierif$ git checkout master
Switched to 'master'olivierif$ git merge --squash dev_snmp
Updating 324063b..f017fec
Fast-forward
Squash commit -- not updating HEAD
R1_Eth1.conf | 2 ++
R1_snmp.conf | 0
render.py | 1 +
snmp_r1.yml | 0
4 files changed, 3 insertions(+)
create mode 100644 R1_snmp.conf
create mode 100644 snmp_r1.ymlolivierif$ git status
On branch master
Changes to be committed:
(use "git restore --staged <file>..." to unstage)
modified: R1_Eth1.conf
new file: R1_snmp.conf
modified: render.py
new file: snmp_r1.ymlolivierif$ git commit -m "merge dev_snmp squash into master"
[master 930e095] merge dev_snmp squash into master
4 files changed, 3 insertions(+)
create mode 100644 R1_snmp.conf
create mode 100644 snmp_r1.ymlolivierif$ git log --pretty=oneline --abbrev-commit -n 5
6726161 (HEAD -> master) merge dev_snmp squash into master
324063b Implement log error in render.py
2a9ce69 Add R1 Eth1 config
15f2799 Add render for Eth1 R1
170caca Add iface variables for R1
As you can see, all the commits in dev_snmp have been squashed into one commit only that is, the merge commit. From a point of view we loose all the history of dev_snmp.
Let’s have a look to the repo graph, but this time using the power of git log
olivierif$ git log --decorate --oneline --graph
* 6726161 (HEAD -> master) merge dev_snmp squash into master
* 324063b Implement log error in render.py
* 2a9ce69 Add R1 Eth1 config
|\
| * 15f2799 Add render for Eth1 R1
| * 170caca Add iface variables for R1
* | 1794e6b Fix render.py line
|/
* e93c376 Add python render for jinja template
* 8a538b4 Add jinja template for ifacesIf we add —-all argument we can also see the squashed dev_snmp
olivierif$ git log --all --decorate --oneline --graph
* 6726161 (HEAD -> master) merge dev_snmp squash into master
| * 9e359ac (dev_snmp) fix line in smp yaml
| * f017fec edit Eth1 R1 render manually
| * 37d77eb add R1 snmp render
| * c48dbff add snmp variables for R1
| * 5501e1c add conditional for R2 in render.py
|/
* 324063b Implement log error in render.py
* 2a9ce69 Add R1 Eth1 config
[...]git rebase
git rebase is a similar strategy to git merge: basicaly you use both when you want a seires of commits from a branch, into another branch. Even though the final resut is similar, there are few differences between merge and rebase (I might tackle them in another post…). As rule of thumb, I usually merge when I completed a work in a branch and I want to deploy it - dev → master. I use rebase when for example I want to keep my development branch up to date with masteror when master has a long series of commits for bug fixes and doing cherry-pick is not really scalable -master → dev.
Example: our master branch moved forward with new commits and merges. dev_snmp is quite behind to master. Because I want to keep my development branch updated (peraphs because I want the latest bugs fixes version in master or maybe because I want to take someone else template as blueprint) I rebase dev_snmp with master
# MASTER
olivierif$ git log --pretty=oneline --abbrev-commit
cd830d8 (HEAD -> master) add snmp template
2c618ee add ntp template
83aa0ab fix typo in interface template
6726161 merge dev_snmp squash into master
324063b Implement log error in render.py
[...]# DEV_SNMP
else$ git checkout dev_snmp
Switched to branch 'dev_snmp'else$ git log --pretty=oneline --abbrev-commit
9e359ac (HEAD -> dev_snmp) fix line in smp yaml
f017fec edit Eth1 R1 render manually
37d77eb add R1 snmp render
c48dbff add snmp variables for R1
[...]else$ git rebase master
Successfully rebased and updated refs/heads/dev_snmp.else$ git log --pretty=oneline --abbrev-commit
cd830d8 (HEAD -> dev_snmp, master) add snmp template
2c618ee add ntp template
83aa0ab fix typo in interface template
6726161 merge dev_snmp squash into master
324063b Implement log error in render.py
[...]
As you can see, the new HEAD of dev_snmp is now the same as the HEAD of master cd830d8 (HEAD -> dev_snmp, master). No new commit is required with rebase, compared to git merge startegy.
Here an example of repo before and after rebase:
# BEFORE REBASE A --- B --- C(HEAD) devel
/
D --- E ----- F --- G(HEAD) master# AFTER REBASE A -- B -- C -- F' -- G'(HEAD) devel
/
D --- E ---- F -- G(HEAD) master
git cherry-pick
This is my favourit! Listen up: git cherry-pick let you pick up a commit from one branch and add into your branch. This is particularly useful for example when someone release a bug fix in master and you need to pick that bug fix and put in your branch.
Let’s assume that I found a bug in interface.j2 while my colleague is working on his dev_iface branch for Eth2_r2.yaml. I fixed what I had to in master and my commit is ready to be picked up by dev_iface. This is how master repo looks like:
olivierif$ git show-branch
! [dev_iface] fix iface variable
! [dev_snmp] fix line in smp yaml
* [master] add snmp template
---
* [master] add snmp template
* [master^] add ntp template
* [master~2] fix typo in interface template
* [master~3] merge dev_snmp squash into master
+ [dev_snmp] fix line in smp yamlIn git show-branch I can see all the branches and their commit in the repository. I can also see that master has an asterisk * that is telling me that is my checkout branch. [master^] add ntp template is the commit before the HEAD (denoted by ^ ) and [master~2] fix typo.. is 2 commit before HEAD (denoted by ~2 ).
In order to have the bug fix, my team mate has to pick [master~2] into his branch.
someone$ git checkout dev_iface
Switched to branch 'dev_iface'someone$ git log --pretty=oneline --abbrev-commit
c049288 (HEAD -> dev_iface) fix iface variable
15f2799 Add render for Eth1 R1
170caca Add iface variables for R1
e93c376 Add python render for jinja template
8a538b4 Add jinja template for ifacessomeone$ git cherry-pick master~2
[dev_iface fa4f660] fix typo in interface template
Date: Sun May 10 09:45:15 2020 +0200
1 file changed, 1 insertion(+)someone$ git log --pretty=oneline --abbrev-commit
fa4f660 (HEAD -> dev_iface) fix typo in interface template
c049288 fix iface variable
15f2799 Add render for Eth1 R1
170caca Add iface variables for R1
e93c376 Add python render for jinja template
8a538b4 Add jinja template for ifaces
Et voila! The master~2 commit is now part of dev_iface being the HEAD of the branch with a different SHA commit compared to master . Note that you could cherry pick the master commit using also the SHA: git cherry-pick master 83aa0ab
At this point our repo can be represented like this:
... F ----- I'(HEAD) dev_iface
/
... H --- I --- L --- M(HEAD) mastergit revert
The last one of our list, a simple one. You almost made it! Do not give up now!
Bascially, git revert is the negative of git cherry-pick. With cherry-pick you can pick up a commit from a branch and introduce it into another branch, with revert you can undo a comit burried deeply into your beanch, without alter the existing commit history.
Let’s revert a commit in master. Let’s say that we want to revert commit 83aa0ab in master. Here how the repo looks like before the revert:
A -- B -- C -- D -- E -- F -- G(HEAD) master# MASTER
olivierif$ git log --decorate --oneline --graph
* cd830d8 (HEAD -> master, dev_snmp) add snmp template
* 2c618ee add ntp template
* 83aa0ab fix typo in interface template
* 6726161 merge dev_snmp squash into master
* 324063b Implement log error in render.py
* 2a9ce69 Add R1 Eth1 config
|\
| * 15f2799 Add render for Eth1 R1
| * 170caca Add iface variables for R1
* | 1794e6b Fix render.py line
|/
* e93c376 Add python render for jinja template
* 8a538b4 Add jinja template for ifaces
..and here the repo after the revert operation:
olivierif$ git revert 83aa0ab
[master 27572d1] Revert "fix typo in interface template"
1 file changed, 1 deletion(-)olivierif$ git log --decorate --oneline --graph
* 27572d1 (HEAD -> master) Revert "fix typo in interface template"
* cd830d8 (dev_snmp) add snmp template
* 2c618ee add ntp template
* 83aa0ab fix typo in interface template
* 6726161 merge dev_snmp squash into master
* 324063b Implement log error in render.py
* 2a9ce69 Add R1 Eth1 config
|\
| * 15f2799 Add render for Eth1 R1
| * 170caca Add iface variables for R1
* | 1794e6b Fix render.py line
|/
* e93c376 Add python render for jinja template
* 8a538b4 Add jinja template for ifaces
A -- B -- C -- D -- E -- F -- G -- E'(HEAD) master
We have now reverted our commit, introudcing though a new commit that became our HEAD in master
The End.