Git Module / Submodule Automated Setups!

Git

Let's Git on with it!

Breaking larger code bases into a set of reusable modules is the hallmark of reusability, especially when applications have shared dependencies... Clearly you don't want to do the same thing over and over and over right? Isn't that basically the definition of insanity?
Well, fear not eager eyed reader, this is for you!

In this article I'm going to show how you can use tools built into Git to handle dependencies without any custom dependency management systems like composer, npm, yarn, bla bla bla! (Full-disclosure: you'll still be using those, but at least you don't have to bake stuff in to your codebase)

The Anatomy of a Submodule File!

File Name: .gitmodules
Function: To house all values required to determine submodules within a Git Repository.

[submodule "vendor/crashoz/uuid_v4"]
	path = vendor/crashoz/uuid_v4
	url = https://github.com/crashoz/uuid_v4.git
[submodule "vendor/google/googletest"]
	path = vendor/google/googletest
	url = https://github.com/google/googletest.git
[submodule "vendor/google/benchmark"]
	path = vendor/google/benchmark
	url = https://github.com/google/benchmark.git
[submodule "vendor/sewenew/redis-plus-plus"]
	path = vendor/sewenew/redis-plus-plus
	url = https://github.com/sarahjabado/redis-plus-plus.git
[submodule "vendor/unittest-cpp"]
	path = vendor/unittest-cpp
	url = https://github.com/unittest-cpp/unittest-cpp.git

'.gitmodules' file from https://github.com/BlankFoxGirl/cardinal-cpp

Adding New Modules

To add a new submodule, run the command;
$ git submodule add <RepositoryURL> <VendorPath>/<SubmodulePath>

Installing / Updating Submodules

To install your submodules for the first time, you will need to ensure that the destination path within your codebase exists.

In the case of the sample module file, we would need to create the vendor directory.

$ mkdir vendor

Hot Tip: Make sure you set your .gitignore file to ignore your vendor / third-party dependency directory, or you'll commit third party code to your Repo. Very messy stuff.

Installing using 'Git Submodule' Commands.

Note: This might not always be available on your system.

Init our submodules;

$ git submodule init

Update our submodules;

$ git submodule update

Installing using Bash

Note: We recommend using the bash file below for this purpose.

#!/bin/bash

install_submodules_manually() {
    CONTENTS=$(cat .gitmodules | grep -E '(path|url).*' | cut -d " " -f 3)
    CWD=$(pwd)

    LAST=""
    for item in ${CONTENTS[@]};
    do
    cd "$CWD"
    STRINGHEAD=$( echo "$item" | cut -d '/' -f 1)
        if [[ "$STRINGHEAD" == "vendor" ]]
        then
            if [[ -d "$CWD/$item" ]]
            then
                LAST=$(echo "$item")
                echo "$item already exists."
                continue
            fi
            mkdir -p "$item"
            cd "$item"
            LAST=$(echo "$item")
        else
            if [[ -d "$CWD/$LAST/.git" ]]
            then
                echo "$LAST already installed."
                continue
            fi
            cd "$CWD/$LAST"
            echo "Installing $item"
            git init .
            git remote add origin $item
            res=$(git pull origin master 2>&1)
            if [[ "$res" == "fatal: couldn't find remote ref master" ]]
            then
                git pull origin main
            fi
        fi
    done
}

if [[ -f ".gitmodules" ]]; then
	install_submodules_manually
fi

install_submodules.sh

Using the file above, we run;

Set the execute permission to the bash file
$ chmod +x install_submodules.sh
Execute the bash file.
$ ./install_submodules.sh

One-Stop Submodule Installation Shop

Let's put it all together for convenience purposes okay?

#!/bin/bash

VALIDATION_MODULE="crashoz"
VENDOR_DIRECTORY="vendor"

install_submodules_manually() {
    CONTENTS=$(cat .gitmodules | grep -E '(path|url).*' | cut -d " " -f 3)
    CWD=$(pwd)

    LAST=""
    for item in ${CONTENTS[@]};
    do
    cd "$CWD"
    STRINGHEAD=$( echo "$item" | cut -d '/' -f 1)
        if [[ "$STRINGHEAD" == "$VENDOR_DIRECTORY" ]]
        then
            if [[ -d "$CWD/$item" ]]
            then
                LAST=$(echo "$item")
                echo "$item already exists."
                continue
            fi
            mkdir -p "$item"
            cd "$item"
            LAST=$(echo "$item")
        else
            if [[ -d "$CWD/$LAST/.git" ]]
            then
                echo "$LAST already installed."
                continue
            fi
            cd "$CWD/$LAST"
            echo "Installing $item"
            git init .
            git remote add origin $item
            res=$(git pull origin master 2>&1)
            if [[ "$res" == "fatal: couldn't find remote ref master" ]]
            then
            	# Fall back to main branch.
                git pull origin main
            fi
        fi
    done
}

install_modules () {
    CWD=$(pwd)
    if [[ ! -d "$CWD/$VENDOR_DIRECTORY" ]]
    then
        mkdir "$CWD/$VENDOR_DIRECTORY"
        git submodule init
        git submodule update
    fi
    if [[ ! -d "$CWD/$VENDOR_DIRECTORY/$VALIDATION_MODULE" ]]
    then
        echo "Unable to auto-install using GIT, trying another route."
        install_submodules_manually
    else
        echo "Submodules installed."
    fi
}

if [[ -f ".gitmodules" ]]; then
	install_modules
fi

install.sh

Set the execute permission on the install.sh file.
$ chmod +x install.sh
Execute the install.sh file.
$ ./install.sh

Conclusion

Submodules are a fantastic way to split up a monolithic codebase into smaller codebases for better modularity, dependency resolution, and code de-duplication.
It's especially helpful when it's infeasible to use a package management system like NPM, Yarn, PIP, dotnet nuget, or composer.

We recommend you familiarise yourself with the Git SCM Documentation on Submodules, even though reading documentation can be very boring. https://git-scm.com/book/en/v2/Git-Tools-Submodules

💡

Version Control with Git

Author: Prem Kumar Ponuthorai & Jon Loeliger

View on Amazon