This entry is part 3 of 4 in the series Best Practices

Introduction

Git is a widely used distributed version control system that provides powerful tools to manage changes in a codebase, collaborate with other developers, and track the history of a project. As a platform engineer, you will likely work with Git on a daily basis, so it’s important to establish best practices to ensure a smooth and efficient workflow. In this article, we will cover several best practices for using Git as a platform engineer.

Setting up a repository

The first step in using Git is to set up a repository for your project. This can be done either locally or on a remote server, such as GitHub or GitLab. When setting up a repository, it’s important to choose an appropriate name and structure for your project.

To create a new repository on GitHub, for example, you can follow these steps:

  1. Sign in to your GitHub account and click the “+” icon in the top-right corner of the screen.
  2. Select “New repository” from the dropdown menu.
  3. Choose a name and description for your repository.
  4. Select the “Public” or “Private” option, depending on your needs.
  5. Click the “Create repository” button.

Adding protection to the main branch

It’s important to protect the main branch (usually named “master” or “main”) to prevent accidental or unauthorized changes. This can be done by setting up branch protection rules in your repository settings. You can require pull requests to be reviewed before merging, enforce certain checks (such as passing tests or code quality checks) before merging, and limit who can push to the main branch.

To set up branch protection rules on GitHub, for example, you can follow these steps:

  1. Go to your repository’s settings page.
  2. Click on the “Branches” tab.
  3. Select the main branch (e.g. “master” or “main”).
  4. Click the “Edit” button next to “Branch protection rules”.
  5. Choose the protections you want to enable, such as requiring pull request reviews, enforcing status checks, or restricting who can push to the branch.

Branch naming

When creating a new branch, it’s important to choose an appropriate name that reflects the work being done. A common convention is to use a prefix to indicate the type of branch (such as “feature/”, “bugfix/”, or “hotfix/”), followed by a brief description of the work being done. For example, “feature/add-login-page” or “bugfix/fix-typo-in-readme”.

Tagging

Tagging is a way to mark a specific commit as important or significant, such as a release or a milestone in the project. Tags can be annotated (with a message and metadata) or lightweight (just a name). Annotated tags are recommended for releases, as they provide more information about the tag and are easier to search and filter.

To create an annotated tag in Git, you can use the following command:

git tag -a <tagname> -m "<message>" <commit>

Commit messages best practices

Commit messages are an important way to communicate the changes being made to a codebase. Good commit messages should be clear, concise, and provide enough information to understand the change being made. A common convention is to use a short summary in the first line (limited to 50-72 characters), followed by a longer description if needed, and optionally a footer with references to issues or other related changes.

Example:

Add login page

This commit adds a new login page to the website, with a form for entering
username and password. The form is validated on the client side with
JavaScript.

Fixes #123

Using issues to refine the problem before the work is completed

Issues are a way to track and organize work

in your Git repository. They can be used to report bugs, suggest new features, or discuss improvements to the project. Using issues can help refine the problem before the work is completed, as it allows for discussions and feedback from other team members or stakeholders.

To create a new issue in GitHub, for example, you can follow these steps:

  1. Go to your repository on GitHub.
  2. Click on the “Issues” tab.
  3. Click the “New issue” button.
  4. Enter a title and description for the issue.
  5. Optionally assign the issue to a team member, label it, or add it to a project board.
  6. Click the “Submit new issue” button.

Using pull requests to merge in changes

Pull requests are a way to propose changes to a codebase and request them to be merged into the main branch. Pull requests can be reviewed and discussed by other team members before they are merged, which helps ensure the quality of the codebase.

To create a new pull request in GitHub, for example, you can follow these steps:

  1. Go to your repository on GitHub.
  2. Create a new branch for your changes.
  3. Make the desired changes to the codebase.
  4. Commit the changes and push them to the new branch.
  5. Go to the “Pull requests” tab and click the “New pull request” button.
  6. Select the base branch (usually the main branch) and the new branch with your changes.
  7. Enter a title and description for the pull request.
  8. Optionally assign the pull request to a team member, label it, or add it to a project board.
  9. Click the “Create pull request” button.

Using rebase to bring older repositories up to date before raising a PR

Before creating a pull request, it’s important to make sure your branch is up to date with the latest changes in the main branch. This can be done by using the “rebase” command to integrate the changes from the main branch into your branch.

To rebase your branch in Git, you can use the following command:

git checkout <your-branch>
git rebase <main-branch>

This will replay your changes on top of the latest changes in the main branch, creating a linear history of the changes.

Viewing previous commits and editing git history

Git allows you to view the history of a project, including all the previous commits and changes. This can be done using the “git log” command,

which shows a list of all the commits in reverse chronological order.

To view the git log in Git, you can use the following command:

git log

If you need to edit the history of a project, such as removing a commit or changing a commit message, you can use the “git rebase -i” command to interactively edit the history.

To edit the git history interactively in Git, you can use the following command:

git rebase -i <commit>

This will open a text editor with a list of all the commits in your history, allowing you to edit the commit messages or remove commits.

Amending the last commit

If you need to make a quick change to the last commit, such as fixing a typo or adding a file, you can use the “git commit –amend” command to add the changes to the last commit.

To amend the last commit in Git, you can use the following command:

git add <file>
git commit --amend

This will open a text editor with the commit message for the last commit, allowing you to edit it if needed.

Reverting a branch to a previous commit

If you need to revert a branch to a previous commit, such

as to undo a mistake or rollback changes, you can use the “git revert” command. This will create a new commit that undoes the changes introduced by a previous commit.

To revert a commit in Git, you can use the following command:

git revert <commit>

This will create a new commit that undoes the changes introduced by the specified commit.

Keeping a changelog in a repository

A changelog is a file in a repository that documents all the changes made to the project over time. It can help track the evolution of the project, and make it easier for team members and stakeholders to understand what changes have been made and when.

The Keep a Changelog website provides a recommended format for creating a changelog file in a repository, which includes sections for different types of changes, such as “Added”, “Changed”, “Deprecated”, “Removed”, “Fixed”, and “Security”.

Using semantic versioning

Semantic versioning is a system for assigning version numbers to software releases, based on a set of rules for how the version number should be incremented depending on the type of changes made to the codebase.

The Semantic Versioning website provides a specification for semantic versioning, which includes three numbers separated by dots: major, minor, and patch. The major version number is incremented for incompatible changes, the minor version number is incremented for backwards-compatible additions, and the patch version number is incremented for backwards-compatible bug fixes.

Using pre-commit to introduce git hooks

Pre-commit is a tool for introducing git hooks into a repository, which can help validate the codebase before allowing a commit to be made. Git hooks are scripts that are run automatically by Git at certain points in the workflow, such as before a commit is made.

Pre-commit provides a collection of pre-made hooks that can be used to check for common issues, such as trailing whitespace, missing or malformed docstrings, and syntax errors.

To use pre-commit in a Git repository, you can follow these steps:

  1. Install pre-commit using pip:
pip install pre-commit
  1. Create a “.pre-commit-config.yaml” file in the root of your repository, with the desired hooks:
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.0.1
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
  1. Run “pre-commit install” to install the hooks:
pre-commit install
  1. Make a change to the codebase and try to commit it. Pre-commit will run the configured hooks before allowing the commit to be made.

Conclusion

Git is a powerful tool for version control and collaboration, and understanding best practices for using Git can help improve the quality and reliability of a codebase. By setting up a repository, adding protection to the main branch, using branch naming, tagging, and commit messages best practices, using issues and pull requests, and using tools like pre-commit, you can create a more efficient and effective workflow for your team.

This entry is part 1 of 4 in the series Best Practices

Synopsis

This technical guide provides a detailed overview of best practices for working with Packer in a busy DevOps team. It includes information on concepts such as idempotency and naming standards, as well as code examples and templates for organizing Packer code in a git repository. The guide also covers considerations for security and provides templates for a README file, HCL file, and .gitignore file for a Packer repository.

Summary

This technical guide provides best practices for working with Packer in a busy DevOps team. It covers important concepts such as idempotency and naming standards, as well as providing code examples and structured into appropriate sections. The guide also includes information on how to organize Packer code in a git repository, including considerations for security, as well as templates for a README file, HCL file, and .gitignore file for a Packer repository.

Introduction

Packer is a popular tool for automating the creation of machine images. In a busy DevOps team, it is important to follow best practices when working with Packer to ensure that the codebase is maintainable and easy to work with.

One key concept to keep in mind when working with Packer is idempotency. An idempotent operation is one that has the same result whether it is performed once or multiple times. In other words, if an operation is idempotent, it will not change the system state if it is run multiple times with the same parameters. This is important in Packer because it allows you to run builds multiple times without causing unintended changes to the system.

To ensure idempotency in Packer, it is important to use the only and except parameters in the provisioner block. The only and except parameters allow you to specify the conditions under which a provisioner should run, such as the operating system or the type of machine image being built. Using these parameters ensures that Packer will only run a provisioner if the specified conditions are met.

Naming standards are another important aspect of working with Packer in a busy DevOps team. It is a good idea to use consistent naming conventions for Packer templates and variables to make the codebase easier to read and understand.

Code Examples

Here is an example of a Packer template that follows a consistent naming convention:

{
  "variables": {
    "aws_access_key": "{{env `AWS_ACCESS_KEY_ID`}}",
    "aws_secret_key": "{{env `AWS_SECRET_ACCESS_KEY`}}",
    "aws_region": "us-east-1"
  },
  "builders": [
    {
      "type": "amazon-ebs",
      "access_key": "{{aws_access_key}}",
      "secret_key": "{{aws_secret_key}}",
      "region": "{{aws_region}}",
      "source_ami": "ami-0f2176987ee50226e",
      "instance_type": "t2.micro",
      "ssh_username": "ec2-user",
      "ami_name": "packer-example {{timestamp}}"
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "sudo yum update -y",
        "sudo yum install -y nginx"
      ]
    }
  ]
}

In this example, the variables are named aws_access_key, aws_secret_key, and aws_region, and the Packer template is named packer-template.json.

Organizing Packer Code in a Git Repo

When working with Packer in a busy DevOps team, it is important to organize the codebase in a way that is maintainable and easy to work with. One way to do this is to split the Packer code into different files and directories within a git repository.

One way to organize the Packer code is to separate the provisioners, builders, and variables into different files. This can make it easier to find and modify specific parts of the codebase. For example, you could create a provisioners directory to store all of the provisioner scripts, a builders directory to store the Packer templates, and a variables directory to store the variable definitions.

It is also important to consider security when organizing the Packer code in a git repository. Sensitive information such as access keys and secrets should not be stored in the repository in plaintext. Instead, you can use tools such as Hashicorp’s Vault to securely store and manage sensitive information.

Template README.md for a Packer Repo:

# Packer Repository

This repository contains Packer templates and scripts for building machine images.

## Directory Structure

The repository is organized as follows:

- `builders`: Packer templates for building machine images
- `provisioners`: Scripts for provisioning machine images
- `variables`: Variable definitions for Packer templates

## Usage

To build a machine image using a Packer template, run the following command:

```bash
packer build -var-file=variables/example.json builders/example.json
```

Replace example.json with the appropriate file names for your build.

## Contributing

To contribute to this repository, follow these steps:

+ Fork the repository
+ Create a new branch for your changes
+ Make your changes and commit them to the new branch
+ Push the branch to your fork
+ Create a pull request from your fork to the main repository

Please make sure to follow the repository's style guidelines and to run any relevant tests before submitting a pull request.

## License

This repository is licensed under the MIT License.

## Template HCL file with Headers Summarized:

## Packer Template

This Packer template is used to build a machine image.

### Builders

The following builders are used in this template:

 + Amazon Elastic Block Store (EBS)

### Provisioners

The following provisioners are used in this template:

 + Shell

### Variables

The following variables are used in this template:

+ `aws_access_key: AWS access key`
+ `aws_secret_key: AWS secret key`
+ `aws_region: AWS region`

### Usage

To build a machine image using this Packer template, run the following command:

```bash
packer build -var-file=variables/example.json template.json
```

Replace example.json with the appropriate file name for your variables.

### Contributing

To contribute to this Packer template, follow these steps:

+ Fork the repository
+ Create a new branch for your changes
+ Make your changes and commit them to the new branch
+ Push the branch to your fork
+ Create a pull request from your

Conclusion

By following best practices such as ensuring idempotency and using consistent naming conventions, and organizing Packer code in a git repository in a structured and secure way, DevOps teams can effectively work with Packer to automate the creation of machine images. By following these guidelines, teams can ensure that their codebase is maintainable and easy to work with, enabling them to deliver new features and updates more efficiently.

This entry is part 5 of 5 in the series Learning Ansible

Synopsis

The following guide covers the steps for setting up and configuring an Ansible project using git for source control. It covers the creation of a new git repository, installing and configuring a Python virtual environment, adding requirements to the repository, adding and committing changes to the repository, and configuring pre-commit for automated testing. It also covers basic git workflow principles and best practices, including the use of feature branches, pull requests, and automated testing.

Introduction

Ansible is a powerful tool for automating and managing IT infrastructure. When working with ansible, it is important to follow best practices, including using source control to manage your projects. Git is a popular choice for source control, and in this guide, we will cover how to set up a new ansible project using git.

In this exercise, we will create a new git repository for an ansible project, create a Python virtual environment (venv) to manage dependencies, and configure pre-commit to enforce best practices. We will also cover how to create a requirements.txt file for the repository, and how to exclude the virtual environment from the repository.

Exercise

Create a new git repository, this will be used for an Ansible project

cd $HOME
mkdir projects
cd projects
mkdir ansible-project
cd ansible-project
git init
echo "# ansible Project" > Readme.md
git add Readme.md
git commit -m "Initial commit"
git branch -M main
# Create a new repo in github
git remote add origin git@github.com:{username}/{repo}.git
# Substitute the real values for {username} and {repo} in the command above
git push -u origin main

Create a Python3 virtual environment (venv)

# Create a new virtual environment
virtualenv venv

# Activate the virtual environment
source venv/bin/activate

# Install Ansible
pip install ansible==2.9.7

# Install Pre Commit
pip install pre-commit

# Install Jinja2
pip install jinja2

The Python virtual environment allows us to create an isolated environment for our Python project. This enables us to have a consistent set of packages and versions that are required for the project, regardless of what other packages may be installed on the local machine.

Create a requirements.txt file for the repository

# Run pip freeze to list all the packages that are installed at the moment and their versions
pip freeze

# We want to take note of the main packages and their versions
pip freeze | egrep -i "ansible|pre-commit|jinja2" > requirements.txt

These commands assume that the terminal session that is in use is in an active Python virtual environment. The requirements.txt file will contain a list of the packages and their specific versions that are required for the project. This file is useful for keeping track of the packages that are required for the project, and for reproducing the same environment on other machines.

Add the requirements.txt file to the repository, commit, and push to GitHub

git add requirements.txt
git commit -m "Added python requirements.txt file to repository"
git push origin main

Deactivate the virtualenv

deactivate

The terminal session has now been returned to its previous state, and running a Python command at this point will not use the virtual environment to access modules.

Omit the virtual environment from the git repository

touch .gitignore 
echo "venv" >> .gitignore

Add the .gitignore file to the repository, commit and push to GitHub

git add .gitignore
git commit -m "Added git ignore file"
git push origin main

Note, using two right angle brackets >> will append the entry “venv” to the file .gitignore, if you run the command twice, you will get two entries in your ,gitignore file. If you would like to create and overwrite the file you can use a single right angle bracket e.g. >

Prepare the .pre-commit-config.yaml file

echo -e "repos:\n - repo: https://github.com/ansible/ansible-lint\n rev: stable\n hooks:\n - id: ansible-lint" >> .pre-commit-config.yaml 
echo -e "repos:\n - repo: https://github.com/pre-commit/mirrors-yamllint\n rev: v1.23.0\n hooks:\n - id: yamllint" >> .pre-commit-config.yaml 
echo -e "repos:\n - repo: https://github.com/pre-commit/mirrors-flake8\n rev: v3.8.4\n hooks:\n - id: flake8" >> .pre-commit-config.yaml

The .pre-commit-config.yaml file is used to configure pre-commit

Add the .pre-commit-config.yaml file to the repository, commit and push to GitHub

git add .pre-commit-config.yaml
git commit -m "Added .pre-commit-config.yaml file"
git push origin main

Configure pre-commit to run in the git repository

pre-commit install

Add the pre-commit hooks to the repository, commit and push to GitHub

git add .git/hooks/pre-commit
git commit -m "Added pre-commit hooks to repository"
git push origin main

Verify pre-commit has been installed correctly

Run the following command to confirm that pre-commit is installed correctly:

pre-commit run --all-files

If the installation was successful, you should see output similar to the following:

Checking for added files... 
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for https://github.com/pre-commit/mirrors-ansible-lint.
[INFO] Initializing environment for https://github.com/pre-commit/mirrors-flake8. 
[INFO] Initializing environment for https://github.com/pre-commit/mirrors-yamllint.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Installing environment for https://github.com/pre-commit/mirrors-ansible-lint.
[INFO] Installing environment for https://github.com/pre-commit/mirrors-flake8.
[INFO] Installing environment for https://github.com/pre-commit/mirrors-yamllint. ...

If you receive an error, it may be because pre-commit is not installed correctly. In this case, try uninstalling and reinstalling pre-commit:

pip uninstall
pre-commit
pip install pre-commit

Once pre-commit is installed correctly, you can begin using it to enforce your chosen coding standards and practices.

Conclusion

By following the steps in this guide, you should now have a fully configured ansible project that is ready for development. You should have installed ansible and other required python modules into a virtual environment created a requirements.txt file and set up pre-commit to enforce your chosen coding standards. You should now be able to start developing your ansible project with confidence, knowing that your code will be automatically checked and validated before it is committed to your git repository.