Table of contents
  1. Setting Up a Self-Hosted Agent in Docker Windows Container
    1. Common Error with start.sh
    2. Steps to Create Docker Container
      1. Create and Navigate to Folder
      2. Save Dockerfile
      3. Save start.sh
      4. Build Docker Image
      5. Start the Image
    3. What all types of Agents can we have?
      1. Microsoft-Hosted Agents
      2. Self-Hosted Agents
        1. On-Premises Self-Hosted Agents
        2. Cloud Self-Hosted Agents
      3. Why Use Different Containers
      4. When to Use On-Premises Windows Containers

Setting Up a Self-Hosted Agent in Docker Windows Container

In Azure DevOps, an agent is a software that runs on a machine and executes build and deployment tasks. These tasks are defined in your pipelines and can be executed on various environments, including Windows, Linux, and macOS. Agents are essential for continuous integration and continuous deployment (CI/CD) as they handle the actual operations required to build, test, and deploy your applications.

There are two main types of agents in Azure DevOps:

  1. Microsoft-Hosted Agents: These agents are managed by Microsoft and automatically provisioned for you. They are ephemeral, meaning a new agent is created for each run and discarded afterward.
  2. Self-Hosted Agents: These agents are managed by you. You have full control over the machine where the agent runs. This type of agent is useful when you need more control over the environment, want to install specific software, or need to connect to resources within your private network.

This guide focuses on setting up a self-hosted agent in a Ubuntu Container.

Common Error with start.sh

The main error you will get is that the start.sh file is not findable. This happens if you create a file for Linux from Windows. The only resolution that works confidently is to convert the file using the link:

https://toolslick.com/conversion/text/dos-to-unix

Just upload the start.sh file, convert, and download. Then use as is.

Note: The Notepad++ method of changing the format won’t work. Neither will PowerShell. These methods are not reliable.

Steps to Create Docker Container

Create and Navigate to Folder

Create a folder azp-agent-in-docker and navigate to it.

Save Dockerfile

Save the following content in azp-agent-in-docker/azp-agent-linux.dockerfile:

FROM ubuntu:22.04
ENV TARGETARCH="linux-x64"
# Also can be "linux-arm", "linux-arm64".

RUN apt update
RUN apt upgrade -y
RUN apt install -y curl git jq libicu70

WORKDIR /azp/

COPY ./start.sh ./
RUN chmod +x ./start.sh

# Create agent user and set up home directory
RUN useradd -m -d /home/agent agent
RUN chown -R agent:agent /azp /home/agent

USER agent
# Another option is to run the agent as root.
# ENV AGENT_ALLOW_RUNASROOT="true"

ENTRYPOINT [ "./start.sh" ]

Save start.sh

Save this content to azp-agent-in-docker/start.sh:

#!/bin/bash
set -e

if [ -z "${AZP_URL}" ]; then
  echo 1>&2 "error: missing AZP_URL environment variable"
  exit 1
fi

if [ -z "${AZP_TOKEN_FILE}" ]; then
  if [ -z "${AZP_TOKEN}" ]; then
    echo 1>&2 "error: missing AZP_TOKEN environment variable"
    exit 1
  fi

  AZP_TOKEN_FILE="/azp/.token"
  echo -n "${AZP_TOKEN}" > "${AZP_TOKEN_FILE}"
fi

unset AZP_TOKEN

if [ -n "${AZP_WORK}" ]; then
  mkdir -p "${AZP_WORK}"
fi

cleanup() {
  trap "" EXIT

  if [ -e ./config.sh ]; then
    print_header "Cleanup. Removing Azure Pipelines agent..."

    # If the agent has some running jobs, the configuration removal process will fail.
    # So, give it some time to finish the job.
    while true; do
      ./config.sh remove --unattended --auth "PAT" --token $(cat "${AZP_TOKEN_FILE}") && break

      echo "Retrying in 30 seconds..."
      sleep 30
    done
  fi
}

print_header() {
  lightcyan="\033[1;36m"
  nocolor="\033[0m"
  echo -e "\n${lightcyan}$1${nocolor}\n"
}

# Let the agent ignore the token env variables
export VSO_AGENT_IGNORE="AZP_TOKEN,AZP_TOKEN_FILE"

print_header "1. Determining matching Azure Pipelines agent..."

AZP_AGENT_PACKAGES=$(curl -LsS \
    -u user:$(cat "${AZP_TOKEN_FILE}") \
    -H "Accept:application/json;" \
    "${AZP_URL}/_apis/distributedtask/packages/agent?platform=${TARGETARCH}&top=1")

AZP_AGENT_PACKAGE_LATEST_URL=$(echo "${AZP_AGENT_PACKAGES}" | jq -r ".value[0].downloadUrl")

if [ -z "${AZP_AGENT_PACKAGE_LATEST_URL}" -o "${AZP_AGENT_PACKAGE_LATEST_URL}" == "null" ]; then
  echo 1>&2 "error: could not determine a matching Azure Pipelines agent"
  echo 1>&2 "check that account "${AZP_URL}" is correct and the token is valid for that account"
  exit 1
fi

print_header "2. Downloading and extracting Azure Pipelines agent..."

curl -LsS "${AZP_AGENT_PACKAGE_LATEST_URL}" | tar -xz & wait $!

source ./env.sh

trap "cleanup; exit 0" EXIT
trap "cleanup; exit 130" INT
trap "cleanup; exit 143" TERM

print_header "3. Configuring Azure Pipelines agent..."

./config.sh --unattended \
  --agent "${AZP_AGENT_NAME:-$(hostname)}" \
  --url "${AZP_URL}" \
  --auth "PAT" \
  --token $(cat "${AZP_TOKEN_FILE}") \
  --pool "${AZP_POOL:-Default}" \
  --work "${AZP_WORK:-_work}" \
  --replace \
  --acceptTeeEula & wait $!

print_header "4. Running Azure Pipelines agent..."

chmod +x ./run.sh

# To be aware of TERM and INT signals call ./run.sh
# Running it with the --once flag at the end will shut down the agent after the build is executed
./run.sh "$@" & wait $!

Build Docker Image

Run the following command within that directory:

docker build --tag "azp-agent:linux" --file "./azp-agent-linux.dockerfile" .

The final image is tagged azp-agent:linux.

Start the Image

Now that you have created an image, you can run a container. This installs the latest version of the agent, configures it, and runs the agent. It targets the specified agent pool (the Default agent pool by default) of a specified Azure DevOps or Azure DevOps Server instance of your choice:

docker run -e AZP_URL="<Azure DevOps instance>" -e AZP_TOKEN="<Personal Access Token>" -e AZP_POOL="<Agent Pool Name>" -e AZP_AGENT_NAME="Docker Agent - Linux" --name "azp-agent-linux" azp-agent:linux

You might need to specify --interactive and --tty flags (or simply -it) if you want to be able to stop the container and remove the agent with Ctrl + C.

docker run --interactive --tty < . . . >

If you want a fresh agent container for every pipeline job, pass the --once flag to the run command.

docker run < . . . > --once

What all types of Agents can we have?

Microsoft-Hosted Agents

Microsoft-hosted agents are managed by Azure DevOps. They are easy to use because they come pre-configured with common tools and are always up to date. You don’t need to maintain them. However, they might not be the best choice if you need custom software or specific configurations.

Self-Hosted Agents

On-Premises Self-Hosted Agents

These agents run on your own servers. They are great for accessing internal resources like databases and file shares. You have full control over the environment, which is good for running specialized software or meeting security requirements.

Cloud Self-Hosted Agents

These agents run on virtual machines in the cloud, like Azure VMs. They are perfect if you need to scale up or down based on demand. You get the benefits of the cloud without having to manage physical hardware.

Why Use Different Containers

  • Windows Containers: Best for running Windows applications like .NET Framework and ASP.NET. They ensure compatibility and make deployment easier.
  • Ubuntu Containers: Ideal for open-source software, Python apps, Node.js, and other tools. They are lightweight and flexible.

When to Use On-Premises Windows Containers

Use these when you need to run Windows apps that need access to your internal resources. They give you the benefits of containers while allowing access to your internal systems and data.