Running Scheduled Console Apps with .Net Core, Cron and Docker

Running Scheduled Console Apps with .Net Core, Cron and Docker

by | 4 min read
Published:
Updated:

I am a big fan of .Net Core at the moment. Mostly because I can do development natively on my Mac and the fact I can run them in Docker containers (did I mention I love Docker?).

With .Net Core you can build any app you can currently build in vanilla .Net, such as console apps, Web APIs, MVC apps. As you know, console apps generally come in 2 flavours, run all the time and run on a schedule. Typically the scheduled console apps are just hooked up to Windows Task Scheduler and forgotten about (until they break).

Given .Net Core is supposed to be cross-platform, what is the cross-platform equivalent for Windows Task Scheduler? I had a look at building in a timer loop or using a framework such as FluentScheduler but this just felt like adding bulk to an otherwise small console application.

🚀 Are you looking to level up your engineering career?

You might like my free weekly newsletter, The Curious Engineer, where I give career advice and tackle complex engineering topics.

📨 Don't miss out on this week's issue

Then a friend reminded me that Docker contains are running Linux. Of course, how simple we can just use Cron!

All Hail Cron

For the uninitiated, Cron is Linux’s version of Windows Task Scheduler. It doesn’t have a UI but is controlled by files placed in /etc/cron.d/.

As I found out, running cron successfully in Docker with a .Net Core console app comes with a few gotchas!

Gotcha #1: No environment variables

For my first attempt, I created a schedule file which simply ran my dotnet core console app using cron on a schedule.

My file looked like this:

* * * * * root dotnet /app/dotnet-cron.dll >> /var/log/cron.log 2>&1

#* * * * * *
#| | | | | |
#| | | | | +-- Year (range: 1900-3000)
#| | | | +---- Day of the Week (range: 1-7, 1 standing for Monday)
#| | | +------ Month of the Year (range: 1-12)
#| | +-------- Day of the Month (range: 1-31)
#| +---------- Hour (range: 0-23)
#+------------ Minute (range: 0-59)
# Cron requires a blank space at the end of the file so leave this here.

Which brings us on to Gotcha #1. Cron runs in its own environment with all the environment variables stripped out.

Typically any environment changes in a Docker container are controlled using environment variables. With a Microsoft extension, you can use AddEnvironmentVariables to override the settings in your appsettings.json file.

You are probably used to seeing environment variables specified in your docker compose file like this:

Database:ConnectionString=Server=192.168.1.45;uid=db_user;pwd=THG763hdN;

As we are dealing with compiled images the only way to cope with this is to export your environment variables at runtime and have them set by cron before your application runs.

My approach was to use printenv and write them all to a file prepending them with the export command. Which brings us on to Gotcha #2.

Gotcha #2: Invalid environment variables

So let us see what happens when we export one of our environment variables:

bash: export: `Database:ConnectionString=Server=192.168.1.45': not a valid identifier

By what I can only assume is black magic, Docker manages to set environment variables even though they have invalid characters. In this case, it is the colon (:) that is causing the issue. Luckily .Net Core accepts a double underscore instead which does work with the Linux export function.

While looking into this I did find something interesting. If you use double underscore the environment variables are available when the CMD line is run at the end of your Dockerfile. However, if you use colons these are actually set after your container has been set up.

Docker Compose also throws in another curve ball into the mix by setting an affinity:container environment variable which not only contains a colon also has a double equals!

In the end, I used the following script to output all my environment variables into a separate script. I had to use sed to wrap the values in quotes otherwise this caused yet another headache.

#!/bin/bash

echo '#!/bin/bash' > /app/set_env.sh
printenv | sed '/^affinity:container/ d' | sed -r 's/^([a-zA-Z_]+[a-zA-Z0-9_-]*)=(.*)$/export \1="\2"/g' >> /app/set_env.sh
chmod +x /app/set_env.sh

My Dockerfile then runs this script before running cron: I then just tell to run the following script which sets the environment variables and runs my console application.

#!/bin/bash

# Set environment variables copied from container
source /app/set_env.sh;

# Run your dotnet console app
dotnet /app/dotnet-cron.dll

Gotcha #3: Cross-platform considerations

The last gotcha only occurred when I got other people to build and run my Docker image on their machines. It worked fine the fellow Mac users but for those on Windows, the resulting image would never run the console on schedule.

I had seen this problem before. Those with any experience working with Linux and Windows will know what I am talking about, carriage returns. Windows line endings are different from Linux in that they consist of not just a new line (\n) but a carriage return (\r) as well.

So in the Dockerfile I stripped out the carriage returns from all of the Linux based files using sed. The command looks like this:

sed -i 's/\r//' export_env.sh

Final Result

The final result can be seen here on my GitHub page: https://github.com/alexhyett/dotnetcron If this has been useful or you know of a better way of doing any of the above then please comment below.


🙏 Was this helpful? If you want to say thanks, I love coffee ☕️ , any support is appreciated.


ALSO ON ALEXHYETT.COM

What is Event Driven Architecture?

What is Event Driven Architecture?

  • 14 April 2023
One of the leading architecture patterns used with microservices is event-driven architecture. Event-driven architecture has many benefits…
Hosting n8n for Free with Railway

Hosting n8n for Free with Railway

  • 30 January 2023
I have been using n8n for a couple of months now, and it has allowed me to automate so much of my daily workflow. These are some of the…
Using GitHub Actions to Deploy to S3

Using GitHub Actions to Deploy to S3

  • 26 March 2021
Recently I went through the process of setting up Drone CI on my Raspberry Pi. The plan was to use my Raspberry Pi as a build server for…
Getting Started with AWS Step Functions

Getting Started with AWS Step Functions

  • 12 March 2021
I have recently been looking into AWS Step Functions. For those not familiar with them, Step Functions are Amazon’s way of providing a state…
Useful Docker Commands Worth Saving

Useful Docker Commands Worth Saving

  • 12 February 2021
I use docker every day. All the applications I write at work or at home end up in docker containers. Most of the time though, I am only…
Grafana Monitoring on a Raspberry Pi

Grafana Monitoring on a Raspberry Pi

  • 28 January 2021
As you might have seen from my last few posts I have quite a lot running on my Raspberry Pi. I am currently using a Raspberry Pi 2 B which…
How to set up Drone CI on Raspberry Pi (and why you shouldn't)

How to set up Drone CI on Raspberry Pi (and why you shouldn't)

  • 27 January 2021
I wanted to put together my home build server using my Raspberry Pi. After looking at the options I picked Drone CI, it has a nice interface…
Traefik vs Nginx for Reverse Proxy with Docker on a Raspberry Pi

Traefik vs Nginx for Reverse Proxy with Docker on a Raspberry Pi

  • 20 January 2021
I use my Raspberry Pi as my own personal home server. Up until recently, I have been using nginx as a reverse proxy for my docker containers…