Senior Developers are Getting Rejected for Jobs

About how 5 months ago I went through a screening for a job. It was a referral from a friend and it had been a while since I responded to a recruiter. I was surprised how the process has changed in the past 5 years.

After the initial phone screen they sent me to a 3rd party site (HackerRank) to solve three programming puzzles in a one hour time box. It was my first attempt at this. The first two were easy but the last one was trickier. My solution didn’t pass all the unit tests. It passed something like 8/10 tests and there was no time left to debug it.

At this point I was filtered out of the company’s selection process. It turned out to be a mixed blessing due to some health issues that came up (I could not have made the trips to HQ to get oriented). However, the whole experience planted a seed in my mind.

Since then I’ve been familiarizing myself with this type of problem solving. I work through one every week.

It’s happening all over the place

I have a good friend looking for his next job, a PhD in Comp Sci with over 10 years experience. Almost every option he explores has some type of programming challenge gate either on a 3rd party site or in person. He bought a copy of Cracking the Coding Interview to get up to speed but it takes time to develop that skill. He’s been passed over by some great companies in the meantime.

The issue came up in the Megamaker Group and one person shared this very personal response:

I’m approaching my mid 40s and left my last startup (founding CTO role) in December. I have failed at least 10 programming tests and developer interviews since then. I have been developing technology for nearly 20 years including electronics and firmware (I’m an electronic engineering technologist by training) and full-blown distributed web applications with IoT integration. I’ve created software that powers full product traceability in high volume specialized manufacturing facilities around the world. Yet, I can’t get a job as a developer because I always fail these programming test gates.

This tweet from Max Howell went viral a couple of years ago. It’s funny, sad, and true all at the same time

Creater of homebrew rejected by google
https://twitter.com/mxcl/status/608682016205344768?lang=en

Fact. There are a lot of senior developers out there that are in for a wake-up call when they try to find their next position at a different company.


Developers Hate Them

When this topic comes up some developers say …

“I’ll typically end an interview if this type of stuff shows up”

or

“Being able to solve a puzzle like that means nothing, can you talk to clients? Can you actually build a functioning webapp? Can you Google stuff when needed? Can you learn what ever is thrown at you? Not can you find the most efficient way to order this weird data”

The counter argument is that poor software developers dislike being tested and they are the ones that the company is trying to weed out. However, it’s also possible that some strong, independent-minded, senior developers opt-out of the process because they have other options.

I agree these challenges are not representative of day-to-day tasks for a professional software developer. The wording is not always the best and I’m not sure the information is always complete (or at least I don’t always infer the completeness of the information from the problem description). Often they are math word problems. Having a formal education is an advantage.

The hiring company is guaranteed to screen out candidates that would have made amazing team members. For example, when Daniel Buchmueller was turned down at Netflix…

Tweet about Netflix passing over rockstar developer
https://twitter.com/rrubyist/status/1124448304555798529

Companies Love Them

In terms of programming challenges for hiring we need to realize the the world has changed. Remote work is becoming widely accepted in the software industry. International teams are increasingly common.

You now have a larger pool of great developers you can add to your team but, on the downside, this has drastically increased the number of applications you have to sort through in order to find the right candidate. Could you imagine sifting through 500 applications for a single position?

Companies Easily get 500 applications
https://twitter.com/ideasasylum/status/1126500299470807046

In addition, I’ve heard of getting into an interview where the candidate was way under qualified (couldn’t write a hello world type program). That would be extremely frustrating. Personally, I’d be looking for the nearest exit while offering the candidate some encouragement.

Programming puzzles as a hiring gate solve both these problems. To a company it is worth skipping over a few great candidates in order to simplify the review and selection process. With a now unlimited pool of applicants they can afford to do that. The numbers suggest that there will always be more good developers in the pipeline.

For this reason I believe that Programming challenge hiring gates are here to stay and will become even more common in the future.


It is true that everyone is hiring and senior developers have never been in more demand. But don’t think that you can walk into another job based on market demand and the number of years on your resume. Prepare for the programming test gates now while you have the luxury of time.


I have a newsletter for software developers.

Host Your Blazor Client on Netlify

This week I’ve been looking into moving my blog from wordpress self-hosted to a generated static site hosted on netlify. I was literally blown away at how dead simple it is to get up and running on netlify.

Unfortunately that migration process slowed down when I started converting wordpress posts to markdown format (even with the exitwp tool), however, a seed was planted.

Later this week when I was getting started creating my first Wasm/Blazor client from the VS2019 templates I had a hunch that I could host this on netlify. Once again, it was easier than I could have imagined.

Sidebar: Blazor is changing so quickly right now. And I’m using sdk 3.0 preview 4. I’m sure any screenshots will be outdated within a week or so.

Create a Basic Blazor Client Project Using the VS2019 Template

First get the latest update of VS2019 then make sure you have the latest .Net Core SDK 3.0 PREVIEW. Enable the preview in visual studio so you can actually use 3.0. In my opinion to get the best Blazor at the moment you have to be close to the front lines.

Checkbox for enabling preview versions of .Net Core SDK in Visual Studio 2019
Enable .Net Core SDK Preview Usage in Tools Menu > Options > Projects and Solutions > .Net Core

Now install the Blazor extension for Visual Studio. This isn’t necessary to build Blazor projects but it has some handy tools like a template for creating a client-side only Blazor app.

Use Manage Extensions to into the Blazor extension for Visual Studio
Extensions Menu > Manage Extensions

Create a ASP.NET Core Application. You may have to select ASP.NET Core 3.0 from the sdk dropdown menu.

Use the New Project Wizard to create a project using the Blazor (Client-side) template.

Publish the Blazor Client Site and Upload to Netlify

You can run the project from visual studio using the run button. Do that. Then right click on the project in Solution Explorer and select Publish. Use the folder option and select a location for the static output files (I use the default location).

Use the Publish wizard to save the wasm and associated static files for a local folder.

And now the fun part. Navigate to that folder where the static files were created and drag the dist folder into the browser of your netlify account. Magic. Deployment is automatic. It even handles SSL for you.

Drag the dist folder from your publish location into the Sites page of your Netlify account.

Learning In Public

This week I took some time off my day job. As I was tilling our vegetable garden area I was listening to the free code camp podcast.

This particular episode featured Shawn Wang. He took the work in public concept that I’ve seen elsewhere and applied it to learning – personal and professional development. He impressed me so much that I went and read his essay titled Learn In Public.

It instantly struck me to the core.

A New Mindset

This is the logical next step of my increase in transparency for 2019. So far I’ve been publishing weekly developer journal updates that have been at times quite personal, especially when I talked about how sick I was over the winter. And how it will affect me for the rest of my life.

It also touches on why I’ve been unsuccessful blogging regularly over the past year. I tried to seem like the expert of everything. But I’m not. Some things I do know. Most things I do not know. I’m pretty good at figuring new things out. When I couldn’t make everything perfect I stopped sharing.

From now on I’ll be writing more frequently. But it will be more in the style of learning in public. More building. More incomplete repos on github. More solving programming problems. More frequent small updates. More typos. Twitter posts using the #NoZeroDays hashtag or the #LearnInPublic hashtag.

My 2019 Learning Topics

To that end I have narrowed the theme of my study and sharing for the coming year

1. Web Assembly / Blazor

I’ve been digging into javascript for a while (mostly react) but I can’t say I’ve ever thought it was the ideal solution for me to use.

I think of javascript as an old European city like London that unfolded over time. No straight roads. No clear street naming convention. Roundabouts. Charming. Unique history. Loved by many.

To me web assembly is more like a modern city built on a grid system like New York. Engineered from the ground up with a blueprint for a purpose. Not perfect. Just different.

In addition Blazor ties in nicely with Signal R which has also caught my interest.

2. Functional Programming / F#

More than a couple of years ago I had some Haskell training and it was a good experience. Then 5 years ago I was smitten with clojure, but never got into it. Today I enjoy the functional parts of C# like LINQ and I use them a lot.

This week I was reading the Stack Overflow 2019 Developer Survey results. I noticed the pay discrepancy between C# developers and F# developers. Spoiler: Clojure and F# developers have the highest salary.

Seems like it is worth looking into.

I’m back!

This is who I am and what I’m about. Please follow along with me. I hope it will enrich your work.

Multidimensional Arrays in C#

Yesterday I was working with arrays and got hung up on a basic error message, a couple of them.

Object of type 'System.Int32[,]' cannot be converted to type 'System.Int32[][]
Cannot convert from an int[,] to an int[][]

There was not much on the internet that direct hit on these error strings (maybe it is too easy). Because of my java background it tripped me up for a few minutes.

Declaring a multidimensional array is not the same in C# as it is in Java.

//valid java, not valid in c#
int[][] array = new int[6][6];

In c# there are multidimensional arrays (think “matrix”) and jagged arrays (think “array of arrays”).

Multidimensional Array. Key points: simpler syntax but slower (more overhead). 2nd dimension the same length for all.

new int[6,6] {{-1, -1, 0, -9, -2, -2 },
{ -2, -1, -6, -8, -2, -5 },
{ -1, -1, -1, -2, -3, -4 },
{ -1, -9, -2, -4, -4, -5 },
{ -7, -3, -3, -2, -9, -9 },
{ -1, -3, -1, -2, -4, -5 } }

Jagged Array. 2nd dimension length can vary.

new int[6][] {new int[6] {-1, -1, 0, -9, -2, -2 },
new int[6] { -2, -1, -6, -8, -2, -5 },
new int[6] { -1, -1, -1, -2, -3, -4 },
new int[6] { -1, -9, -2, -4, -4, -5 },
new int[6] { -7, -3, -3, -2, -9, -9 },
new int[6] { -1, -3, -1, -2, -4, -5 } }

Once you realize this not-so-subtle difference there are lots of resources on the web available.

Resources

https://stackoverflow.com/questions/4648914/why-we-have-both-jagged-array-and-multidimensional-array

https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/arrays/multidimensional-arrays

https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/arrays/jagged-arrays

C# Is Dangerous

I was asked why C# is still relevant in 2018. What makes a language relevant to me? Someone pays me to work on it. Preferably a lot of money. With that there is this risk reward of developing rare skills. To me Cobol is relevant in 2018. A few people make a lot of money maintaining Cobol systems. However, (at least where I live) I don’t see a lot of ads for Cobol developers. I see a bunch of C# ads and a ton for Javascript developers. You need to seek the balance between valuable skills and available work.

Within a few days another friend asked me what programming language to learn as a first language in 2018. You might think I would suggest C#, but I didn’t. The truth is, as a professional software engineer I have little affinity for C#. I’ve written some articles on C# tooling because that’s what I look at day in and day out. But I would also be happy and very effective working in another language.

Building software is a bit like building houses. Take a builder who does brick houses vs one who does wood framing. Each material has strengths. Brick is fireproof. Wood withstands earthquakes. Each has regions of better pricing and availability. A superior builder knows building systems and when to use each material. Can a builder switch from building brick houses to wood-framed houses? A professional builder should be able to switch with time and study. The underlying building principles remain constant. But the materials, tooling, and techniques are different. Switching could take some trial and error.

Superior software engineers know principles over languages. GRASP Patterns. SOLID Principles. These do not change from language to language.

Now to the heart of the matter. Microsoft tries hard to reduce workload for developers. You click on new project in visual studio, select a template, and immediately you have a working web application with a database and everything. If this is your introduction to software development then you are building your software engineering base on sandy ground. For example, you will see dependency injection introduced by the template but not necessarily understand how it is working and why it is important. Without a proper foundation an IDE is a crutch as much as a tool.

This is why C# is dangerous. The Microsoft tooling makes it easy for inexperienced developers to build poor software. And this problem is not exclusive to C# but I worry that is it more prevalent with C# (and Visual Basic.NET).

C# as a relevant language in 2018 is not in question. It has any feature available that I’ve come to expect. Like a building system (think brick vs wood) some of these are better in C# and some are better in other languages. All of these are available in most languages but no language that I’ve found is perfect.

Openness / portability – Free. Cross-platform. Open source. Public design reviews. Weekly open calls. (contrast this to the new commercial licensing for java)

Modern language constructs– generics/ lambda expressions / anonymous functions / delegates. The list goes on and the specification evolves regularly. Microsoft hires some really smart people to oversee this.

Package management – NuGet. So convenient for software re-use. Integral to dotnet core.

Great libraries – well-documented, reliable, mostly open-source, and clean. Anything I need to do I can find a package and easily reference it.

Supported by cloud infrastructure – There are lots of different ways to host your application in the cloud. PAAS options on azure. Or container-based solutions using docker on pretty much any host (e.g. digital ocean droplets).

CSharp Side Hustle – Set Up Your Project And Git Repository

Where do we start? At the beginning. The very beginning. This material is so basic I almost skipped over it. But … there are some very explicit reasons why you set up your CSharp project and git repository for a side hustle project in this manner. And specific reasons why you choose certain tools. This is the first step to creating a web application using Angular ASP DotNet with Authentication. Basic flow shown in the video. More explanation in the article below.

As a reminder, a side hustle project should be a line of business application (do not try building the next facebook or re-inventing email). A side hustle project should be bootstrapped and self-sustaining.

Keep your side hustle project as simple as possible: one visual studio solution, one docker image, one git repository, one host. Do not over-complicate your web application prematurely. You will have several months of low traffic before you need to scale.

Overall Architecture – Javascript SPA (Angular) + Web API Back End (ASP DotNet Core) + PostgreSQL database via Entity Framework Core. There are some newer frameworks available, but for a side hustle new technology is a project killer. Stick to what you know. In my case Angular is well-supported in dotnet and is what I have the most experience with.

Install Tooling

Microsoft Visual Studio Community 2017 – Sign up for the free developer account, Visual Studio Dev Essentials. It seems the trend is away from Visual Studio in ASP DotNet Core project tutorials. I do not know why. Maybe the emphasis is on the cross-platformability of DotNet core. But the vast majority of DotNet developers are working in Windows. Visual Studio Community is free for individual developers. It has a much richer feature set than Visual Studio Code. Use it.

Notepad++ – It will save you the hassle of learning VI to write your commit messages for git.

Git for Windows – There is a git plugin with Visual Studio. Don’t do it. Learn git the way it was designed to be used … on the command-line. When installing accept all the defaults except for text editor. Select Notepad++. Thank me later.

Tortoise Git – nice for visualization. I use it for repository browsing only.

Scaffold Project from Template

You want Angular + Docker + Authentication. There is no perfect project template so it becomes a question of which project template will allow the least amount of manual grunt work.

  • Web API Template with Docker Support – we could use this one.
  • Angular SPA Template (without Docker Support) – this is probably the least work. It is easy to add docker files afterwards.
  • ASP DotNet MVC with Authentication – not really that helpful since it uses ASP DotNet Core Identity. In my experience I haven’t seen this working with an Angular client so I plan on using Identity Server.

Create Remote Repository and Push

This may be an unnecessary step for you. Since you’re only one person working on a side hustle project there is no need to coordinate or merge changes. You can do version management with a local git repository. The only thing you get from pushing to a remote repository is offsite backup – for the moment (in the future automated builds). So if you’ve got a rock solid backup mechanism in place maybe you want to keep your code in a local git repository.

Visual Studio Team Services – Free private git repositories as part of Visual Studio Dev Essentials. To be honest I use a self-hosted gerrit instance at work. I’m trying out the git repositories on VSTS for private code hosting. But Git is definitely the tool you want to know for managing your source code – especially since Microsoft Bought Git Hub. Why not use GitHub? Of course I agree GitHub is the fountain of all goodness. But … it costs $7/month for a private repository. When your project is earning $7/month you can choose to spend it on GitHub source code hosting. In the meantime use a free option since there is one available.

Conclusion

Keep it simple. There are so many ways that you could make this more complex than it needs to be. I’ve seen projects that separate the javascript client from the backend api. Different repository and different hosts, cross-origin requests (look that one up). Docker makes that more doable and maybe you will end up there with your side hustle project over time. But for now don’t over complicate things.

Opt for free over paid when the functionality is equivalent. Even if it is less cool. I know exactly zero individual devs who put their code in a git repository at VSTS for side hustle projects. Or who would rather be trying JetBrains Rider instead of Visual Studio Community? Keep your project self-sustaining.

In the next article we’re going to set up the project for docker support and hook up the database to back some web api calls.

Setup PostgreSQL with Docker for Your C# Database

Time to set up your database for your C# side hustle application. Stick to what you know. For most of us coming from a corporate environment we know SQL Server. But it is better to avoid SQL Server for side hustle applications because the license fees are so high.

The next best database at this time for C# applications on Entity Framework is PostgreSQL. It is feature-rich, mature, and well-supported in terms of the database itself and the entity framework providers.

Setting up your PostgreSQL Database Server in Docker

My preference is to spin it up in docker. Why? It’s temporary. Changeable. Simple version management and upgrades. Easy production setup.

Aside: One thing I’ve seen lately that I strongly want to discourage – creating custom builds of application docker images for the only purpose copying over configuration files. Don’t do that. It is unnecessary work and complexity. Use a volume with a mount to keep any configuration you need on the docker host. It makes configuration changes simpler and it is easier to manage the overall solution in a docker compose file(think many containers). I will show you how to do this.

You will probably want to spin up your development PostgreSQL docker container manually. There are two ways to persist your PostgreSQL data on the host machine: Volumes or Bind Mounts. If you’re on a Windows host then you definitely should use volumes since they are managed by Docker, not dependent on the host directory structure, and have consistent behaviour on Windows and Linux hosts. Even on any docker host, volumes have many advantages over bind mounts. However, in my experience bind mounts have been around longer and are more common.

Note: You will have problems if you start the PostgreSQL container with a Bind Mount on a Windows host. Read this article for more details. Deploy to a linux docker host in production.

So first create the volume:

docker volume create postgresql-volume

You can then use docker volume commands inspect or manipulate the volume:

docker volume ls
docker volume inspect <volume-name>
docker volume rm <volume-name>

Docker Volume Commands

 

Now you can spin your your postgresql container referencing the forementioned volume, postgresql-volume.

docker run -p 5432:5432 --name postgresql -e POSTGRES_PASSWORD=password -d -v 'postgresql-volume:/var/lib/postgresql/data' postgres:10.4

Note: It is a good idea to manage the version of the container you are running. Simply omitting the container version or using latest each time could cause unanticipated changes.

Connect to the command-line management interface on your docker container (to exit afterwards type ‘\q’).

docker exec -it postgresql psql -U postgres

For more details about using psql watch this video: https://www.youtube.com/watch?v=A8dErdDMqb0

docker run -p 5432:5432 –name MarketGardenManagerDB -e POSTGRES_PASSWORD=password -d -v ‘C:/Users/gmccallum/source/MarketGardenManager/db:/var/lib/postgresql/data’ postgres:10.4

Due to a limitation with the postgres docker container have to use a docker volume(above) instead of a volume mapping (below). Reference: http://www.lukaszewczak.com/2016/09/run-postgresql-using-docker-with.html

For development I like to keep my database data files in the project folder. Then they are closer at hand to blow them away if you want to re-create from scratch and refresh the container.

It is worth noting the different paradigm here. Infrastructure as code. Before the first push to production all the docker container details will be in the docker compose file and checked into a git repository. Any change to the server configuration is tracked with version control. My SQL Server server instances at my day job are set up and and maintained for as long as possible. Static versus transient. With docker containers you can easily rebuild the server from scratch every time there is a deployment.

Setup your PostgreSQL Administration Tool (in Docker)

The PostgreSQL equivalent of SQL Server Management Studio is pgAdmin. It is a web-based tool. There is a windows installer that runs it on a web server locally for you. Or .. you can do what I do … simply spin up the docker container.

Before starting the container ensure that that path to your configuration directory exists (C:/Users/gmccallum/.pgadmin). Then run the command to download and start the container:

docker run -p 8080:80 --name pgAdmin -v "C:/Users/gmccallum/.pgadmin:/var/lib/pgadmin" -e "PGADMIN_DEFAULT_EMAIL=glen.mccallum@protonmail.com" -e "PGADMIN_DEFAULT_PASSWORD=password" -d dpage/pgadmin4

Note: After what I said above about Volumes being better than Bind Mounts … I went and used a Bind Mount for the pgadmin data folder. Old habits die hard. I will switch this to be a Volume for permanent use.

Open it up in your browser at http://localhost:8080. Since this is run locally for you on your dev machine only you can skip the SSL certifcate and https configuration. However, if you plan on accessing this tool over a network (or the internet) see the notes on the docker hub page for further details.

pgAdmin Welcome Screen

 

By default docker containers run in bridged networking mode. So when you’re connecting from your pgAdmin container to your postgresql container you have to either connect to the host on the mapped port 5432. Or you can connect to the postrgresql container directly on port 5432. Considering the possible-transient nature of the IP address on your host machine I would consider connecting from container to container directly.

Connect to your PostgreSQL instance by right clicking on ‘Servers’ in the left pain, then ‘Create>’, ‘Server…’.

You can get the IP address of the postgresql container by using the inspect command from your host. Then scroll to the bottom and find the ip address.

docker inspect postgresql

Docker Inspect Container

Then use that IP from the pgAdmin Create Server interface and the password from above when you spun up the postresql container (‘postgres’ is the default username).

pgAdmin To Container Connection

 

To start and stop the pgAdmin container (my equivalent to opening and closing an application) just run:

docker start pgAdmin
docker stop pgAdmin

Connect to PostgreSQL From your C# Application

Wes Doyle has an awesome video on how to do this. I dunno how he does the whole thing in one take with zero mistakes. I do it the exact same way.

appsettings.json

"ConnectionStrings": {
"MyConnectionString": "User ID=postgres;Password=password;Server=localhost;Port=5432;Database=MyDatabase"
},

Startup.ConfigureServices

services.AddDbContext(
                options => options.UseNpgsql(Configuration.GetConnectionString("MyConnectionString")),
                    ServiceLifetime.Scoped);

Announcing My 2018 C# Side Hustle

In the first couple of posts it has been challenging to scope the topics. The way in which you apply a technology can be quite subjective based on what you’re trying to accomplish. So an article is either exhaustive enough to cover every possibility … or the article makes certain assumptions about the needs of the reader in order to limit the possibilities. For us, this situation is no longer.

I’m announcing my 2018 C# Side Hustle.

Side Hustle is a loose term, since I’m not expecting income to start rolling in or graphs that resemble hockey sticks. What I am hoping for is a tool to meet my own needs,  namely: (1) the functional benefits to my life for having this tool; (2) a vehicle for learning and evaluating new C# technologies in the context of development and production, and (3) a case study for illustrating topics covered on this site.

Product Vision

  • For professional market gardeners
  • Who need to know what to grow, how much to grow, and when to plant
  • The market garden manager is a responsive web application
  • That creates production objectives, plans CSA shares, calculates planting dates, develops crop calendars/garden plans, and keeps records
  • Unlike excel spreadsheets
  • Our product is ready to start planning your garden immediately, takes less time overall, easily adjustable during the season. (bonus: supports multiple users, is available in the garden on a mobile device, and produces weekly action lists)

Background

I have a small acreage. And with that came a sizable vegetable garden and a couple of small greenhouses. We’ve lived here for two summers, now approaching the 3rd.

The first year we lived here I bought the book The Market Gardener by Jean-Martin Fortier and spent 9 hours planning my crop schedule in excel. It was a work of art. Then … I had a late start due to ground-breaking work (the plots had been severely neglected). The vegetables were growing slower than I had anticipated. The excel schedule was difficult to adjust. By mid-season the garden plan was out the window. Although I made a second succession of some varieties the harvest was fair.

Second year I tried to re-use the crop plan from the first year. It was sort-of working … not really. But not far into the summer the goats broke into the garden and absolutely destroyed it. That took the wind out of my sails for that year. The harvest was poor.

Now I’m starting the 3rd summer and I still need a tool to help me map out my crops, order my seeds, and manage the movement of my plants through the seeding to harvesting cycle. Enter the Market Garden Manager. For serious gardeners, small farmers, and professional growers that know organization = success.

Rules

  1. Use the technology you know best. For me this is C#. I’d like to learn Go or Scala but getting bogged down is the enemy. Learn as little new technology as possible. It is a distraction from your main objective – to produce a product of value.
  2. Spend as little money as possible. It takes a long time to take a side hustle from nothing to self-sustaining. I expect years. Ongoing OS licenses, database licenses, development tool licenses will force you to pull the plug when you’re feeling discouraged. It all adds up. And this isn’t your corporate day-job. You don’t have that money to burn. Production-worthy open source platforms/tools should always take priority
  3. Marketing is as important as building. 50/50. What you build is as important as how you build it. Engaging your potential user base is the only way to ensure that you’re building something of value to them. If that means some advertising to get attention then so be it. These days there are other ways to get attention the I will be exploring.

The next series of articles will cover step-by-step the foundations of building any side hustle application in C#: basic idea/design, development process, persistence, API development, UI development, and authentication.

The Simplest Reverse Proxy for ASP.NET Core Using Docker

In my earlier post about hosting an ASP.NET core application in docker for production use I fudged a little bit in terms of what it means to production-worthy. Using the dotnet run command starts the web application on the Kestral web server. Kestral is a fine web server for development and maybe production use in an intranet environment but should not be exposed to the internet in a production environment.

You need a reverse proxy. Microsoft has two pages of documentation about this.

  1. Host ASP.NET Core on Linux with Apache
  2. Host ASP.NET Core on Linux with Nginx

The titles for these pages are misleading since the web app is not hosted by Nginix (or Apache). The ASP.NET Core application is running in Kestral and the reverse proxy server is in front managing the requests, logging, SSL termination, etc. Additionally this documentation is neither simple nor straightforward.

Here what follows is THE most painless way for setting up a reverse proxy server for an ASP.NET Core application server running in a docker container.

Overall you need to spin up two additional docker containers. First container is the reverse-proxy server itself, based on nginx. Second container is optional and manages the SSL cerficates from letsencrypt. Documentation for both containers is quality. I followed exactly.

(1) Start the Reverse Proxy Container

docker run -d -p 80:80 -p 443:443 \
--name nginx-proxy \
-v /opt/https_certs:/etc/nginx/certs:ro \
-v /etc/nginx/vhost.d \
-v /usr/share/nginx/html \
-v /var/run/docker.sock:/tmp/docker.sock:ro \
--label com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy \
jwilder/nginx-proxy

(This command is slightly different if you have purchased an SSL certificate separately. See the documentation)

(2) Start the Certificate Management Container

#docker run -d \
--name nginx-letsencrypt \
--volumes-from nginx-proxy \
-v /opt/https_certs:/etc/nginx/certs:rw \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-e NGINX_PROXY_CONTAINER=nginx-proxy \
jrcs/letsencrypt-nginx-proxy-companion

(3) Add a few environment variables to the docker-compose file for your ASP.NET Core web application

web:
environment:
- VIRTUAL_HOST=myrazorapp.glenmccallum.com
- LETSENCRYPT_HOST=myrazorapp.glenmccallum.com
- LETSENCRYPT_EMAIL=glen.mccallum@protonmail.com
expose:
- "5000"
image: glenmccallum/myrazorapp:latest
restart: always

(4) Start the container for your Web Application

docker compose up -d

(5) Wait for the magic to happen

It might take a while for the certificate management container to get the SSL certificate fetched and configured for your web application.

If  you’re impatient here are a couple of other commands to check on things.

# check on certificate status
docker exec nginx-letsencrypt /app/cert_status
# renew the SSL certificate(s) manually
docker exec nginx-letsencrypt /app/force_renew

BONUS: These containers will support/redirect multiple applications for domains/subdomains on the same host.

Basic ASP.net Core Razor App in Docker: From Inception to Production

I’m becoming a fan of Docker. In fact, every new web application I build will be deployed in a docker container. For so many reasons – such as consistency, portability, and resource efficiency (you can read about all those reasons on the internet).

However, in terms of documentation about getting started with .net applications on docker I found a gap. Microsoft documentation is specific to Microsoft tools/pipeline. And I’m not married to Visual Studio or VSTS. I don’t want to push a button in Visual Studio and have my container magically show up on an Azure VM – at least not yet. I want to learn the details and manual steps to deploy a container to my own linux server. I want to be able to choose where I host my applications. And I need to know what is happening under the hood.

My intention with this article is to take you from nothing to asp.net core application in a basic production docker container. There are a number of steps:

  1. Build an asp.net application in your development environment
  2. Add support for docker
  3. Build and publish a docker image
  4. Write a docker compose file
  5. Download and run your docker image in a container on your production linux server

You’re going to see this in the most straightforward fashion possible. Many topics (even some best practices) have been left out for the sake of simplicity. You can read about them in future articles. For example, automated builds of docker images from commits to a git repository and remote management of swarms of docker containers with docker machine.

Before we get started, there are a couple of prerequisites that you need installed on your computer.

  1. text editor
  2. .net core sdk
  3. Docker for Windows
  4. An account on Docker Hub
  5. Empty linux server sitting around that you can play on

Yes, I said Docker for Windows! (this from a .net linux guy). This post is showing how to build an application targeting a linux host from a Windows 10 development machine. Since I’m working entirely from the command-line the steps are almost identical to do this on a linux (or mac) development environment.

1. Create an ASP.net core web application

First thing, scaffold out a new application (you should be familiar with this).

>mkdir MyRazorApp
>cd MyRazorApp
>dotnet new razor

Output:

dotnet new razor

You can run it same as any dotnet core application. And open it in the browser (note the default port: 5000)

dotnet run myrazorapp

 

2. Add docker support to your project

Still in the MyRazorApp directory you can open the project in the text editor of your choice. I’m using VS Code (cause it supports dockerfile syntax). Launch it directly from powershell using this command.

>code .

In the root of that project directory create a new file named dockerfile. Populate it with the following and then save.

FROM microsoft/aspnetcore:latest

WORKDIR /webapp

COPY /webapp /webapp

EXPOSE 5000/tcp

ENV ASPNETCORE_URLS http://*:5000

ENTRYPOINT dotnet MyRazorApp.dll

In VS Code it looks like this (nice syntax highlighting!):

MyRazorApp Dockerfile in VSCode

 

Publish a release version of your dotnet project (since you plan to publish the docker image later for production use). Build a new Docker image. Then at this point you can launch your application inside of a docker container on your development machine directly from the local command-line in powershell.

>dotnet publish -c release -o webapp 
>docker build -t myrazorapp .
>docker run -d -p 8080:5000
>docker ps

Output (note the redirected port: 8080).

MyRazorApp Running Inside Docker on Local Machine

 

Obviously, since it is running on my development machine it is not yet a production deployment. However, now it is running in linux. When we ran it previously using dotnet run it was running in Windows. You don’t want this image running forever on your dev machine so these are the commands to tear it down.

>docker stop <containerid>
>docker ps -all
>docker rm <containerid> #if you want to remove the container after it is stopped
>docker ps -all

Docker Stop Container

3. Publish your Docker image to a repository

You can put your MyRazorApp docker image in a public or private repository – on docker hub or outside of docker hub private hosted even (a topic for another post). In addition it is customary to tag your images before publishing.

>docker login
>docker tag myrazorapp glenmccallum/myrazorapp
>docker push glenmccallum/myrazorapp
>docker logout

Output:

Push MyRazorApp to Docker Hub

 

And the MyRazorApp docker image appears in your Docker Hub dashboard!

MyRazorApp on Docker Hub

4. Create a Docker Compose file

For a basic example like MyRazorApp this is step optional. You can deploy manually. But for a more complex system a docker compose file may come into play earlier in the development cycle. Say if you’re deploying a separate docker container as a database server in addition to the web server.

The one time I’ve been involved with purchasing software deployed in a docker container the company sent us a docker compose file and granted us access to their private docker hub repository. That was how it was distributed. So you distribute only your docker compose file. Then a system administrator deploying your application (in the form of a docker image) can pull it directly from a public or private repository.

Docker compose has evolved. As container technologies have matured, the scope of the docker compose file has increased in subsequent versions (version 1, version 2, and version 3 as of this writing). It is now a tool for orchestrating multi-container distributed applications. It has networking and load-balancing capabilities and operates on a swarm (group of docker machines). In this section we’ll use the most basic version and only define one docker container within it – just enough to dip our toe in the water.

It does not get much more basic that this docker-compose.yml file:

web:
 image: glenmccallum/myrazorapp:latest
 ports:
 - "80:5000"
 restart: always

5. Download and run your docker image as a container on your production linux server

For simplicity I’m using Ubuntu Server 16.04.4 LTS running as a virtual server in hyper-v. But it could be a linux server running anywhere. The steps are the same even if you’re connecting via ssh to a droplet at Digital Ocean.

The tools of choice are Git Bash as an ssh client and WinSCP as an sftp client.

Start an ssh session to your ubuntu server in git bash. For the next 2 steps follow the instructions at the links. This instructions are well documented already.

Install Docker Engine (now part of Docker CE). https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1

Install Docker Compose. https://docs.docker.com/compose/install/

Then create a new directory in your home folder.

>mkdir MyRazorApp

Copy over the docker compose file using WinSCP.

WinSCP Docker Compose File

 

Back over to the ssh session in git bash:

>cd MyRazorApp
>sudo docker-compose up -d

Output:

Deploy MyRazorApp Container

And if it works you should see the application running on your linux server on port 80.

MyRazorApp in Docker on Ubuntu

 

It worked! This one post has so many areas we can investigate deeper. Let me know in the comments what you would like to learn more about.