CSharp Side Hustle – Set Up Your Project And Git Repository

Where do we start? At the beginning. The very beginning. This material is so basic I almost skipped over it. But … there are some very explicit reasons why you set up your CSharp project and git repository for a side hustle project in this manner. And specific reasons why you choose certain tools. This is the first step to creating a web application using Angular ASP DotNet with Authentication. Basic flow shown in the video. More explanation in the article below.

As a reminder, a side hustle project should be a line of business application (do not try building the next facebook or re-inventing email). A side hustle project should be bootstrapped and self-sustaining.

Keep your side hustle project as simple as possible: one visual studio solution, one docker image, one git repository, one host. Do not over-complicate your web application prematurely. You will have several months of low traffic before you need to scale.

Overall Architecture – Javascript SPA (Angular) + Web API Back End (ASP DotNet Core) + PostgreSQL database via Entity Framework Core. There are some newer frameworks available, but for a side hustle new technology is a project killer. Stick to what you know. In my case Angular is well-supported in dotnet and is what I have the most experience with.

Install Tooling

Microsoft Visual Studio Community 2017 – Sign up for the free developer account, Visual Studio Dev Essentials. It seems the trend is away from Visual Studio in ASP DotNet Core project tutorials. I do not know why. Maybe the emphasis is on the cross-platformability of DotNet core. But the vast majority of DotNet developers are working in Windows. Visual Studio Community is free for individual developers. It has a much richer feature set than Visual Studio Code. Use it.

Notepad++ – It will save you the hassle of learning VI to write your commit messages for git.

Git for Windows – There is a git plugin with Visual Studio. Don’t do it. Learn git the way it was designed to be used … on the command-line. When installing accept all the defaults except for text editor. Select Notepad++. Thank me later.

Tortoise Git – nice for visualization. I use it for repository browsing only.

Scaffold Project from Template

You want Angular + Docker + Authentication. There is no perfect project template so it becomes a question of which project template will allow the least amount of manual grunt work.

  • Web API Template with Docker Support – we could use this one.
  • Angular SPA Template (without Docker Support) – this is probably the least work. It is easy to add docker files afterwards.
  • ASP DotNet MVC with Authentication – not really that helpful since it uses ASP DotNet Core Identity. In my experience I haven’t seen this working with an Angular client so I plan on using Identity Server.

Create Remote Repository and Push

This may be an unnecessary step for you. Since you’re only one person working on a side hustle project there is no need to coordinate or merge changes. You can do version management with a local git repository. The only thing you get from pushing to a remote repository is offsite backup – for the moment (in the future automated builds). So if you’ve got a rock solid backup mechanism in place maybe you want to keep your code in a local git repository.

Visual Studio Team Services – Free private git repositories as part of Visual Studio Dev Essentials. To be honest I use a self-hosted gerrit instance at work. I’m trying out the git repositories on VSTS for private code hosting. But Git is definitely the tool you want to know for managing your source code – especially since Microsoft Bought Git Hub. Why not use GitHub? Of course I agree GitHub is the fountain of all goodness. But … it costs $7/month for a private repository. When your project is earning $7/month you can choose to spend it on GitHub source code hosting. In the meantime use a free option since there is one available.

Conclusion

Keep it simple. There are so many ways that you could make this more complex than it needs to be. I’ve seen projects that separate the javascript client from the backend api. Different repository and different hosts, cross-origin requests (look that one up). Docker makes that more doable and maybe you will end up there with your side hustle project over time. But for now don’t over complicate things.

Opt for free over paid when the functionality is equivalent. Even if it is less cool. I know exactly zero individual devs who put their code in a git repository at VSTS for side hustle projects. Or who would rather be trying JetBrains Rider instead of Visual Studio Community? Keep your project self-sustaining.

In the next article we’re going to set up the project for docker support and hook up the database to back some web api calls.

Setup PostgreSQL with Docker for Your C# Database

Time to set up your database for your C# side hustle application. Stick to what you know. For most of us coming from a corporate environment we know SQL Server. But it is better to avoid SQL Server for side hustle applications because the license fees are so high.

The next best database at this time for C# applications on Entity Framework is PostgreSQL. It is feature-rich, mature, and well-supported in terms of the database itself and the entity framework providers.

Setting up your PostgreSQL Database Server in Docker

My preference is to spin it up in docker. Why? It’s temporary. Changeable. Simple version management and upgrades. Easy production setup.

Aside: One thing I’ve seen lately that I strongly want to discourage – creating custom builds of application docker images for the only purpose copying over configuration files. Don’t do that. It is unnecessary work and complexity. Use a volume with a mount to keep any configuration you need on the docker host. It makes configuration changes simpler and it is easier to manage the overall solution in a docker compose file(think many containers). I will show you how to do this.

You will probably want to spin up your development PostgreSQL docker container manually. There are two ways to persist your PostgreSQL data on the host machine: Volumes or Bind Mounts. If you’re on a Windows host then you definitely should use volumes since they are managed by Docker, not dependent on the host directory structure, and have consistent behaviour on Windows and Linux hosts. Even on any docker host, volumes have many advantages over bind mounts. However, in my experience bind mounts have been around longer and are more common.

Note: You will have problems if you start the PostgreSQL container with a Bind Mount on a Windows host. Read this article for more details. Deploy to a linux docker host in production.

So first create the volume:

docker volume create postgresql-volume

You can then use docker volume commands inspect or manipulate the volume:

docker volume ls
docker volume inspect <volume-name>
docker volume rm <volume-name>

Docker Volume Commands

 

Now you can spin your your postgresql container referencing the forementioned volume, postgresql-volume.

docker run -p 5432:5432 --name postgresql -e POSTGRES_PASSWORD=password -d -v 'postgresql-volume:/var/lib/postgresql/data' postgres:10.4

Note: It is a good idea to manage the version of the container you are running. Simply omitting the container version or using latest each time could cause unanticipated changes.

Connect to the command-line management interface on your docker container (to exit afterwards type ‘\q’).

docker exec -it postgresql psql -U postgres

For more details about using psql watch this video: https://www.youtube.com/watch?v=A8dErdDMqb0

docker run -p 5432:5432 –name MarketGardenManagerDB -e POSTGRES_PASSWORD=password -d -v ‘C:/Users/gmccallum/source/MarketGardenManager/db:/var/lib/postgresql/data’ postgres:10.4

Due to a limitation with the postgres docker container have to use a docker volume(above) instead of a volume mapping (below). Reference: http://www.lukaszewczak.com/2016/09/run-postgresql-using-docker-with.html

For development I like to keep my database data files in the project folder. Then they are closer at hand to blow them away if you want to re-create from scratch and refresh the container.

It is worth noting the different paradigm here. Infrastructure as code. Before the first push to production all the docker container details will be in the docker compose file and checked into a git repository. Any change to the server configuration is tracked with version control. My SQL Server server instances at my day job are set up and and maintained for as long as possible. Static versus transient. With docker containers you can easily rebuild the server from scratch every time there is a deployment.

Setup your PostgreSQL Administration Tool (in Docker)

The PostgreSQL equivalent of SQL Server Management Studio is pgAdmin. It is a web-based tool. There is a windows installer that runs it on a web server locally for you. Or .. you can do what I do … simply spin up the docker container.

Before starting the container ensure that that path to your configuration directory exists (C:/Users/gmccallum/.pgadmin). Then run the command to download and start the container:

docker run -p 8080:80 --name pgAdmin -v "C:/Users/gmccallum/.pgadmin:/var/lib/pgadmin" -e "PGADMIN_DEFAULT_EMAIL=glen.mccallum@protonmail.com" -e "PGADMIN_DEFAULT_PASSWORD=password" -d dpage/pgadmin4

Note: After what I said above about Volumes being better than Bind Mounts … I went and used a Bind Mount for the pgadmin data folder. Old habits die hard. I will switch this to be a Volume for permanent use.

Open it up in your browser at http://localhost:8080. Since this is run locally for you on your dev machine only you can skip the SSL certifcate and https configuration. However, if you plan on accessing this tool over a network (or the internet) see the notes on the docker hub page for further details.

pgAdmin Welcome Screen

 

By default docker containers run in bridged networking mode. So when you’re connecting from your pgAdmin container to your postgresql container you have to either connect to the host on the mapped port 5432. Or you can connect to the postrgresql container directly on port 5432. Considering the possible-transient nature of the IP address on your host machine I would consider connecting from container to container directly.

Connect to your PostgreSQL instance by right clicking on ‘Servers’ in the left pain, then ‘Create>’, ‘Server…’.

You can get the IP address of the postgresql container by using the inspect command from your host. Then scroll to the bottom and find the ip address.

docker inspect postgresql

Docker Inspect Container

Then use that IP from the pgAdmin Create Server interface and the password from above when you spun up the postresql container (‘postgres’ is the default username).

pgAdmin To Container Connection

 

To start and stop the pgAdmin container (my equivalent to opening and closing an application) just run:

docker start pgAdmin
docker stop pgAdmin

Connect to PostgreSQL From your C# Application

Wes Doyle has an awesome video on how to do this. I dunno how he does the whole thing in one take with zero mistakes. I do it the exact same way.

appsettings.json

"ConnectionStrings": {
"MyConnectionString": "User ID=postgres;Password=password;Server=localhost;Port=5432;Database=MyDatabase"
},

Startup.ConfigureServices

services.AddDbContext(
                options => options.UseNpgsql(Configuration.GetConnectionString("MyConnectionString")),
                    ServiceLifetime.Scoped);

Announcing My 2018 C# Side Hustle

In the first couple of posts it has been challenging to scope the topics. The way in which you apply a technology can be quite subjective based on what you’re trying to accomplish. So an article is either exhaustive enough to cover every possibility … or the article makes certain assumptions about the needs of the reader in order to limit the possibilities. For us, this situation is no longer.

I’m announcing my 2018 C# Side Hustle.

Side Hustle is a loose term, since I’m not expecting income to start rolling in or graphs that resemble hockey sticks. What I am hoping for is a tool to meet my own needs,  namely: (1) the functional benefits to my life for having this tool; (2) a vehicle for learning and evaluating new C# technologies in the context of development and production, and (3) a case study for illustrating topics covered on this site.

Product Vision

  • For professional market gardeners
  • Who need to know what to grow, how much to grow, and when to plant
  • The market garden manager is a responsive web application
  • That creates production objectives, plans CSA shares, calculates planting dates, develops crop calendars/garden plans, and keeps records
  • Unlike excel spreadsheets
  • Our product is ready to start planning your garden immediately, takes less time overall, easily adjustable during the season. (bonus: supports multiple users, is available in the garden on a mobile device, and produces weekly action lists)

Background

I have a small acreage. And with that came a sizable vegetable garden and a couple of small greenhouses. We’ve lived here for two summers, now approaching the 3rd.

The first year we lived here I bought the book The Market Gardener by Jean-Martin Fortier and spent 9 hours planning my crop schedule in excel. It was a work of art. Then … I had a late start due to ground-breaking work (the plots had been severely neglected). The vegetables were growing slower than I had anticipated. The excel schedule was difficult to adjust. By mid-season the garden plan was out the window. Although I made a second succession of some varieties the harvest was fair.

Second year I tried to re-use the crop plan from the first year. It was sort-of working … not really. But not far into the summer the goats broke into the garden and absolutely destroyed it. That took the wind out of my sails for that year. The harvest was poor.

Now I’m starting the 3rd summer and I still need a tool to help me map out my crops, order my seeds, and manage the movement of my plants through the seeding to harvesting cycle. Enter the Market Garden Manager. For serious gardeners, small farmers, and professional growers that know organization = success.

Rules

  1. Use the technology you know best. For me this is C#. I’d like to learn Go or Scala but getting bogged down is the enemy. Learn as little new technology as possible. It is a distraction from your main objective – to produce a product of value.
  2. Spend as little money as possible. It takes a long time to take a side hustle from nothing to self-sustaining. I expect years. Ongoing OS licenses, database licenses, development tool licenses will force you to pull the plug when you’re feeling discouraged. It all adds up. And this isn’t your corporate day-job. You don’t have that money to burn. Production-worthy open source platforms/tools should always take priority
  3. Marketing is as important as building. 50/50. What you build is as important as how you build it. Engaging your potential user base is the only way to ensure that you’re building something of value to them. If that means some advertising to get attention then so be it. These days there are other ways to get attention the I will be exploring.

The next series of articles will cover step-by-step the foundations of building any side hustle application in C#: basic idea/design, development process, persistence, API development, UI development, and authentication.

The Simplest Reverse Proxy for ASP.NET Core Using Docker

In my earlier post about hosting an ASP.NET core application in docker for production use I fudged a little bit in terms of what it means to production-worthy. Using the dotnet run command starts the web application on the Kestral web server. Kestral is a fine web server for development and maybe production use in an intranet environment but should not be exposed to the internet in a production environment.

You need a reverse proxy. Microsoft has two pages of documentation about this.

  1. Host ASP.NET Core on Linux with Apache
  2. Host ASP.NET Core on Linux with Nginx

The titles for these pages are misleading since the web app is not hosted by Nginix (or Apache). The ASP.NET Core application is running in Kestral and the reverse proxy server is in front managing the requests, logging, SSL termination, etc. Additionally this documentation is neither simple nor straightforward.

Here what follows is THE most painless way for setting up a reverse proxy server for an ASP.NET Core application server running in a docker container.

Overall you need to spin up two additional docker containers. First container is the reverse-proxy server itself, based on nginx. Second container is optional and manages the SSL cerficates from letsencrypt. Documentation for both containers is quality. I followed exactly.

(1) Start the Reverse Proxy Container

docker run -d -p 80:80 -p 443:443 \
--name nginx-proxy \
-v /opt/https_certs:/etc/nginx/certs:ro \
-v /etc/nginx/vhost.d \
-v /usr/share/nginx/html \
-v /var/run/docker.sock:/tmp/docker.sock:ro \
--label com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy \
jwilder/nginx-proxy

(This command is slightly different if you have purchased an SSL certificate separately. See the documentation)

(2) Start the Certificate Management Container

#docker run -d \
--name nginx-letsencrypt \
--volumes-from nginx-proxy \
-v /opt/https_certs:/etc/nginx/certs:rw \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-e NGINX_PROXY_CONTAINER=nginx-proxy \
jrcs/letsencrypt-nginx-proxy-companion

(3) Add a few environment variables to the docker-compose file for your ASP.NET Core web application

web:
environment:
- VIRTUAL_HOST=myrazorapp.glenmccallum.com
- LETSENCRYPT_HOST=myrazorapp.glenmccallum.com
- LETSENCRYPT_EMAIL=glen.mccallum@protonmail.com
expose:
- "5000"
image: glenmccallum/myrazorapp:latest
restart: always

(4) Start the container for your Web Application

docker compose up -d

(5) Wait for the magic to happen

It might take a while for the certificate management container to get the SSL certificate fetched and configured for your web application.

If  you’re impatient here are a couple of other commands to check on things.

# check on certificate status
docker exec nginx-letsencrypt /app/cert_status
# renew the SSL certificate(s) manually
docker exec nginx-letsencrypt /app/force_renew

BONUS: These containers will support/redirect multiple applications for domains/subdomains on the same host.

Basic ASP.net Core Razor App in Docker: From Inception to Production

I’m becoming a fan of Docker. In fact, every new web application I build will be deployed in a docker container. For so many reasons – such as consistency, portability, and resource efficiency (you can read about all those reasons on the internet).

However, in terms of documentation about getting started with .net applications on docker I found a gap. Microsoft documentation is specific to Microsoft tools/pipeline. And I’m not married to Visual Studio or VSTS. I don’t want to push a button in Visual Studio and have my container magically show up on an Azure VM – at least not yet. I want to learn the details and manual steps to deploy a container to my own linux server. I want to be able to choose where I host my applications. And I need to know what is happening under the hood.

My intention with this article is to take you from nothing to asp.net core application in a basic production docker container. There are a number of steps:

  1. Build an asp.net application in your development environment
  2. Add support for docker
  3. Build and publish a docker image
  4. Write a docker compose file
  5. Download and run your docker image in a container on your production linux server

You’re going to see this in the most straightforward fashion possible. Many topics (even some best practices) have been left out for the sake of simplicity. You can read about them in future articles. For example, automated builds of docker images from commits to a git repository and remote management of swarms of docker containers with docker machine.

Before we get started, there are a couple of prerequisites that you need installed on your computer.

  1. text editor
  2. .net core sdk
  3. Docker for Windows
  4. An account on Docker Hub
  5. Empty linux server sitting around that you can play on

Yes, I said Docker for Windows! (this from a .net linux guy). This post is showing how to build an application targeting a linux host from a Windows 10 development machine. Since I’m working entirely from the command-line the steps are almost identical to do this on a linux (or mac) development environment.

1. Create an ASP.net core web application

First thing, scaffold out a new application (you should be familiar with this).

>mkdir MyRazorApp
>cd MyRazorApp
>dotnet new razor

Output:

dotnet new razor

You can run it same as any dotnet core application. And open it in the browser (note the default port: 5000)

dotnet run myrazorapp

 

2. Add docker support to your project

Still in the MyRazorApp directory you can open the project in the text editor of your choice. I’m using VS Code (cause it supports dockerfile syntax). Launch it directly from powershell using this command.

>code .

In the root of that project directory create a new file named dockerfile. Populate it with the following and then save.

FROM microsoft/aspnetcore:latest

WORKDIR /webapp

COPY /webapp /webapp

EXPOSE 5000/tcp

ENV ASPNETCORE_URLS http://*:5000

ENTRYPOINT dotnet MyRazorApp.dll

In VS Code it looks like this (nice syntax highlighting!):

MyRazorApp Dockerfile in VSCode

 

Publish a release version of your dotnet project (since you plan to publish the docker image later for production use). Build a new Docker image. Then at this point you can launch your application inside of a docker container on your development machine directly from the local command-line in powershell.

>dotnet publish -c release -o webapp 
>docker build -t myrazorapp .
>docker run -d -p 8080:5000
>docker ps

Output (note the redirected port: 8080).

MyRazorApp Running Inside Docker on Local Machine

 

Obviously, since it is running on my development machine it is not yet a production deployment. However, now it is running in linux. When we ran it previously using dotnet run it was running in Windows. You don’t want this image running forever on your dev machine so these are the commands to tear it down.

>docker stop <containerid>
>docker ps -all
>docker rm <containerid> #if you want to remove the container after it is stopped
>docker ps -all

Docker Stop Container

3. Publish your Docker image to a repository

You can put your MyRazorApp docker image in a public or private repository – on docker hub or outside of docker hub private hosted even (a topic for another post). In addition it is customary to tag your images before publishing.

>docker login
>docker tag myrazorapp glenmccallum/myrazorapp
>docker push glenmccallum/myrazorapp
>docker logout

Output:

Push MyRazorApp to Docker Hub

 

And the MyRazorApp docker image appears in your Docker Hub dashboard!

MyRazorApp on Docker Hub

4. Create a Docker Compose file

For a basic example like MyRazorApp this is step optional. You can deploy manually. But for a more complex system a docker compose file may come into play earlier in the development cycle. Say if you’re deploying a separate docker container as a database server in addition to the web server.

The one time I’ve been involved with purchasing software deployed in a docker container the company sent us a docker compose file and granted us access to their private docker hub repository. That was how it was distributed. So you distribute only your docker compose file. Then a system administrator deploying your application (in the form of a docker image) can pull it directly from a public or private repository.

Docker compose has evolved. As container technologies have matured, the scope of the docker compose file has increased in subsequent versions (version 1, version 2, and version 3 as of this writing). It is now a tool for orchestrating multi-container distributed applications. It has networking and load-balancing capabilities and operates on a swarm (group of docker machines). In this section we’ll use the most basic version and only define one docker container within it – just enough to dip our toe in the water.

It does not get much more basic that this docker-compose.yml file:

web:
 image: glenmccallum/myrazorapp:latest
 ports:
 - "80:5000"
 restart: always

5. Download and run your docker image as a container on your production linux server

For simplicity I’m using Ubuntu Server 16.04.4 LTS running as a virtual server in hyper-v. But it could be a linux server running anywhere. The steps are the same even if you’re connecting via ssh to a droplet at Digital Ocean.

The tools of choice are Git Bash as an ssh client and WinSCP as an sftp client.

Start an ssh session to your ubuntu server in git bash. For the next 2 steps follow the instructions at the links. This instructions are well documented already.

Install Docker Engine (now part of Docker CE). https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1

Install Docker Compose. https://docs.docker.com/compose/install/

Then create a new directory in your home folder.

>mkdir MyRazorApp

Copy over the docker compose file using WinSCP.

WinSCP Docker Compose File

 

Back over to the ssh session in git bash:

>cd MyRazorApp
>sudo docker-compose up -d

Output:

Deploy MyRazorApp Container

And if it works you should see the application running on your linux server on port 80.

MyRazorApp in Docker on Ubuntu

 

It worked! This one post has so many areas we can investigate deeper. Let me know in the comments what you would like to learn more about.