© Les Jackson 2020
L. JacksonThe Complete ASP.NET Core 3 API Tutorialhttps://doi.org/10.1007/978-1-4842-6255-9_8

8. Environment Variables and User Secrets

Les Jackson1 
(1)
Melbourne, VIC, Australia
 

Chapter Summary

In this chapter we discuss what runtime environments are and how to configure them; we’ll then discuss what user secrets are and how to use them.

When Done, You Will

  • Understand what runtime environments are.

  • How to set them via the ASPNETCORE_ENVIRONMENT variable.

  • Understand the role of launchSettings.json and appsettings.json files.

  • What user secrets are.

  • How to use user secrets to solve the problem we had at the end of the last chapter.

Environments

When developing anything, you typically want the freedom to try new code, refactor existing code, and basically feel free to fail without impacting the end user. Imagine if you had to make code changes directly to a live customer environment? That would be
  • Stressful for you as a developer

  • Showing great irresponsibility as an application owner

  • Potentially impactful to the end user

Therefore, to avoid such a scenario, most, if not all organizations, will have some kind of “Development” environment where developers can roam free and go for it, without fear of screwing up.

../images/501438_1_En_8_Chapter/501438_1_En_8_Figa_HTML.jpgLes’ Personal Anecdote

If you’ve ever worked as part of a development team, you’ll know the preceding statement is not quite true. Yes, you can break things in the development environment without fear of impacting customers, but if you break the build, you will have the wrath of the other members of your team to deal with!

I know this from bitter experience.

Anyway, you’ll almost always have a Development environment, but what other environments can you have? Well, jumping to the other end of the spectrum, you’ll always have a Production environment . This is where the live production code sits and runs as the actual application, be it a customer-facing web site or in our case an API available for use by other applications.

You will typically never make code changes directly in production; indeed deployments and changes to production should be done, where possible, in as automated (and trackable) a way as possible, where the “human hand” doesn’t intervene to any large extent.

So, are they the only two environments you can have? Of course not, and this is where you’ll find the most differences in the real world. Most usually you will have some kind of “intermediate” environment (or environments) that sits in between Development and Production; it’s primary use is to “stage” the build in as close to a Production environment as possible to allow for integration and even user testing. Names for this this environment vary, but you’ll hear Microsoft refer to it as the “Staging” environment; I’ve also heard it called PR or “Production Replica.”

../images/501438_1_En_8_Chapter/501438_1_En_8_Figb_HTML.jpgLes’ Personal Anecdote

Replicating a Production environment accurately can be tricky (and expensive), especially if you work in a large corporate environment with lots of “legacy” systems that are maintained by different third-party vendors – coordinating this can be a nightmare.

There are of course ways to simulate these legacy systems, but again, there is really no substitute for the real thing. If you’re not simulating the legacy systems your app is interacting with precisely, that’s when you find those lovely bugs in production.

I remember being caught out with SQL case sensitivity on an Oracle DB while on site at a customer deployment. An easy fix when I realized the issue, but something as simple as that can be stressful and also damaging to your own reputation!

Our Environment Setup

We are going to dispense with the Staging or Production Replica environment and use only Development and Production – this is more than sufficient to demonstrate the necessary concepts we need to cover. Refer to the following diagram to see my environmental setup (yours should mirror this to a large extent).
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig1_HTML.jpg
Figure 8-1

Development and Production Environments

As you can see, the “components” that are there are effectively the same; it’s really only the underlying platform that is different (a local Windows PC vs. Microsoft Azure).

We’ll park further discussion on the Production Environment for now and come back to that in later chapters; for now, we’ll focus on our Development environment.

The Development Environment

How does our app know which environment it’s in? Quite simply – we tell it!

This is where “Environment Variables” come into play, specifically the ASPNETCORE_ENVIRONMENT variable. Environment variables can be specified, or set, in a number of different ways depending on the physical environment (Windows, OSX, Linux, Azure, etc.). So, while they can be set at the OS level, our discussion will focus setting them in the launchSettings.json file (this can be found in the Properties folder of your project) for now.

../images/501438_1_En_8_Chapter/501438_1_En_8_Figc_HTML.jpg Environment variables set in the launchSettings.json file will override environment variables set at the OS layer; that is why for the purposes of our discussion, we’ll just focus on setting out values in the launchSettings.json file.

A fuller discussion on multiple environments in ASP.NET Core can be found here.1

Opening the launchSettings.json file in the API project; you should see something similar to the following.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig2_HTML.jpg
Figure 8-2

LaunchSettings.json File

When you issue dotnet run at the .NET CLI the first profile with "commandName " : "Project" is used. The value of commandName specifies the webserver to launch. commandName can be any one of the following:
  • IISExpress

  • IIS

  • Project (which launches the Kestrel web server)

In the preceding highlighted profile section, there are also additional details that are specified including the “applicationUrl” for both http and https and well as our environmentVariables; in this instance we only have one: ASPNETCORE_ENVIRONMENT , set to: Development.

So, when an application is launched (via dotnet run)
  • launchSettings.json is read (if available).

  • environmentVariables settings override system/OS-defined environment variables.

  • The hosting environment is displayed.

For example, see Figure 8-3.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig3_HTML.jpg
Figure 8-3

Our environment is set to Development

So What?

At this stage I hear you all saying, “Yeah that’s great and everything, but so what?”

Good question; I’m glad you asked that question!2

Looking back at our simple environment setup, we need to connect to our Development database and eventually our Production database, and in almost all instances, they will be different, with different
  • Endpoints (e.g., Server Name/IP address, etc.)

  • Different log-in credentials, etc.

Therefore, depending on our environment, we’ll want to change our configuration.

I’m using the database connection string as an example here, but there are many other configurations that will change depending on the environment. That is why it is so important we are aware of our environment.

Make the Distinction

OK, so what approach should you take within your application to make determinations on configuration based on the development environment (e.g., use this connection string for Development and this one for Production)? Well there are a number of different answers to that; to my mind there are two broad approaches:
  1. 1.

    “Manually” determine the environment in your code, and take the necessary action.

     
  2. 2.

    Leverage the power and behavior of the .NET Core Configuration API.

     
We’re going to go with option 2. While option 1 is a possibility (indeed this pattern is used in many of the default .NET Core Projects – see the following example), I personally prefer to decouple code from configuration where possible, although it’s not always possible – that is why we’ll go with option 2.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig4_HTML.jpg
Figure 8-4

Code-based determination of environment

The preceding snippet is taken from our very own Startup class, where the default project template uses the IsDevelopment parameter to determine which exception page to use.

Order of Precedence

OK, so we’re going to leverage from the behavior of the .NET Core Configuration API to change the config as required for our two different environments (we’ve already made use of this when we configured the connection string for the DB Context).

Let’s quickly revisit the Program Class startup sequence for our app as covered in Chapter 4.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig5_HTML.jpg
Figure 8-5

Configuration sources and order of preference

You’ll see I’ve added some extra detail:
  • The launchSettings.json file is loaded when we issue the dotnet run command and set the value for ASPNETCORE_ENVIRONMENT.

  • A number of configuration sources that are used by the CreateDefaultBuilder method.

  • By default these sources are loaded in the precedence order specified previously, so appsettings.json is loaded first, followed by appsettings.Development.json, and so on.

../images/501438_1_En_8_Chapter/501438_1_En_8_Figd_HTML.jpg It is really important to note here that The Last Key Loaded Wins.

What this means (and we’ll demonstrate this below) is that if we have two configuration items with the same name, for example, our connection string, PostgreSqlConnection, that appears in different configuration sources, for example, appsettings.json and appsettings.Development.json, the value contained in appsettings.Development.json will be used.

So, you’ll notice here that Environment Variables will take precedence over the values in appsettings.json. This is the opposite of how this works when we talk about launchSettings.json. As previously mentioned, the contents of launchSettings.json take precedence over our system-defined environment variables.

So be careful!

I’ve referenced a great Blog Post on the Order of Precedence with Configuring ASP.NET Core here,3 for a further overview.

It’s Time to Move

OK, let’s put a bit of this theory into practice and demonstrate what we mean.
  • Go into your appsettings.json file, and copy the ConnectionStrings key–value pair that contains our PostgreSqlConnection connection string.

  • Make sure you have the correct values4 for User ID and Password.

  • Insert this JSON segment into the appsettings.Development.json file – see Figure 8-6.

This means we will have the same configuration element in both appsettings.json and appsettings.Development.json .
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig6_HTML.jpg
Figure 8-6

Appsettings.Development.json

Again, if you’re unsure that your JSON is well-formed, use something like http://jsoneditoronline.org/ to check.

Save the files you’ve made any changes to, run your API, and make the same call – it all still works as usual.

Let’s Break It

OK, so to prove the point we were previously making
  • Stop your API from running (Ctrl + c).

  • Go back into appsettings.Development.json file, and edit the Password parameter in the connection string so that authentication to the PostgreSQL Server will fail – see Figure 8-7.

  • Save your file.
    ../images/501438_1_En_8_Chapter/501438_1_En_8_Fig7_HTML.jpg
    Figure 8-7

    The wrong credentials

OK, now run the app again, and try to make the API Call.

Looking at the terminal output, you’ll see you get a database connection error; this is because the last value for our connection string was invalid.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig8_HTML.jpg
Figure 8-8

As expected, we can't connect

Fix It Up

OK, so let’s fix this:
  • Edit your appsettings.Development.json file, and correct the value for the Password parameter

  • Delete the ConnectionStrings json from the appsettings.json file.

This means that only our appsettings.Development.json file now contains our connection string; your appsettings.json file should now look like that in Figure 8-9.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig9_HTML.jpg
Figure 8-9

Cleaned up Appsettings.json

This means that currently, we only have a valid source for our connection string when running in a Development environment.

../images/501438_1_En_8_Chapter/501438_1_En_8_Fige_HTML.jpgLearning Opportunity

What will happen if you edit the launchSettings.json file and change the value of ASPNETCORE_ENVIRONMENT to “Production”?

Do this, run your app, and explain why you get this result.

We will cover our Production connection string in the Chapter 13.

User Secrets

We’ve covered the different environments you can have and why you have them and have even reconfigured our app to have a development environment-only connection string. But we still have not solved the issue we were left with at the end of the previous chapter – that being that, our User ID and Password are still in plaintext and are therefore available to anyone who has access to our source code – for example, someone looking at our repo in GitHub.

We solve that here.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig10_HTML.jpg
Figure 8-10

Secrets.json in the scheme of things

What Are User Secrets?

Well I gave you a bit of a clue in this chapter already.

In short, they are another location where you can store configuration elements; some points to note
  • User Secrets are “tied” to the individual developer – that is, you!

  • They are abstracted away from our source code and are not checked into any code repository.

  • They are stored in the “secrets.json” file.

  • The secrets.json file is unencrypted but is stored in a file system-protected user profile folder on the local dev machine.

This means that individual users can store (among other things) the credentials that they use to connect to a database. As the file is secured by the local file system, they remain secure (assuming no one has log-in access to your PC).

In terms of what you can store, this can be anything; it’s just string data. We’re now going to set up User Secrets for our development connection string.

Setting Up User Secrets

We need to make use of something called The Secret Manager Tool in order to make use of user secrets; this tool works on a project-by-project basis and therefore needs a way to uniquely identify each project. For this we need to make use of GUIDs.

../images/501438_1_En_8_Chapter/501438_1_En_8_Figf_HTML.jpgLearning Opportunity

Find out what GUID stands for, and do a little bit of reading on what they are and where they can be used (assuming you don’t know this already!)

Cast your mind back to Chapter 2 where we set up our development lab, and one of the extensions we suggested for VS Code was Insert GUID – well now we get to use it!

In VS Code open your CommandAPI.csproj file, and in the <PropertyGroup> xml element, place the xml highlighted in the following:
<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <UserSecretsId></UserSecretsId>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.EntityFrameworkCore" Version="3.0.0" />
    .
    .
    .
</Project>
  • Place your cursor in between the opening <UserSecretsId> and the closing </UserSecretsId> elements.

  • Open the VS Code “Command Palette”:
    • Press F1

    • Or Ctrl + Shift + P

    • Or View ➤ Command Palette

  • Type “Insert.”

../images/501438_1_En_8_Chapter/501438_1_En_8_Fig11_HTML.jpg
Figure 8-11

Insert GUID

  • Insert GUID should appear; select it and select the first GUID Option.
    ../images/501438_1_En_8_Chapter/501438_1_En_8_Fig12_HTML.jpg
    Figure 8-12

    Select this GUID Format

  • This should place the auto-generated GUID into the xml elements specified; see the following example.
    ../images/501438_1_En_8_Chapter/501438_1_En_8_Fig13_HTML.jpg
    Figure 8-13

    GUID Inserted into the .CSPROJ File

Now save your file.

Deciding Your Secrets

Now we come to actually adding our secrets via The Secret Manager Tool, which will generate a secrets.json file.

Before we do that though, we have a decision to make in regard to our connection string. Do we
  1. 1.

    Want to store our entire connection string as a single secret.

     
  2. 2.

    Store our User Id and Password as individual secrets and retain the remainder of the connection string in the appsettings.Developent.json file.

     

Either will work, but I’m going to go with option 2 where we will store the individual components as “secrets.”

So, to add our two secrets:
  • Ensure you have generated the GUID as described earlier, and save the .csproj file.

  • At a terminal command (and make sure you’re “inside” the CommandAPI project folder), type

dotnet user-secrets set “UserID” “cmddbuser”
You should get a “Successfully saved UserID…” message.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig14_HTML.jpg
Figure 8-14

Adding our first user secret

Repeat the same step and add the “Password” secret
dotnet user-secrets set “Password” “pa55w0rd!”

Again, you should get a similar success message.

Where Are They?

So where did our secrets end up? That’s right, in our secrets.json file. You can find this file in a system-protected user profile folder on your local machine at the following location:
  • Windows: %APPDATA%MicrosoftUserSecrets<user_secrets_id>secrets.json

  • Linux/OSX: ~/.microsoft/usersecrets/<user_secrets_id>/secrets.json

So, on my machine, it can be found here.5
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig15_HTML.jpg
Figure 8-15

Location of Secrets.Json on Windows

Open this file, and have a look at the contents:
{
  "UserID": "cmddbuser",
  "Password": "pa55w0rd!"
}

It’s just a simple, non-encrypted JSON file.

Code It Up

OK, so now to the really exciting bit where we’ll actually use these secrets to build out our full connection string.

Step 1: Remove User ID and Password

We want to remove the “offending articles” from our existing connection string in our appsettings.Development.json file.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig16_HTML.jpg
Figure 8-16

Removal of sensitive connection string attributes

So our appsettings.Development.json file should now contain only
{
  "Logging": {
    "LogLevel": {
      "Default": "Debug",
      "System": "Information",
      "Microsoft": "Information"
    }
  },
  "ConnectionStrings":
  {
    "PostgreSqlConnection":
      "Host=localhost;Port=5432;Database=CmdAPI;Pooling=true;"
  }
}

Make sure you save your file.

Step 2: Build Our Connection String

Move over into our Startup class, and add the following code to the ConfigureServices method (noting the inclusion of the new using statement at the top):
.
.
.
using Npgsql;
namespace CommandAPI
{
    public class Startup
    {
        public IConfiguration Configuration {get;}
        public Startup(IConfiguration configuration) => Configuration = configuration;
        public void ConfigureServices(IServiceCollection services)
        {
            var builder = new NpgsqlConnectionStringBuilder();
            builder.ConnectionString =
              Configuration.GetConnectionString("PostgreSqlConnection");
            builder.Username = Configuration["UserID"];
            builder.Password = Configuration["Password"];
            services.AddDbContext<CommandContext>
                (opt => opt.UseNpgsql(builder.ConnectionString));
            services.AddControllers();
            services.AddScoped<ICommandAPIRepo, SqlCommandAPIRepo>();
        }
.
.
.
Again, for clarity I’ve circled the new/updated sections below:
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig17_HTML.jpg
Figure 8-17

Updated Startup class

  1. 1.

    We need to add a reference to Npqsql in order to use NpgsqlConnectionStringBuilder.

     
  2. 2.
    This is where we
    1. a.

      Create a NpgsqlConnectionStringBuilder object, and pass in our “base” connection string PostgreSqlConnection from our appsettings.Development.json file.

       
    2. b.

      Continue to “build” the string by passing in both our UserID and Password secret from our secrets.json file.

       
     
  3. 3.

    Replace the original connection string with the newly constructed string using our builder object.

     

Save your work, build it, then run it. Fire up Postman, and issue our GET request to our API. You should get a success!

../images/501438_1_En_8_Chapter/501438_1_En_8_Figg_HTML.jpgCelebration Checkpoint

You have now dynamically created a connection string using a combination of configuration sources, one of which is User Secrets from our secrets.json file!

Just cast your mind back to the following diagram.
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig18_HTML.jpg
Figure 8-18

Revisit of precedence

The .NET Configuration layer by default provides us access to the configuration sources as shown in Figure 8-18; in this case we used a combination of 2 + 3.

Wrap It Up

Again, we covered a lot in this chapter; the main points are
  • We moved our connection string to a development-only config file: appsetting.Development.json.

  • We removed the sensitive items from our connection string.

  • We moved the sensitive items (User ID and Password) to secrets.json via The Secret Manager Tool.

  • We constructed a fully working connection string using a combination of configuration sources.

All that’s left to do is commit all our changes to Git then push up to GitHub!

Moving over to our repository and taking a look in the appsettings.Development.json file, we see an innocent connection string without user credentials (the secrets.json file is not added to source control)!
../images/501438_1_En_8_Chapter/501438_1_En_8_Fig19_HTML.jpg
Figure 8-19

Clean Appsettings.json on GitHub

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.16.254