Creating an IIS Web Server Farm with DFSR and Shared Configuration

In a previous article we configured a single IIS webserver using powershell commands.

Now we want to add some additional servers that will run the same web-sites so that we can load balance incoming requests.

The first step is to create one or more additional web servers using the instructions in the previous article however in this case we don’t need to create folders, file shares or create any websites or application pools in IIS. We are going to sync all of that stuff from our master server using DFS Replication.

Installing DFS Replication on each web server

  • Install-WindowsFeature -Name FS-DFS-Replication -Confirm

Create WebFarmFiles Folder (if required) on each web server

  • New-Item -ItemType Directory -Path C:\WebFarmFiles

Install management tools on a Full GUI machine

Unfortunately the DFS management tools (even the powershell cmdlets) can only be installed on a Windows computer with a full GUI at this stage for some reason. Therefore, you will need to have another server (or workstation) that you can use to manage your DFS Replication Group. I am not sure why this is the case, but hopefully Microsoft fix it soon.

  • Install-WindowsFeature -Name RSAT-DFS-Mgmt-Con -Confirm
  • Get-DfsReplicationGroup this cmdlet should run once the tools are installed.

Create a DFS Replication Group

As we have been forced onto a Full GUI machine, you could do the DFS setup using the DFS Management console. Since we are having so much fun though let’s keep going with setting everything up via Powershell.

Run the following command from the Full GUI management machine. Replace ‘YourGroup’ and ‘your.domain’ with the relevant values and feel free to modify the description to suit.

  • New-DfsReplicationGroup -GroupName YourGroup -DomainName your.domain -Description
    "DFS Replication between YourGroup servers to sync content and configuration" -Confirm

Add Members to the DFSR Group

Run the following command for each ComputerName that needs to be added to YourGroup.
Add-DfsrMember -GroupName YourGroup -ComputerName WEB0X -Confirm

Add DFS Replicated Folder

  • New-DfsReplicatedFolder -GroupName YourGroup -FolderName WebFarmFiles -Confirm

Add Connections between web servers

The command below creates a bi-directional connection between the servers WEB01 and WEB02. You can configure various configurations with this command, for example you may like a ‘hub and spoke’ type setup where WEB02, WEB03 etc are all connected to WEB01 but not each other. You might also choose to connect each server with every other server in more of a ‘full mesh’ setup. ie (WEB01 <-> WEB02, WEB01 <-> WEB03 AND WEB02 <-> WEB03).
Add-DfsrConnection -GroupName YourGroup -SourceComputerName WEB01 -DestinationComputerName WEB02 -Confirm

Configure the Primary server’s membership (and set the Staging Quota to 8GB)

  • Set-DfsrMembership -GroupName YourGroup -FolderName WebFarmFiles -ContentPath C:\WebFarmFiles -StagingPathQuotaInMB 8192 -ComputerName WEB01 -PrimaryMember $true -Confirm

Configure the other members

  • Set-DfsrMembership -GroupName YourGroup -FolderName WebFarmFiles -ContentPath C:\WebFarmFiles -StagingPathQuotaInMB 8192 -ComputerName WEB02, WEB03 -Confirm

Associated cmdlets to explore:
Get-DfsrBacklog -GroupName YourGroup -SourceComputerName WEB0X -DestinationComputerName WEB0Y
More here: https://docs.microsoft.com/en-us/powershell/module/dfsr/set-dfsrconnection?view=win10-ps
Get-EventLog -LogName 'DFS Replication' -Newest 20 to check for DFSR errors
Get-EventLog -LogName 'DFS Replication' -Newest 20 | Format-List message for full message details
Restart-Service -Name DFSR this may need to be run on all existing servers after adding a new member to the group

Exporting IIS Configuration for Sharing

On the initial web server that we set up in the last article, you should already have a website and application pool set up. Rather than creating all of those settings again, let’s export the configuration so that we can use it on all of our web servers. Placing this file in our DFS folder will ensure that all of the web servers stay in sync as configuration changes are made.
New-Item -ItemType Directory -Path C:\WebFarmFiles\Configuration
$KeyEncryptionPassword = ConvertTo-SecureString -AsPlainText -String "SecurePa$$w0rd" -Force
Export-IISConfiguration -PhysicalPath "C:\WebFarmFiles\Configuration" -KeyEncryptionPassword $keyEncryptionPassword

You should now have 3 files in the C:\WebFarmFiles\Configuration folder.

Enabling IIS Shared Configuration

Now that our IIS configuration from the initial server has been exported. All of our web servers (including the initial server) need to be set to look at our Shared Configuration files. As DFS is handling the synchronisation of these files between our servers, we can simply point each one to the C:\WebFarmFiles\Configuration folder and they will all be able to read and write changes to the configuration. On each server, run:
$KeyEncryptionPassword = ConvertTo-SecureString -AsPlainText -String "SecurePa$$w0rd" -Force
Enable-IISSharedConfig -PhysicalPath "C:\WebFarmFiles\Configuration\" -KeyEncryptionPassword $KeyEncryptionPassword

And that’s pretty much it. Your web servers should now all be up and running with the same sites that you had configured on the initial server. You can test each one by updating your hosts file to point to the individual IP address of each server and testing in the browser one by one. And obviously the next step from here is to configure a load balancer like HAProxy or NGINX to direct traffic across all of the servers in a fair and reasonable fashion. Stay tuned for the next episode.

Install and configure IIS on Windows Server Core 2016

In a previous post we covered using the System Preparation Tool to convert a VM into a VM Template in XenServer. Once we have used this template to create a new VM, it’s time to set it up as an IIS web server to host some ASP.Net MVC applications.

Revisiting the Basics

Network Settings

When creating a new VM from the template the network settings in the template will also be copied. If it was set to DHCP that will be fine but if the template had a static IP, you should change the IP address to a different one now so that you don’t run into an IP conflict (ie. Two machines on the network using the same IP address).
start powershell
sconfig
– Select 8) Network Settings
– Select the relevant Network Adapter from the list
– Select 1) Set Network Adapter Address
– Enter S for (S)tatics
– Enter the static IP address
– Enter the subnet mask
– Enter the default gateway
– If required select 2) Set DNS Servers

Advanced Networking

In some cases you may need to get a little more fancy with your networking. For example you may need to set your default gateway to a gateway router that can get your traffic out to the Internet, but you have a backend gateway router that handles communication to IP addresses on your private LAN. In this case you can use the route command to tell Windows to send traffic out through different gateway routers.
route print will show current routes, note the current default gateway route (0.0.0.0)
route add 10.0.0.0 mask 255.0.0.0 10.x.x.x -p will send all traffic destined for IP addresses in the 10.0.0.0/8 subnet (ie. Any address starting with ’10.’) out through the 10.x.x.x IP address (backend gateway router). The -p signifies that the route will be persistent and therefore will stick around after a reboot.
route print will now show your new persistent route both in the Active Routes section and below that under Persistent Routes.

Now that you have this route to the private LAN in place, you can change the default gateway address to the ‘Internet’ gateway server without loosing access to your server over the private LAN. This can be done by reconfiguring the network settings again using sconfig or by simply deleting the default route and adding another one.
route delete 0.0.0.0
route add 0.0.0.0 mask 0.0.0.0 10.y.y.y -p will send all traffic destined for an IP that can’t be handled by a more specific route out via the 10.y.y.y router. In this case you would replace the 10.y.y.y with the IP address of your Internet gateway router.

Enable Echo Requests (pings)

This step is optional but if you are going to monitor your server with something like Nagios you probably want to make sure it is online. This will enable the default rule to allow inbound IPv4 pings.
Set-NetFirewallRule -Name FPS-ICMP4-ERQ-In -Enabled True

Checking Internet Access

Many websites rely on web based resources (API’s etc). Now would be a good time to check that your new server has Internet access (unless you are purposely restricting it).
Invoke-WebRequest https://google.com -UseBasicParsing

This will show a big red error if it can’t hit Google, or a 200 status code if it can.

Join an Active Directory Domain

If you need to join your server to a domain to make management easier, follow these steps otherwise continue on to the next section to install IIS.
sconfig
– Select 2) Computer Name
– Set the new computer name and reboot the server
– After the reboot completes, log in again with the Administrator user
sconfig
– Select 1) Domain/WorkGroup
– Type D for (D)omain
– Enter the name of the domain you wish to join and the relevant administrator credentials
– You will be prompted to change the computer name again, click No as we have already done this.
– Click Yes on the Restart prompt
– After rebooting users should be able to login with your domain credentials.

Switching users on the Server Core login screen

If you are using Remote Desktop you should have a normal sign in experience but if you are still looking at the server’s console with just a CMD window on screen, it may not be immediately obvious how to switch users to log in with your domain credentials instead of the default administrator account. Here’s how:
– To change users hit the ESC at the LoginUI.exe screen
– This will present another sign-on options screen, hit ESC again
– Select Other User
– Enter your domain credentials and log in.

Installing the Web Server Role

Powershell comes with some very useful tools for managing the Window Features that are installed on a server
start powershell to open a powershell window
Install-WindowsFeature -Name Web-Server -Confirm will install IIS.
Get-WindowsFeature will show you a list of all available features and show which are installed.

At this point you should have a base install of IIS running the default website on port 80. If you open a browser and type in the IP address of the server you should see the default IIS website.

Install ASP.NET Support

  • Install-WindowsFeature -Name Web-Asp-Net45, Web-Net-Ext45 -Confirm

Installing IIS Diagnostic, Performance and Security Goodies

  • Install-WindowsFeature -Name Web-Custom-Logging, Web-Log-Libraries, Web-Request-Monitor, Web-Http-Tracing -Confirm
  • Install-WindowsFeature -Name Web-Performance -IncludeAllSubFeature -Confirm
  • Install-WindowsFeature -Name Web-Security -IncludeAllSubFeature -Confirm

Installing and Enabling Remote Management for IIS

This will allow us to use the IIS Manager window on another computer to manage our server. Even though we’re installing this now, I won’t be using it to configure the server in the interest of trying to do as much as possible via powershell. The idea is to script all of the server setup so that it can be entirely automated.
Install-WindowsFeature -Name Web-Mgmt-Service -Confirm
Set-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\WebManagement\Server\ -Name EnableRemoteManagement -Value 1
Set-Service -Name WMSvc -StartupType Automatic
Start-Service -Name WMSvc

Note: You will need to install the IIS Manager on the machine that you will be using to manage the server/s. To do this, run:
Install-WindowsFeature -Name Web-Mgmt-Tools -Confirm

Website File Structure

The default directory for storing website files for IIS is C:\inetpub\wwwroot. When configuring your websites though you can put the files wherever you like. To make things simpler if you want to sync your website files between multiple web servers or apply special permissions etc, I find it best to store files in a seperate folder.

To keep things organise when hosting multiple websites across multiple domains I like to organise the content on my IIS servers in the following folder structure:

C:
|_WebFarmFiles
|_Content
|_domain1.com
|_subdomain1
|_project1
|_blah.aspx
|_project2
|_subdomain2
|_domain2.com
|_subdomain1

So if you have a site that will live at the URL http://subdomain1.domain1.com/project1/blah.aspx then blah.aspx would be saved to the C:\WebFarmFiles\Content\domain1.com\subdomain1\project1\ folder.

This setup may look a little confusing at first but it will make sense if/when you need to host multiple sites and quickly find things. Of course your system of organising files may vary and it is, of course, personal preference.

  • New-Item -ItemType Directory C:\WebFarmFiles\Content\domain1.com\subdomain1 this should create all the required parent folders for us automatically.

Getting files onto the server

Create Network Share

  • New-SmbShare -Name WebFarmFiles -Path C:\WebFarmFiles -FullAccess "domain\group1", "domain\group2"
  • Copy files from another machine onto this one using the share \\server\WebFarmFiles.

You could also use robycopy or other utilities to copy files from another network share or download files from github etc.

Set up your first Website

Let’s say we copied some files to \\server\WebFarmFiles\Content\domain1.com\subdomain1 which are intended to be accessed at the URL http://subdomain1.domain1.com. Let’s also say that we want this website to run in it’s own Application Pool so that we can manage it’s resource usage easily rather than everything running in the DefaultAppPool

Create the IIS Application Pool

  • New-WebAppPool -Name subdomain1.domain1.com

Associated cmdlets to explore:
Remove-WebAppPool
Get-WebAppPoolState | Select *
Restart-WebAppPool -Name subdomain1.domain1.com

Change the App Pool Identity

In some cases, the process running your application may need to access files on the network with specific user permissions.
Set-ItemProperty IIS:\AppPools\app-pool-name -name processModel -value @{userName="domain\user";password="password";identitytype=3}

Set the App Pool startMode

If your application is a big one, you may wish to set it to AlwaysRunning so that the first visitor doesn’t have to wait for it to initialise:
Set-ItemProperty IIS:\AppPools\app-pool-name -Name startMode -Value AlwaysRunning
Get-ItemProperty IIS:\AppPools\app-pool-name -Name startMode to check the setting.

Create the IIS WebSite

  • New-Website -Name subdomain1.domain1.com -ApplicationPool subdomain1.domain1.com -HostHeader subdomain1.domain1.com -PhysicalPath C:\WebFarmFiles\Content\domain1.com\subdomain1\

Associated cmdlets to explore:
Get-Website
Remove-WebSite -Name subdomain1.domain1.com
Stop-Website -Name subdomain1.domain1.com
Start-Website -Name subdomain1.domain1.com

The new website should now be running and you can access it by pointing the subdomain1.domain1.com URL at your servers IP address either just from your local machine by modifying your hosts file or by modifying the DNS records for the domain1.com domain. These methods are not covered in this article.

Adding an additional binding

In some cases you may have a need to point two different URL’s at the same website.

  • New-Binding -Name subdomain1.domain1.com -HostHeader subdomain1.domain3.com

In this case, the ‘Name’ of the binding relates to the WebSite it will be linked to.

Associated cmdlets to explore:
Get-WebBinding
Get-WebBinding | Select-Object * for a more advanced view
Remove-WebBinding -HostHeader subdomain1.domain3.com

ASPState Database on SQL AlwaysOn Availability Group

Running your ASPState database in a SQL AlwaysOn Availability Group provides redundancy in case there is a SQL server failure.

Unfortunately the default process for setting up the ASPState database does not take this configuration into account and you may find that your ASPState database has an ever expanding data file that is using all of your disk space.

When the ASPState database is created, there is also a SQL Agent job that is also created which deletes expired sessions to keep the database size at a reasonable level. This job is usually called:

ASPState_Job_DeleteExpiredSessions

In the usual setup, this job runs every minute and simply fires of a Store Procedure which lives in the ASPState database itself using the following T-SQL statement:

EXECUTE DeleteExpiredSessions

Generally this all works well. The issue arises when the Availability Group is failed over to a secondary availability replica. Because the SQL Agent job is not present on this server, the expired sessions are no longer deleted and the database begins to grow.

You may notice on the SQL server that was originally the primary availability replica that there are errors in the SQL Agent job history similar to this:

Date 24/04/2018 11:45:00 AM
Log Job History (ASPState_Job_DeleteExpiredSessions)

Step ID 1
Server [serverName]
Job Name ASPState_Job_DeleteExpiredSessions
Step Name ASPState_JobStep_DeleteExpiredSessions
Duration 00:00:08
Sql Severity 16
Sql Message ID 3906
Operator Emailed
Operator Net sent
Operator Paged
Retries Attempted 0

Message
Executed as user: [domain\user]. Failed to update database "ASPState" because the database is read-only. [SQLSTATE 25000] (Error 3906). The step failed.

This is because the database the job is trying to run against has now become the read-only secondary replica database. If you are quickly running out of disk space and need to fix this problem, then log on to the primary replica and simply run the following:

USE [ASPState]
GO

EXECUTE DeleteExpiredSessions

The next logical step would be to simply set up a matching SQL Agent Job on the all servers that will delete the old sessions regardless of which server is the primary replica at the time. This is definitely the next step, however this will continue to generate errors on all the servers that are running as secondary replicas.

Fortunately though, SQL Server provides a better long term solution that will resolve the issue and save us writing out unnecessary errors. All we need to do is update the T-SQL statement that the Agent jobs run (on each of the replica servers) to look like the following:

IF sys.fn_hadr_is_primary_replica('ASPState') = 1
BEGIN
EXECUTE DeleteExpiredSessions
END
ELSE
BEGIN
SELECT 'This server is not the primary replica for ASPState at this time, skipping maintenance.'
END

This will check whether the server that the job is running on is the primary replica for the ASPState database and only run the maintenance stored procedure if it is.

Here’s what it would look like:

aspstate agent job
ASPState SQL Agent Job