Setting NTP on Ubuntu servers in AWS with ansible

A simple, but often overlooked task when running a bunch of servers is making sure they’re all time-synced. Thankfully AWS makes sure a reliable time source is always accessible from your servers. The documentation from AWS is linked below.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html

Here’s a simple ansible playbook to get you started:

- hosts: all
  vars:
  remote_user: youruser
  become: true
  become_user: root
  become_method: sudo
  become_flags: '-i'
  tasks:
  - name: install chrony
    apt:
            name: chrony
            state: latest
  - name: set chrony ntp config
    template:
            src: /etc/ansible/resources/chrony.conf
            dest: /etc/chrony/chrony.conf
            owner: root
            group: root
            mode: 0644
  - name: enable chrony             
    systemd:
            name: chrony.service
            state: started
            enabled: yes

Make sure you replace the source of the chrony.conf file with the appropriate file. My chrony.conf is linked below. The link-local address should be included to take advantage of AWS’s time servers.

server 169.254.169.123 prefer iburst minpoll 4 maxpoll 4
pool ntp.ubuntu.com        iburst maxsources 4
keyfile /etc/chrony/chrony.keys
driftfile /var/lib/chrony/chrony.drift
#log tracking measurements statistics
logdir /var/log/chrony
maxupdateskew 100.0
rtcsync
makestep 1 3

This should result in your servers having the correct time, assuming you set the time-zone correctly.

Good luck!

Banning IP addresses using ufw

It’s been a while!

ufw is a popular frontend for iptables. Sometimes you just really need to get rid of an IP address that’s knocking on the doors for no good reason. This is where the pyban script comes in handy.

It’s a simple little python script that just bans incoming connections from the IP address in question.

#!/usr/bin/python3

import socket
import sys
import os


if len(sys.argv) != 2:
        print("Invalid number of Arguments")
        sys.exit(0)

def valid_ipv4(address):
        try:
                socket.inet_aton(address)
                return 1
        except socket.error:
#               print("Invalid IP Address")
                return 0

def valid_ipv6(address):
        try:
                socket.inet_pton(socket.AF_INET6, address)
                return 1
        except socket.error:
#               print("Not IPv6 either!")
                return 0

commandstring = "ufw insert 1 deny from " + sys.argv[1] + " comment pyban"

if valid_ipv4(sys.argv[1]) == 1:
        print("Banning IP address: " + sys.argv[1])
        os.system(commandstring)
        sys.exit(0)
elif valid_ipv6(sys.argv[1]) == 1:
        print("Banning IPv6 address: " + sys.argv[1])
        os.system(commandstring)
        sys.exit(0)
else:
        print("Please supply a valid IP address")
        sys.exit(0)

Save the script to to a file called “pyban” and make it executable.

using it is quite simple:

pyban 192.0.2.20

Hope this is of some use to you!

A simple tool for checking TCP connectivity

This is a short one.

One of the most common things I do is to simply check if something works. The quicker I can find out if it works or not, the better. What this little program does can be easily accomplished with various other tools, such as telnet, test-netconnection (a powershell cmdlet), nmap, psping and the list goes on.

Yet, this little tool has become a bread and butter tool for me, the reason mostly being simplicity. Telnet is no longer installed by default on windows servers. test-netconnection only works in powershell and not in a cmd terminal. Nmap and psping both require you to know the syntax, as simple as they may be.

For some reason, I’ve stuck with using portcheck.exe

The tool is extremely stripped down, there are no flags, no bells and whistles, it will check TCP connectivity to a host on a port, and it will do nothing else. It’s this simplicity that’s made it useful to me personally, so I’ve decided to share it.

https://palmar.org/software/portcheck.exe

The syntax is extremely simple:

> portcheck.exe example.org 80

Successfully connected to server.

> portcheck.exe example.org 21

The connection attempt timed out.

I always keep a handy c:\tools folder on my windows machines, and I add it to the path of the machine, so I can tab-complete and call the programs in there from wherever I am. Another option is sticking the portcheck.exe file in a directory that is included in the path by default, such as c:\windows. I highly recommend the tools approach though. Here’s an older article on how to add stuff to your path quickly.

The source code for the portcheck tool is available on github. Break it in any way you see fit! I hope this tool helps someone.

Nginx SSL configuration

As a disclaimer, this blog isn’t really something a lot of people read, but for the few visitors that pop by, I have moved on to a network engineering job from my previous systems administration job, so I might focus less on powershell than previously, although of course it’ll remain a part of what I do.

So I’ve been toying around with letsencrypt to secure my personal homepage palmar.org. The service works quite well. I decided to see if I could secure my web server properly using the letsencrypt certificate, and it was actually relatively simple.

There’s just a few things to take care of when you’ve installed the certificate (which I’m not going to cover).

First, you need to point your ssl listener to the certificate and it’s private key:

 ssl on;
 ssl_certificate /etc/ssl/palmar.org/fullchain.pem;
 ssl_certificate_key /etc/ssl/palmar.org/privkey.pem;

Second, you have to configure the preferred encryption algorithms the server presents to the client. This list is an ever changing beast as vulnerabilities emerge, so what works today (5. december 2015) might not be relevant in a year or two. We’ll make sure our most preferred algorithms support perfect forward secrecy and are secure.

Also, it’s a good idea to disable SSLv3 and older protocols.

 ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

 ssl_ciphers 'ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4';

Third, it’s not enough to just prefer secure algorithms, we actually have to secure our Diffie-Hellman key exchange. To make a long story short, DH works by the client and the server exchanging prime numbers in order to obtain perfect forward secrecy. The larger the prime number group used in generating the keys, the more difficult it is to break. Commonly DH groups would be 512 or 1024 bits, but we’ll generate a 2048 bit DH group to completely secure our server. We’ll do this using openssl.

openssl dhparam -out dhparams.pem 2048

If you want to read more on this subject, here’s an excellent resource: https://weakdh.org/

We’ll then point to our newly created DH group in the server configuration:

 ssl_dhparam /etc/ssl/DH/dhparams.pem;

Finally we’re going to enable HTTP Strict Transport Security with a reasonably long duration.

 add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";

And voila, we’re done!

Looks good, doesn’t it? 🙂

Here’s the entire configuration:

 

server {
 listen 443;
 server_name palmar.org;

 root /var/www/html;
 index index.html index.htm;

 ssl on;
 ssl_certificate /etc/ssl/palmar.org/fullchain.pem;
 ssl_certificate_key /etc/ssl/palmar.org/privkey.pem;

 ssl_session_timeout 5m;

 ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

 ssl_ciphers 'ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4';
 ssl_dhparam /etc/ssl/DH/dhparams.pem;
 ssl_prefer_server_ciphers on;

 add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";
}

 

Copying a file from a remote session using Powershell

One of the challenges of using powershell remoting between domains is that while you can certainly set up remoting, there is no Powershell way of getting files from this remote session. In order to tackle this problem, mostly because sometimes I just want to work with things locally, or even back them up locally, I wrote a fairly short and simple script to copy a file back to my local workstation.

I should probably use this opportunity to point out that if you’re using the ISE environment to open sessions to your remote computers, you can, as of WMF5, simply punch “Psedit yourfile.txt” into the scripting pane and it will open the remote file in your ISE environment. You can get the latest WMF preview here.

Anyway, sometimes the goal isn’t just to edit a remote file, but for some reason to bring it from the remote machine to your local machine. And for that I created the get-sessionfile.ps1 script.

param
(
    [Parameter(Mandatory = $true)][string]$computer,
    [Parameter(Mandatory = $true)][string]$remotepath,  
    [Parameter(Mandatory = $true)][string]$localpath,
    [Parameter(Mandatory = $true)][pscredential]$creds
)

$session = New-PSSession $computer -UseSSL -Credential $creds
$remotecontent = Invoke-Command -Session $session {Get-Content $args[0]} -args $remotepath
set-content $localpath $remotecontent -Force
Remove-PSSession $session

The script itself is very little magic. You provide the require parameters. The script assumes you’re doing this from one domain to another with no trust between them, because if you’re working inside a domain, you could usually just use UNC paths to copy things. If you want to use this without SSL and alternative credentials, just delete the mandatory Parameter pscredential, and remove the two flags from the New-PSSession cmdlet.

What this does is essentially read the contents of whatever file you want to copy, and write those exact contents to a file created locally, essentially “copying” the file, although this will not of course maintain meta-data about the file, such as the last write time.

There are a ton of ways of improving the script, the most glaringly obvious one is there is currently no way to send the file back, but this should at least help you get going, and solve exactly one problem.

Enabling TLS 1.2 using powershell.

This is going to be a quick and dirty post. Ideally I should probably wrap these in a function and take a parameter with just the servername for running the script, but this is what I did when I enabled TLS 1.2 for my environment, so this is what I’m posting. The intent here is to quickly allow you to start using TLS 1.2 with minimal configuration.

md "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2"
md "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server"
md "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client"

new-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server" -name "Enabled" -value 1 -PropertyType "DWord"
new-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server" -name "DisabledByDefault" -value 0 -PropertyType "DWord"
new-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client" -name "Enabled" -value 1 -PropertyType "DWord"
new-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client" -name "DisabledByDefault" -value 0 -PropertyType "DWord"

Set-ItemProperty -path "HKLM:\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002" -name "Functions" -value "TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384,TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_RC4_128_MD5,SSL_CK_RC4_128_WITH_MD5,SSL_CK_DES_192_EDE3_CBC_WITH_MD"

This script does a few things. It creates the necessary folders in the registry and adds the keys. Finally the script reconfigures the priority order of cipher suites used in your ssl handshake. If you want to support Forward Secrecy, which I highly recommend, simply replace the last line of the script with this one:

 

Set-ItemProperty -path "HKLM:\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002" -name "Functions" -value "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384,TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384,TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_RC4_128_MD5,SSL_CK_RC4_128_WITH_MD5,SSL_CK_DES_192_EDE3_CBC_WITH_MD"

To enable the script for a particular server you can then do the following

Invoke-Command -ComputerName servername -Filepath script.ps1

Where servername is the name of your server and script.ps1 is the script above saved to a powershell script file. Alternatively if you have a list of your servers in a text file, which seems to be common practice, you can do the following:

Invoke-Command -ComputerName (cat serverlist.txt) -Filepath script.ps1

Powershell remoting using SSL and credSSP

There is a lot of excellent articles on the internet about Powershell remoting, and I can only assume interest in the feature is going to increase as windows administrators move to automating as much of their work as possible.

While I may have missed it, though, I don’t think I ever saw a good breakdown in a single article on the steps required to get your environment ready to handle ssl encrypted credSSP capable sessions.

I am, as of writing this article, working on a module that simplifies this entire process, basically doing it for you, but it’s not entirely production ready, and I think understanding the steps required is interesting anyway. These are the steps I’m going to take to reach my goal of secure credSSP capable sessions.

  1. Enable WinRM on the server, configure authentication and set up a HTTPS listener
  2. Allow incoming connections on 5986 (the default WinRM https port)
  3. Add a certificate to the WinRM service
  4. Add a certificate to the WinRM listener
  5. Enable credSSP role for the server
  6. Enable credSSP role for the client

Powershell remoting runs on top of winrm, and to my best knowledge, not every winrm command has been implemented as a powershell CmdLet, however things start off nicely with the Set-WSManQuickConfig CmdLet. The first few steps of this guide must be run on the server.

Set-WSManQuickConfig -UseSSL

WinRM Quick Configuration
Running the Set-WSManQuickConfig command has significant security implications, as it enables remote management through
 the WinRM service on this computer.
This command:
 1. Checks whether the WinRM service is running. If the WinRM service is not running, the service is started.
 2. Sets the WinRM service startup type to automatic.
 3. Creates a listener to accept requests on any IP address. By default, the transport is HTTP.
 4. Enables a firewall exception for WS-Management traffic.
 5. Enables Kerberos and Negotiate service authentication.
Do you want to enable remote management through the WinRM service on this computer?
[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"):

This takes care of the first two steps for us, WinRM is now up and running on the server, and we don’t have to toy around with firewall rules as the CmdLet takes care of creating one for us.

The next step is to add the required certificates. By default Powershell will expect a certificate whose name matches the server’s name, but this is not necessary. It is entirely up to you whether you create a server specific certificate, or just a generic one to distribute to servers on your domain.

Very important detail of adding your certificate to the WinRM service, is that it must both be present in the WinRM service configuration, and in the HTTPS listener configuration. Another important detail is that the default system account NETWORK SERVICE must have read access to the certificate’s private key.

I have created a simple function that should take care of all this for you, with only the CN of the certificate provided.

 
function Add-SSLCredential($CN)
{
    #First thing is to locate the certificate by CN. Make sure we get the most recent one.
    try
    {
        $certificate = Get-ChildItem CERT:\LocalMachine\My | Where-Object {$_.Subject -match $CN} | sort $_.NotAfter -Descending | select -first 1 -erroraction STOP
        $thumbprint = $certificate.Thumbprint
        $UKCN = $certificate.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName
    }
    catch
    {
        Write-Output "Error: cannot find the certificate"
    }

    #Then we check if the HTTPS listener exists, it probably does if you're following this post
    #That's okay though, we delete it and set up a new listener with our Certificate.
    $checkconfig = winrm e winrm/config/listener
    if($checkconfig -contains "    Transport = HTTPS")
    {
        Write-Host -ForegroundColor Yellow "1. Delete old config"
        winrm delete winrm/config/Listener?Address=*+Transport=HTTPS
    }

    Write-Host -ForegroundColor Yellow "2. Add a certificate to the listener"
    winrm create winrm/config/listener?Address=*+Transport=HTTPS `@`{Hostname=`"$CN`"`; CertificateThumbprint=`"$thumbprint`"`}

    #Then we add the same certificate to the winrm service
    Write-Host -ForegroundColor Yellow "3. Add certificate to the winrm service"
    winrm set winrm/config/service `@`{CertificateThumbprint=`"$thumbprint`"`}

    #And finally, we make sure the NETWORK SERVICE account has access to the private key of the certificate
    Write-Host -ForegroundColor Yellow "4. Allow the Network Service access to the certificate"

    $machinekyepath = "$env:SystemDrive\ProgramData\Microsoft\Crypto\RSA\MachineKeys\"
    $pathtoactualkey = $machinekyepath+$UKCN
    
    $acl = Get-Acl -Path $pathtoactualkey
    $permission="NETWORK SERVICE","Read","Allow"
    $accessRule = New-Object System.Security.AccessControl.FileSystemAccessRule $permission
    $acl.AddAccessRule($accessRule)
    Try
    {
        Set-Acl $pathtoactualkey $acl
    }
    Catch
    {
        Write-Output "Error: unable to set ACL on certificate"
    }
}

Finally, you have to allow the server to receive credentials through credSSP authentication. This is quite simple.

Enable-WSManCredSSP -Role Server

Now we’re all done with the server. The client configuration is much more simple. All we have to do is make sure we allow our credentials to be delegated to the server.

Enable-WSManCredSSP -Role Client -DelegateComputer you.server.here

Note that you can add wildcards too, so you could specify the -DelegateComputer to be *.domain.local if your domain is called domain.local and thus enable credSSP delegation to all computers in the domain.

When the configuration is all done there’s nothing stopping us from entering a new and improved PSSession

Enter-PSSession server.domain.local -UseSSL -Cred $credential -authentication CredSSP

And as promised, if the certificate name doesn’t match the server’s name, or in case you’re using a self-signed certificate, all you have to do is append -SessionOption(New-PSSessionOption -skipcnckeck -skipcacheck) to the command above. The flags will skip the certificate name check and the certificate authority check respectively.

I hope this post isn’t too long and can be useful for someone wanting to configure their environment to use secure credSSP enabled sessions.

Manipulating your PATH variable using Powershell

The PATH variable in windows is something you occasionally have to touch.

The classic command for manipulating PATH is simply

set PATH=%PATH%;x:\your\folder\here

Now this does the trick, and you don’t have to do a lot of work to get powershell to do the same thing, the command is simply

$env:Path = $env:Path+";x:\your\folder\here"

The only problem is that the $env:Path variable is only a temporary variable that is available in your current session. This won’t do if you want to make your changes persistent. In addition, I found making the PATH variable a bit more readable helped when dealing with it.

What we will be doing is leveraging the setx command to make the changes permanent.

 

function get-path
{
    $env:Path -split (";")
}

function add-path($folder)
{
    $addtopath = ";"+$folder
    $env:Path = $env:Path + $addtopath
    $setpath = "setx Path ""$env:Path"" -m"
    Invoke-Expression $setpath
}

function remove-path($folder)
{
    $removepath = ";"+$folder
    $env:Path = $env:Path.Replace($removepath,$NULL)
    $setpath = "setx Path ""$env:Path"" -m"
    Invoke-Expression  $setpath
}

This is the simplest way I could come up with for editing the path variable through powershell and making the changes permanent. This module will throw an error if you don’t have administrative privileges, as those are required for editing the registry, however even in a non-elevated session it will still change your local path variable for that session.

get-path simply displays the path variable in your current session in a comfortable way.

There is a more elegant way of doing this, but I like the simplicity of this one. If you’re looking for a pure powershell implementation though, here’s a link to a great article on Technet

http://blogs.technet.com/b/heyscriptingguy/archive/2011/07/23/use-powershell-to-modify-your-environmental-path.aspx