Backup Windows Server on EC2 with PowerShell and AWS CLI

When we first started working with Windows Server or Amazon EC2, we ran into the problem of how to back up SQL Server databases. My initial thought was to somehow mount S3 as a volume and have SQL Server write backup jobs to it. That turns out to be kind of hard.

After a little head-scratching, I realized that EBS volume snapshots are stored in S3. Therefore, all we need to do is mount an EBS volume and have SQL Server write backup jobs to that and then snapshot that volume. Oh, and unmount it from the running Windows server before snapshot to make sure that we are taking clean snapshots.

It turns out to not be too hard to make this happen on Windows Server 2012 Core using a PowerShell script that drives the AWS CLI and does the hoodoo with the Windows volume manager to mount and dismount the EBS backup volume.

This solution requires PowerShell Storage Cmdlets which ship with Windows Sever 2012/ and Amazon Web Services python-based command-line interface. Works great on Windows Server Core.

The biggest problem that I ran into was that the script would run just fine when invoked manually or when I would trigger the job from schtasks while logged in, but it would hang and fail when run from scheduled tasks in the middle of the night. Long story short: when schtasks runs the job with nobody logged in it gives the .DEFAULT environment instead of the environment of the user context of the scheduled task. That meant that AWSCLI didn’t receive the correct %USERPROFILE% environmental variable and was not able to locate its config file with the user id and key.

The simplest solution was to wrap the invocation of the powershell script in a cmd batch script:

set USERPROFILE=C:\Users\Administrator\

powershell.exe -file “C:\Program Files\Invoke-BackupJob.ps1” -path E:\ -diskNumber 2 -ec2VolumeID vol-<id-number-here> -description “Important Backup” > “C:\Program Files\Utility\Log.txt”

Use the PowerShell Get-Disk cmdlet to figure out the disk number of the EBS volume in Windows and the EC2 console or AWSCLI to figure out the EBS volume ID in EC2.

[gist https://gist.github.com/breiter/7887782]

Cool PowerShell Script Replicates Telnet

Lee Holmes has a cool script to reproduce telnet-like functionality via the TcpClient object in PowerShell.

## Connect-Computer.ps1 
## Interact with a service on a remote TCP port 
param( 
    [string] $remoteHost = "localhost", 
    [int] $port = 23
     ) 

try
{
    ## Open the socket, and connect to the computer on the specified port 
    write-host "Connecting to $remoteHost on port $port" 
    $socket = new-object System.Net.Sockets.TcpClient($remoteHost, $port) 
    if($socket -eq $null) { return; } 

    $stream = $socket.GetStream() 
    $writer = new-object System.IO.StreamWriter($stream) 

    $buffer = new-object System.Byte[] 1024 
    $encoding = new-object System.Text.AsciiEncoding 

    while($true) 
    { 
       ## Allow data to buffer for a bit 
       start-sleep -m 500 

       ## Read all the data available from the stream, writing it to the 
       ## screen when done. 
       while($stream.DataAvailable)  
       {  
          $read = $stream.Read($buffer, 0, 1024)    
          write-host -n ($encoding.GetString($buffer, 0, $read))  
       } 

       ## Read the user's command, quitting if they hit ^D 
       $command = read-host 
      
       ## Write their command to the remote host      
       $writer.WriteLine($command) 
       $writer.Flush() 
    } 
}
finally
{
    ## Close the streams 
    $writer.Close() 
    $stream.Close()
}

This solves the problem of telnet.exe crashing conhost.exe when using Console2.

Update

I tweaked Lee’s code to use try/finally which obviates the need for a special escape sequence to clean up the TCP resources.

WorkAround: Console2 + Telnet.exe on Windows 7 Crashes ConHost.exe

conshost-croak

Entering the Windows telnet.exe prompt mode causes the shell process underneath Console2 to crash on Windows 7. You can reproduce this by just invoking telnet with no arguments in a Console2 session. An alternate route to hit this is exiting a telnet session with CTRL+].

The symptom is Console2 freezing because it is no longer getting text from its slaved native console followed up with a crash dialog for conhost.exe and whetever your shell was.

Unhandled exception at 0xffdd1df9 in conhost.exe: 0xC0000005: Access violation reading location 0x0000000003385460.>    conhost.exe!SB_StreamWriteToScreenBuffer()  + 0x61 bytes   

A workaround is to launch telnet into a separate conhost.exe window.

#force telnet.exe to start in its own console window. 
function telnet { start $env:SystemRoot\System32\telnet.exe $args }

Or use putty –P port –telnet host as an alternative to telnet.exe. (I can’t get plink.exe to be interactive with telnet for me.)

Update

Unless you need something nifty that telnet.exe is doing, a more elegant solution is to Lee Holmes’ Connect-Computer.ps1 PowerShell script as a telnet replacement.

FIX: VirtualBox Host-Only Network Adapter Creates a Virtual “Public Network” Connection That Causes Windows to Disable Services

vbox-host-only-net-net-n-sharing

vbox-host-only-net-connectletVirtualBox creates a “VirtualBox Host-Only Network” device which is essentially a loopback adapter for creating network connections between virtual machines and between the host and virutal machines. Unfortunately, it shows up to Windows as an unidentified public network. Connecting to a “public network” ratchets up your firewall and disables network discovery and SMB/CIFS network shares. This is kind of a big side effect of installing some VM software.Fortunately, Windows does have a way to mark a network device as virtual by creating a registry value.

The type of the device. The default value is zero, which indicates a standard networking device that connects to a network. Set *NdisDeviceType to NDIS_DEVICE_TYPE_ENDPOINT (1) if this device is an endpoint device and is not a true network interface that connects to a network. For example, you must specify NDIS_DEVICE_TYPE_ENDPOINT for devices such as smart phones that use a networking infrastructure to communicate to the local computer system but do not provide connectivity to an external network.

This powerhsell script will find the “VirtualBox Host-Only Ethernet Adapter device entry in the registry and adds the ‘*NdisDeviceType’ value of 1. After reboot, the “VirtualBox Host-Only Ethernet Adapter” will no longer be monitored by the Network and Sharing center.

# tell windows that VirtualBox Host-Only Network Adapter
# is not a true network interface that connects to a network
# see http://msdn.microsoft.com/en-us/library/ff557037(VS.85).aspx
pushd
echo 'Marking VirtualBox Host-Only Network Adapter as a virtual device.'
cd 'HKLM:\system\CurrentControlSet\control\class\{4D36E972-E325-11CE-BFC1-08002BE10318}'
ls ???? | where { ($_ | get-itemproperty -name driverdesc).driverdesc `
-eq 'VirtualBox Host-Only Ethernet Adapter' } |`
new-itemproperty -name '*NdisDeviceType' -PropertyType dword -value 1
echo 'After you reboot the VirtualBox Host-Only Network unidentified public network should be gone.'
popd


Copy and Paste with Clipboard from PowerShell

Many times I want to use PowerShell to manipulate some text that I have in something else like email or a text editor or some such or conversely, I do something at the command line and I want to paste the output into a document of some kind.

I’m not sure why there is no cmdlet for outputting to clipboard.

PS> get-command out-*

CommandType     Name             Definition                                         
-----------     ----             ----------                                         
Cmdlet          Out-Default      Out-Default [-InputObject <PSObject>] [-Verbose]...
Cmdlet          Out-File         Out-File [-FilePath] <String> [[-Encoding] <Stri...
Cmdlet          Out-GridView     Out-GridView [-InputObject <PSObject>] [-Title <...
Cmdlet          Out-Host         Out-Host [-Paging] [-InputObject <PSObject>] [-V...
Cmdlet          Out-Null         Out-Null [-InputObject <PSObject>] [-Verbose] [-...
Cmdlet          Out-Printer      Out-Printer [[-Name] <String>] [-InputObject <PS...
Cmdlet          Out-String       Out-String [-Stream] [-Width <Int32>] [-InputObj...

Fortunately, though Windows ships with a native clip.exe for piping text to clipboard. You can alias it to out-clipboard or just pipe to “clip”.

new-alias  Out-Clipboard $env:SystemRoot\system32\clip.exe

Now you can just pipe to Out-Clipboard or clip just like the other Out-* cmdlets.

The larger issue is that PowerShell doesn’t expose a way to paste text from the clipboard. However, the WinForms API of the .NET Framework has a GetText() static method on the System.Windows.Forms.Clipboard class. There is a catch, though:

The Clipboard class can only be used in threads set to single thread apartment (STA) mode. To use this class, ensure that your Main method is marked with the STAThreadAttribute attribute.

This is unfortunate because PowerShell runs in a multithreaded apartment by default which means that the workaround is to spin up a new PowerShell process which is slow.

function Get-ClipboardText()
{
	$command =
	{
	    add-type -an system.windows.forms
	    [System.Windows.Forms.Clipboard]::GetText()
	}
	powershell -sta -noprofile -command $command
}

A slightly less obvious approach is to paste the text from the clipboard into a non-visual instance of the System.Windows.Forms.Textbox control and then emit the text of the control to the console. This works fine in a standard PowerShell process and is substantially faster than spinning up a new process.

function Get-ClipboardText()
{
	Add-Type -AssemblyName System.Windows.Forms
	$tb = New-Object System.Windows.Forms.TextBox
	$tb.Multiline = $true
	$tb.Paste()
	$tb.Text
}

The TextBox method is about 500 times faster than the sub-process method. On my system, it takes 0.501 seconds to do spin up the sub-process and emit the text from the clipboard versus 0.001 seconds to use the TextBox control in the current process.

I also alias this function to “paste”. The commands I actually type and remember are “clip” and “paste”.

When using paste, I generally am setting a variable or inserting my pasted text into a pipeline within parens.

CAVEAT: Whenever you capture a value without any {CR}{LF}, the clipboard will add it. You may need to trim the newline to get the behavior you expect when pasting into a pipeline.

Here’s some one-liners to illustrate. I have dig from BIND in my path and I have Select-String aliased to grep. Notice how when I paste the IP address I captured to the clipboard from the one-liner gets pasted back with an extra blank line.

PS> ((dig google.com) | grep '^google\.com') -match '(\d+\.\d+\.\d+\.\d+)$' | out-null; $matches[0]
72.14.235.104
PS> ((dig google.com) | grep '^google\.com') -match '(\d+\.\d+\.\d+\.\d+)$' | out-null; $matches[0] | clip
PS> paste
72.14.235.104

PS> ping (paste)
Ping request could not find host 72.14.235.104
. Please check the name and try again.
PS> ping (paste).trim()

Pinging 72.14.235.104 with 32 bytes of data:
Reply from 72.14.235.104: bytes=32 time=192ms TTL=51
Reply from 72.14.235.104: bytes=32 time=208ms TTL=51
Reply from 72.14.235.104: bytes=32 time=212ms TTL=51
Reply from 72.14.235.104: bytes=32 time=210ms TTL=51

Ping statistics for 72.14.235.104:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 192ms, Maximum = 212ms, Average = 205ms

Command Prompt and PowerShell Here with Console.exe

command-hereIn antediluvian turn of the century days, there was an Open Command Window Here PowerToy. When Vista was released, this functionality was baked in to the default Explorer shell. Hold {SHIFT} and right-click on a drive, directory or the background of a directory and you will have an “Open command window here” option in the resulting context menu. Although PowerShell is installed by default on Windows 7, there is no parallel “Open PowerShell window here” option, but it can be easily added.

“Open PowerShell window here”

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\powershell]
@="Open PowerShell window here"
"Extended"=""
[HKEY_CLASSES_ROOT\Directory\shell\powershell\command]
@="C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe -NoExit -Command Set-Location -LiteralPath '%V'"

[HKEY_CLASSES_ROOT\Directory\Background\shell\powershell]
@="Open PowerShell window here"
"Extended"=""
[HKEY_CLASSES_ROOT\Directory\Background\shell\powershell\command]
@="C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe -NoExit -Command Set-Location -LiteralPath '%V'"

[HKEY_CLASSES_ROOT\Drive\shell\powershell]
@="Open PowerShell window here"
"Extended"=""
[HKEY_CLASSES_ROOT\Drive\shell\powershell\command]
@="C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe -NoExit -Command Set-Location -LiteralPath '%V'"

Use Console.exe Windows

If you use command-line a lot in Windows, you will appreciate the advantages of using console.sf.net as the terminal window. The “Open … window here” context menus can be tweaked to open the new command prompts in Console rather than vanilla Windows CSRSS console windows.

"Open command window here” with Console

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\cmd]
@="@shell32.dll,-8506"
"Extended"=""
"NoWorkingDirectory"=""
[HKEY_CLASSES_ROOT\Directory\shell\cmd\command]
@="\"C:\\Program Files\\Console2\\Console.exe\" -r cmd.exe -d \"%V\" -w \"Command Prompt\""

[HKEY_CLASSES_ROOT\Directory\Background\shell\cmd]
@="@shell32.dll,-8506"
"Extended"=""
"NoWorkingDirectory"=""
[HKEY_CLASSES_ROOT\Directory\Background\shell\cmd\command]
@="\"C:\\Program Files\\Console2\\Console.exe\" -r cmd.exe -d \"%V\" -w \"Command Prompt\""

[HKEY_CLASSES_ROOT\Drive\shell\cmd]
@="@shell32.dll,-8506"
"Extended"=""
"NoWorkingDirectory"=""
[HKEY_CLASSES_ROOT\Drive\shell\cmd\command]
@="\"C:\\Program Files\\Console2\\Console.exe\" -r cmd.exe -d \"%V\" -w \"Command Prompt\""

“Open PowerShell window here” with Console

 

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\powershell]
@="Open PowerShell window here"
"Extended"=""
[HKEY_CLASSES_ROOT\Directory\shell\powershell\command]
@="\"C:\\Program Files\\Console2\\Console.exe\" -r C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe -d \"%V\""

[HKEY_CLASSES_ROOT\Directory\Background\shell\powershell]
@="Open PowerShell window here"
"Extended"=""
[HKEY_CLASSES_ROOT\Directory\Background\shell\powershell\command]
@="\"C:\\Program Files\\Console2\\Console.exe\" -r C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe -d \"%V\""

[HKEY_CLASSES_ROOT\Drive\shell\powershell]
@="Open PowerShell window here"
"Extended"=""
[HKEY_CLASSES_ROOT\Drive\shell\powershell\command]
@="\"C:\\Program Files\\Console2\\Console.exe\" -r C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe -d \"%V\""

 

 

Restore Defaults

For PowerShell, remove the powershell keys from beneath HKCR\Directory\shell, HKCR\Directory\Background\shell and HKCR\Drive\shell.

“Open command window here” restore default

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\cmd]
@="@shell32.dll,-8506"
"Extended"=""
"NoWorkingDirectory"=""
[HKEY_CLASSES_ROOT\Directory\shell\cmd\command]
@="cmd.exe /s /k pushd \"%V\""

[HKEY_CLASSES_ROOT\Directory\Background\shell\cmd]
@="@shell32.dll,-8506"
"Extended"=""
"NoWorkingDirectory"=""
[HKEY_CLASSES_ROOT\Directory\Background\shell\cmd\command]
@="cmd.exe /s /k pushd \"%V\""

[HKEY_CLASSES_ROOT\Drive\shell\cmd]
@="@shell32.dll,-8506"
"Extended"=""
"NoWorkingDirectory"=""
[HKEY_CLASSES_ROOT\Drive\shell\cmd\command]
@="cmd.exe /s /k pushd \"%V\""

MSE v2 Command-Line Scanning

Microsoft Security Essentials (MSE) 2.0 adds command-line file scanning to its command-line interface.

MpCmdRun.exe is the command-line interface to MSE.

(Note that MSE has moved from “C:\Program Files\Microsoft Security Essentials” to “C:\Program Files\Microsoft Security Client”. And MpCmdRun.exe has moved to a subdirectory called Antimalware.)

In MSE v1, MpCmdRun was conspicuously missing an option to scan a file from a command line but the problem is rectified in MSE v2 beta.

mpcmdrun-filescan 

The file scanning is a new sub-option of the –Scan argument.

-Scan [-ScanType value]
    0  Default, according to your configuration
    1  Quick scan
    2  Full system scan
    3  Single file custom scan

       [-File ]
            Indicates the file path to be scanned, only valid for custom scan

       [-DisableRemediation]
            This option is valid only for custom scan.
            When specified:
              - File exclusions are ignored.
              - Archive files are scanned.
              - Actions are not applied after detection.
              - Event log entries are not written after detection.
              - Detections from the custom scan are not displayed in the user
                interface.

The usage is slightly awkward. It requires specifying –Scan –ScanType 3 –File <filename>.

A simple powershell function or a batch file.

Powershell

function Scan-File( $file )
{
    $exe = Join-Path $env:ProgramFiles &quot;Microsoft Security Client/Antimalware/MpCmdRun.exe&quot;
	&amp; $exe -Scan -ScanType 3 -File $file
}

Batch

@echo off
setlocal
set path=%programfiles%\Microsoft Security Client\Antimalware;%path%
cmd /c MpCmdRun.exe -Scan -ScanType 3 -File %1

Niftier PowerShell integration is possible by extending the work of the Scripting Guy’s Invoke-SecurityEssentials.ps1 script for MSE v1.0.

%d bloggers like this: