Backup Windows Server on EC2 with PowerShell and AWS CLI

When we first started working with Windows Server or Amazon EC2, we ran into the problem of how to back up SQL Server databases. My initial thought was to somehow mount S3 as a volume and have SQL Server write backup jobs to it. That turns out to be kind of hard.

After a little head-scratching, I realized that EBS volume snapshots are stored in S3. Therefore, all we need to do is mount an EBS volume and have SQL Server write backup jobs to that and then snapshot that volume. Oh, and unmount it from the running Windows server before snapshot to make sure that we are taking clean snapshots.

It turns out to not be too hard to make this happen on Windows Server 2012 Core using a PowerShell script that drives the AWS CLI and does the hoodoo with the Windows volume manager to mount and dismount the EBS backup volume.

This solution requires PowerShell Storage Cmdlets which ship with Windows Sever 2012/ and Amazon Web Services python-based command-line interface. Works great on Windows Server Core.

The biggest problem that I ran into was that the script would run just fine when invoked manually or when I would trigger the job from schtasks while logged in, but it would hang and fail when run from scheduled tasks in the middle of the night. Long story short: when schtasks runs the job with nobody logged in it gives the .DEFAULT environment instead of the environment of the user context of the scheduled task. That meant that AWSCLI didn’t receive the correct %USERPROFILE% environmental variable and was not able to locate its config file with the user id and key.

The simplest solution was to wrap the invocation of the powershell script in a cmd batch script:

set USERPROFILE=C:\Users\Administrator\

powershell.exe -file “C:\Program Files\Invoke-BackupJob.ps1” -path E:\ -diskNumber 2 -ec2VolumeID vol-<id-number-here> -description “Important Backup” > “C:\Program Files\Utility\Log.txt”

Use the PowerShell Get-Disk cmdlet to figure out the disk number of the EBS volume in Windows and the EC2 console or AWSCLI to figure out the EBS volume ID in EC2.

[gist https://gist.github.com/breiter/7887782]

Compiling Mono 3.x from Source with LLVM Code Generation in Ubuntu LTS x64

The packages for Mono in Debian and Ubuntu are lagging far behind what is available from Xamarin for OS X, SUSE and OpenSUSE. The 2.10.x versions of Mono are .NET 2.0 runtime compatible and use Mono’s older Boem garbage collector — which tends to leak objects. The new generational “sgen” garbage collector is far superior All of the tutorials for building on Fedora/RHEL/CentOS and Debian/Ubuntu that I have seen build with the built-in code generation engine rather than LLVM. LLVM provides a significant speedup in the generated code of 20-30% but Mono can’t link against the version of LLVM that ships in Debian Wheezy or Ubuntu. You have to build a tweaked version from source — which is the tricky bit.

My goal was to build the same version of Mono that is shipped by Xamarin for OS X on Ubuntu 12.04 Precise Pangolin, including LLVM. That is mono 3.2.5 at the time of this writing.

Screen Shot 2013 12 01 at 10 58 51 AMPrerequisites

Mono is hosted on GitHub so we need git. It also requires GNU autotools, make and g++ to drive the build. 

sudo apt-get install libtool autoconf g++ gettext make git

We’re building something that isn’t managed by the system package manager, so according to the UNIX Filesystem Hierarchy Standard, the build product should go into the /opt file system. I’m going to use /opt/local and so before going any further let’s add that to the $PATH by editing /etc/environment with vi or your editor of choice:

PATH=“/opt/local/sbin:/opt/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games”

Also, we need to change the current environment:

export PATH=/opt/local/sbin:/opt/local/bin:$PATH

Libgdiplus

The first component of Mono that we need to build from source is libgdiplus. Libgdiplus is an open source implementation of the Windows GDI+ API for graphics and text rendering. Libgdiplus has a number of prerequisites because it basically just maps GDI+ function calls onto equivalent Linux libraries:

sudo apt-get install pkg-config libglib2.0-dev libpng12-dev libx11-dev libfreetype6-dev libfontconfig1-dev libtiff-dev libjpeg8-dev libgif-dev libexif-dev

mkdir ~/src; mdkr ~/src/mono; cd ~/src/mono

git clone https://github.com/mono/libgdiplus.git

cd libgdiplus

./configure –prefix=/opt/local

make

sudo make install

 

Mono-LLVM

Mono has a tweaked fork of LLVM. By default it will try to build x86 and x86_64 targets on Ubuntu x64 but the build will fail because there is no x86 runtime support installed.

cd ~/src/mono

git clone https://github.com/mono/llvm.git

cd llvm

git checkout mono

./configure –prefix=/opt/local –enable-optimized –enable-targets=x86_64

make

sudo make install

Mono with LLVM

Each official release of Mono is tagged in git. You should checkout the version that you want to build. In this case, I’m building Mono 3.2.5 with the LLVM and libgdiplus libraries we already built. Since there is no working version of Mono on the system, the very first time you build you need to do make get-monolite-latest which will pull down a minimal mono-based C# compiler to boostrap the build.

cd ~/src/mono

git clone https://github.com/mono/mono.git

cd mono

git checkout mono-3.2.5

export MONO_USE_LLVM=1

./autogen.sh –prefix=/opt/local –enable-llvm=yes –with-sgen=yes –with-gc=sgen

make get-monolite-latest

make EXTERNAL_MCS=${PWD}/mcs/class/lib/monolite/gmcs.exe

sudo make install

Assuming that all went well, you should now have a working build of mono 3.2.5 (or newer) in /opt/local/bin/ and executing mono –version should include LLVM: yes.

Depending on what you want to do with Mono, you may now want to build XSP (ASP.NET server, Apache module and FastCGI module for nginx) and/or F# 3.x.

XSP

The xsp2 server uses a .NET Framework 2.0/3.0/3.5 API and the xsp4 server provides a 4/4.5 API. The main tweak here is to set the PKG_CONFIG_PATH environmental variable so that the configure script can find mono in /opt/local.

cd ~/src/mono

git clone https://github.com/mono/xsp.git

cd xsp

git checkout 3.0.11

PKG_CONFIG_PATH=/opt/local/lib/pkgconfig ./autogen.sh –prefix=/opt/local

make

sudo make install

F# 3.1

cd ~/src/mono

git clone https://github.com/fsharp/fsharp

cd fsharp

git checkout fsharp_31

./autogen.sh –prefix=/opt/local

make

sudo make install

%d bloggers like this: