Solving Very Slow Visual Studio Build Times in VMWare

ClockIn 2009, we chose 15″ MacBook Pro BTO over Dell, HP and Lenovo offerings. Apple offered us the best hardware and equivalent lease terms but with much simpler servicing done by ETF rather than (often incorrect) paper statements and checks. 99% of the time we ran this MacBooks with Windows 7 under Boot Camp. When I started doing some iOS development, I ran Snow Leopard and Xcode under VirtualBox and when I got fed up with the flakiness, I used VMware Workstation. OS X didn’t run very well under virtualization mostly because accelerated Quartz Extreme drivers don’t exist for VMware Workstation. Still, it actually worked well enough and was much more convenient than dual booting — which is such a huge time suck. When the lease period ended, we renewed with Apple and decided to just use OS X as the host environment for a variety of reasons:

The transition went very smoothly. I was using a VM with Windows Server 2008 and Visual Studio 2010 for primary .NET web development. Configuring IIS Express to serve outside of localhost bound to a host-only adapter is great for cross-browser testing, but it can be even more useful to enable remote access to Fiddler and point external browsers to the Fiddler proxy running in the VM to get both client debugging and HTTP sniffing at the same time. All of this was working out great.
 
At the start of October I downloaded Windows Server 2012 and Visual Studio 2012 and created a new development VM. In order to minimize disk storage, I moved my source tree to a folder in the host OS X environment and exposed it via the “Shared Folders” feature of VMWare Fusion 5.x to Windows. At the same time I started working on a new project.

 

The Slowening

Once the project grew to 20, 30 and 50k lines of C# code, the build times started to become horrifically slow. When combined with running unit tests, build became a big time for a bathroom break or cup of coffee event like building a project in C 15 years ago. Builds would show cdc.exe running at ~50% CPU (e.g. 1 core) and some other stuff totaling ~70% CPU. The VM was not memory bound and network IO was minimal. This was my first substantial project using Code First EF, so I thought maybe the complex object graph is just hard for the C# compiler to deal with.
 
After a few weeks of increasingly painful build times, I was looking at breaking my solution up so that I could build against pre-compiled DLLs — anything to make it go faster. I ran across a post on SuperUser:

… (Full disclosure: I work on VMware Fusion.)

I have heard that storing the code on a “network” drive (either an HGFS share or an NFS/CIFS share on the host, accessed via a virtual ethernet device) is a bad idea. Apparently the build performance is pretty bad in this configuration.

Oh really? Hmmm. Maybe it isn’t that my class libraries are so complex but something else is going on. Here are some empirical measurements of rebuild time of an actual solution:

VMWare shared folder: 	50 sec
OS X SMB share: 	18 sec
within virtual disk:	 9 sec

Wow. Problem solved. Incremental builds are basically instantaneous and a full rebuild takes 9 seconds when the code is hosted inside the VM image. Not only does hosting the source code within the VM virtual disk make the build go 5.5x faster, the CPU time of csc.exe goes way down. I don’t know how the VMWare shared folder is implemented. It appears as a mapped drive to a UNC name to Windows but it is very slow. Moral of the story is just don’t host your source code on the host machine with VMWare. The performance penalty is just not worth it. If you need to share the source code tree inside the VM with the host OS, create a file share from the VM to the host over a host-only adapter.

 

Update

I can confirm this is still a problem in VMWare Fusion 6. I’m hoping maybe the new SMB implementation in Mavericks might greatly improve the performance of sharing source code from the host OS to the VM.

Update

I just updated my virtual machine to Windows Server 2012 R2 (aka Windows 8.1 server). It is running on VMWare Fusion 6. The build time of this large project is now 15 seconds over VMWare Shared Folders. Significant remaining issue is that Visual Studio uses a whole core of CPU to do nothing — just having a large solution open, not editing anything.

Advertisements

How to Create an Xcode 4.0-style Window-based Application in Xcode 4.2

I’ve decided to get up to speed on iOS programming and, to help me with that, I bought Aaron Hillegass’s iOS Programming: The Big Nerd Ranch Guid 2nd edition. I’ve listened to Aaron on podcasts and have heard that his training method is fantastic. The book does seem to be unusually well written for a tech book but the problem I immediately ran into is that it was written for Xcode 4.0 and iOS 4.3 SDK. I have Lion with Xcode 4.2 and iOS 5 SDK. That’s part of the adventure of discovery and change in tech but unfortunately Apple removed the project type that Hillegass’s examples are based on which is a major stumbling block.

Hillegass uses the “Window-based Application” template for iOS as the basis for all of his projects. That template no longer exists in Xcode. The closest thing in Xcode 4.2 is “Empty Application”. However, that’s not quite the same thing because the now-defunct “Window-based Application” template generated a main window XIB view wired to generated controller but the “Empty Application” has no XIB. In order to make any sense out of the book, you have to create the XIB and wire it up yourself. This is no big deal to anybody with a little experience but to a newb like me, it took a bit of wandering in the wilderness at the Big Nerd Ranch forums to figure out how to manually get the project into the state that the exercises in the book will work.

How to Convert a new “Empty Application” into a “Window-based Application”

Start by creating a new iOS Application project using the “Empty Application” project. This will create a very similar project to the “Window-based Application” project described in iOS Programming 2/e with just a few differences:

  1. The main controller class that gets generated is “AppDelegate” rather than “<project-name>AppDelegate”. The
  2. There is no XIB view.
  3. The window property of AppDelegate doesn’t have the IBOutlet macro to make it visible to the Interface Builder tooling.
    1. Prepare the controller to talk to the Interface Builder tooling

    In the controller class interface definition of the we need to add the IBOutlet macro so that the tooling will recognize the controller. In your controller header file, AppDelegate.h, the line to modify begins with @property and ends with *window. Add IBOutlet in front of the UIWindow type as shown below.

#import <UIKit/UIKit.h>

@interface AppDelegate : UIResponder <UIApplicationDelegate>

@property (strong, nonatomic) IBOutlet UIWindow *window;

@end

Save this change.

Update the didFinishLaunchingWithOptions: method

The auto-generated first line of code in the didFinishLaunchingWithOptions: method of AppDelegate.m prevents message routing from working properly. Comment it out as below:

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    //self.window = [[[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]] autorelease];
    // Override point for customization after application launch.
    self.window.backgroundColor = [UIColor whiteColor];
    [self.window makeKeyAndVisible];
    return YES;
}

Save this change.

Create the XIB file

Right-click on the folder containing your AppDelegate files and choose “Add File”. Select “Window” from the iOS User Interface group. Choose the appropriate device family—for the initial Quiz project that is iPhone. The default file name is Window.xib which works fine or you can name it MainWindow.xib to make it just like in the book.

OS X 10.7 (Lion)-2011-11-23-11-18-56

Change the Class of the XIB file’s owner to UIApplication

Select Window.xib in the Navigator. Select the “File’s Owner” cube in the objects pane just left of the layout canvas and change the Class in the properties pane to UIApplication.

OS X 10.7 (Lion)-2011-11-23-11-34-47

 

Add an object for the controller

Filter the tool pallet on the bottom right to “Objects and Controllers” and drag and drop an Object from the toolbox onto the object pane.

OS X 10.7 (Lion)-2011-11-23-11-37-52

Set the class of the controller object

In the properties window, change the class to controller that was generated for your project by the template. That should be AppDelegate. The Object will automatically be renamed with a space added into the Pascal casing. (You can override this by putting something into the Label field of the Identity pane.)

OS X 10.7 (Lion)-2011-11-23-11-45-00

Route the XIB delegate to the controller

Now we want to wire up the MVC message routing for the view. Select the “File’s Owner” cube in the object pane and control+drag the delegate outlet to the App Delegate object.

OS X 10.7 (Lion)-2011-11-23-12-00-15

 

Route the controller object’s window outlet to the window object

Next perform a similar ctrl+drag of the window outlet from the App Delegate object to the Window object.

OS X 10.7 (Lion)-2011-11-23-12-01-08

All done

Now our Empty Application project is set up just like a Window-based Application as described in iOS Programming 2nd Edition. The file names may be a bit different but that should have no practical  effect. The AppDelegate inherits from UIResponder <UIApplicationDelegate> rather than NSObject<UIApplicationDelegate>. This doesn’t seem to be a problem, either.

Now that I have figured out how to create an wire up a main window XIB file, everything seems to work great as I follow along in the book.

Hide VMWare Virtual Network Interfaces from Windows Firewall and Network and Sharing Center

VMWare Workstation creates two virtual network adapters by default. One is the host-only network and the other is for NAT routing. You can add several more to suit your needs for more complex scenarios.

The problem is that these networks cause Windows 7 to think it is attached to a public “unidentified network” which has the side-effect of disabling network and printer sharing. You will experience the same problem with Windows 7 Virtual PC if you create virtual loopback device adapters and the same problem occurs with VirtualBox.

The solution is to mark the virtual network adapters as *NdisDeviceType=1 in the registry:

*NdisDeviceType

The type of the device. The default value is zero, which indicates a standard networking device that connects to a network. Set *NdisDeviceType to NDIS_DEVICE_TYPE_ENDPOINT (1) if this device is an endpoint device and is not a true network interface that connects to a network. For example, you must specify NDIS_DEVICE_TYPE_ENDPOINT for devices such as smart phones that use a networking infrastructure to communicate to the local computer system but do not provide connectivity to an external network.

Note  Windows Vista automatically identifies and monitors the networks a computer connects to. If the NDIS_DEVICE_TYPE_ENDPOINT flag is set, the device is an endpoint device and is not a connection to a true external network. Consequently, Windows ignores the endpoint device when it identifies networks. The Network Awareness APIs indicate that the device does not connect the computer to a network. For end users in this situation, the Network and Sharing Center and the network icon in the notification area do not show the NDIS endpoint device as connected. However, the connection is shown in the Network Connections Folder.

As far as I’m concerned this is just a bug in both VirtualBox and VMware Workstation. They should be marking their virtual network devices as *NdisDeviceType=1 for compatibility with NT 6.x –based operating systems.

Until they do that, I cobbled together a little powershell script to find any NICs created by VMWare and mark them as virtual. The setting takes affect after a reboot.

# tell windows that VMWare Network Adapters
# is not a true network interface that connects to a network
# see http://msdn.microsoft.com/en-us/library/ff557037(VS.85).aspx
pushd
echo "Marking VMWare Virtual Ethernet Adapters as virtual.`r`n"
cd 'HKLM:\system\CurrentControlSet\control\class\{4D36E972-E325-11CE-BFC1-08002BE10318}'
ls ???? | where { ($_ | get-itemproperty).DriverDesc `
-like 'VMware Virtual Ethernet Adapter *' } | `
% { $_ | new-itemproperty -name '*NdisDeviceType' -PropertyType dword -value 1 } | `
% { "`"" + ($_ | get-itemproperty).DriverDesc + "`" -> *NdisDeviceType=1" } 
echo "`r`nReboot to apply changes."
popd

Configuring ASP.Net MVC for the Microsoft Charting Control

There are a couple of gotchas when using either the .NET 3.5 add-on charting control or the one that ships with .NET 4.0 with ASP.Net MVC. These can lead to some head-scratching and consternation at deployment time if you don’t know about them.

MVC Paths Break Chart Images

MVC paths are RESTful which means that a lot of the path in a URL is likely to be arguments to a controller rather than an actual path to a resource. Unfortunately, by default these paths confuse the charting control which assumes that the path in the URL leads to the resource hosting the control. The net result is that invoking RESTful methods on your controller causes your chart images go missing. The solution is to use a little bit of regular expression to create a “ignore” routing rule in Global.asax.cs for ChartImg.axd which is the virtual file that generates the charting control images.

public static void RegisterRoutes( RouteCollection routes )
{
    //MSFT charting control image genertor
    RouteTable.Routes.IgnoreRoute( "{*chartimg}", new { chartimg = @".*/ChartImg\.axd(/.*)?" } ); 
    //standard default route ignore rule
    RouteTable.Routes.IgnoreRoute( "{resource}.axd/{*pathInfo}" );

    routes.MapRoute(
        "Default", // Route name
        "{controller}/{action}/{id}", // URL with parameters
        new { controller = "Home", action = "Index", id = UrlParameter.Optional } // Parameter defaults
    );
}

Charting Control Wants to Write to Your Application Root

Another problem that will show up at deployment time is that the charting control wants to write cached images to the root of your application by default and it wants to do this in the security context of the invoking user. That is, in order for this default to work, anonymous web users need write access to the root of your application. This is a bad, bad, bad idea.

Fortunately this behavior can be configured in the Web.config. You can change the cache directory or use global application memory or session to store the images. Using memory has the advantage that you don’t need to give anybody permission to write into the server file system.

<configuration>
  <appSettings>
    <add key="ChartImageHandler" value="storage=memory;timeout=20;privateImages=false" />
  </appSettings>
<configuration>

The full set of configuration options are described on MSDN.

FIX: VirtualBox Host-Only Network Adapter Creates a Virtual “Public Network” Connection That Causes Windows to Disable Services

vbox-host-only-net-net-n-sharing

vbox-host-only-net-connectletVirtualBox creates a “VirtualBox Host-Only Network” device which is essentially a loopback adapter for creating network connections between virtual machines and between the host and virutal machines. Unfortunately, it shows up to Windows as an unidentified public network. Connecting to a “public network” ratchets up your firewall and disables network discovery and SMB/CIFS network shares. This is kind of a big side effect of installing some VM software.Fortunately, Windows does have a way to mark a network device as virtual by creating a registry value.

The type of the device. The default value is zero, which indicates a standard networking device that connects to a network. Set *NdisDeviceType to NDIS_DEVICE_TYPE_ENDPOINT (1) if this device is an endpoint device and is not a true network interface that connects to a network. For example, you must specify NDIS_DEVICE_TYPE_ENDPOINT for devices such as smart phones that use a networking infrastructure to communicate to the local computer system but do not provide connectivity to an external network.

This powerhsell script will find the “VirtualBox Host-Only Ethernet Adapter device entry in the registry and adds the ‘*NdisDeviceType’ value of 1. After reboot, the “VirtualBox Host-Only Ethernet Adapter” will no longer be monitored by the Network and Sharing center.

# tell windows that VirtualBox Host-Only Network Adapter
# is not a true network interface that connects to a network
# see http://msdn.microsoft.com/en-us/library/ff557037(VS.85).aspx
pushd
echo 'Marking VirtualBox Host-Only Network Adapter as a virtual device.'
cd 'HKLM:\system\CurrentControlSet\control\class\{4D36E972-E325-11CE-BFC1-08002BE10318}'
ls ???? | where { ($_ | get-itemproperty -name driverdesc).driverdesc `
-eq 'VirtualBox Host-Only Ethernet Adapter' } |`
new-itemproperty -name '*NdisDeviceType' -PropertyType dword -value 1
echo 'After you reboot the VirtualBox Host-Only Network unidentified public network should be gone.'
popd


VirtualBox Seemless Mode Glitch with Ubuntu

I recently installed Ubuntu 10.04 Lucid Lynx in VirtualBox. I applied all updates and installed the VM additions. When I tried swtiching into seemless mode, I ran into a problem where all of the window chrome disappeared. Well, the title bars were gone as well as a little bit of the status bar that includes the rounded corners. The title bars were still there but still, I was unable to move or resize windows in seamless mode, which is not ideal.

vbox-seemless-bug

Seemless Mode Only Works Correctly with all Effects Disabled

After fiddling for a while, I realized that Ubuntu had turned on some window effects after rebooting with the VM additions installed. Disabling all window effects solves the problem.

ubuntu-disable-effects

Seamless mode is cool but it isn’t multi-monitor aware. I can’t drag guest OS windows off of the monitor where seamless mode was initiated. If I define the VM as having 2 monitors in VirtualBox, Ubuntu doesn’t see the second one. Also, the screens size is set to the resolution of my smaller monitor in seamless mode which causes the Ubuntu desktop toolbars to float in the middle of my screen. Basically, the entire guest OS desktop has to be on one of my monitors but I can move everything with {right-CTRL+L} (to exit seamless mode), move guest OS window to desired monitor, {right-CTRL+L} (to enter seamless mode on the new monitor). This solution is workable, but hopefully some day VirtualBox seamless mode will automagically present multiple monitors to the guest OS and let me drag windows around arbitrarily.

%d bloggers like this: