Recently I was tasked with helping to convert a large number of C++, C++/CLI and C# projects from Visual Studio (VS) 2008 to 2010. It was neither an easy nor a fun thing to do and while I was doing it I found a near complete lack of information on the internet so I thought I’d share what I discovered in the hope that it helps someone else.
Note: this is not a laundry list of what I don’t like about VS 2010 – I don’t have the time or the inclination to make a list and I’d constantly be remembering or discovering additional things. Where possible, I’m still coding in VS 2008, and I’d prefer to be coding away in VC6.
Background to Visual Studio 2010′s build process
All I had read before I started was a helpful article about custom build rules on the very helpful VirtualDub blog (which I’ve previously mentioned). That already introduces you to the way in which VS has changed. In the case of C++ and C++/CLI, the newer VS has changed the project building system and project file format from .vcproj (a legacy XML format) to .vcxproj which is formatted ready to be used with MSBuild. Actually, that’s a bit too much of a simplification – it’s not like changing the suffix and a bit of an XSLT is going to do the conversion for you, but then it’s a gamble whether or not the auto-conversion system will work either.
My understanding of MSBuild is basically it’s a way of passing a list of things (<item> elements) to tasks to do (elements with their own name). MSBuild only supports a minimum of dependency checking, in essence to ensure each task is only executed once before anything that depends on its outputs. Looking deeper at this, you have to ask yourself where the source level dependency checks happen, and that would be a bloody good question, especially when it all starts falling apart.
Visual Studio 2010 actually picks up the checking the dependencies of your source itself. Of the different ways you could implement this, VS assumes that the build will generate a set of *.tlog files, or actually something like exename.id.read.tlog and exename.id.write.tlog files (like cl.1656.read/write.tlog) which track the files read and written by each process in the build. Some of the tasks, such as CL, can write their own, but other tasks get run inside another process, tracker.exe (possibly spawned by the FileTracker task), which tracks files opened for reading and writing to generate the tlog files. From VS’s point of view, the tlog files are gospel when it comes to working out if MSBuild needs to be invoked.
Getting started with your upgrade
Before you start, there’s some preparation work you can do. First of all, grab Service Pack 1 for VS 2010. Most of the upcoming issues may not (but will probably still) appear if you’ve upgraded – maybe they’ve actually fixed something, but the release notes are surprisingly thin. Overall though, VS 2010 with SP1 seems to crash less.
If the results of the conversion are being committed to source control and you want to maintain your VS 2008 projects, you should branch the .sln files since the conversion process will attempt to overwrite in place. The same is true of .csproj files thanks to the minor tweaks performed by the conversion process. Hell even if you’re not keeping your VS 2008 versions or using source control, it’s definitely worth backing up in case something goes properly wrong, and that’s quite likely.
When (not if) something does go wrong with your C/C++ build, you’ll want as much information as possible, so I’d recommend you take a look at both this stackoverflow page and more specifically the linked page on the VS Project Team Blog that tells you how to some extra debug info out of VC++. The community spirited stackoverflow is probably a better place for help than connect.microsoft.com where you can find solidarity with others with the same problems but fewer solutions.
If nothing obvious goes wrong and you’ve converted your first project, you’ve got to ask yourself, what’s gone wrong under the hood?
Issue #1 Why doesn’t it rebuild?!
Sometimes the build process will spit out empty cl.read.1.tlog files and you’ll find that nothing will rebuild even when changing source and headers. The trick you learn is to tell the difference between the two different “build is up-to-date” messages from VS:
- The one that means VS thinks everything is up-to-date and won’t even bother to kick off MSBuild.
- The one where VS thinks something needs building, kicks off MSBuild and MSBuild says, everything’s up-to-date.
I wanted to post both of these messages, but I didn’t want to rebreak my build. This is where the previously mentioned page on the VS Project Team Blog comes in useful.
When weird dependency issues arise, the best place to go is the logs. First things first, go to the project build options and set the log file settings to diagnostic – make sure that’s the log files and not the output window since several hundred KBs or tens of MBs isn’t unusual for a diagnostic level log. You might as well leave this on since by dumping the diagnostics to a log file, it gets written to the intermediate directory so when things go wrong, it’s there already.
The log itself interleaves VS output and information from the MSBuild tasks. The best way to use the log is to grep for the file you’ve changed, find the places where it’s mentioned and the first clue will be whether the system thinks it’s up to date or not. For reference, other good things to grep for are “missing” and “up-to-date”.
So far, I’ve not found anything that isn’t fixed with a full rebuild, but that’s the crappiest solution available.
Issue #2 Why does it keep rebuilding?!
Nothing ruins that “It builds!” feeling more than the realization that it just keeps rebuilding and is never happy, even when nothing’s changed. At this point, I refer you to the stackoverflow post I mentioned above. In our case, most of this was due to the following bug:
Issue #3 WTF happened to my .rules files?!
A new MSBuild based build system in 2010 means that all of the bespoke .rules files you crafted to save you lots of effort in VS 2008 won’t fly any more. In fact the autoconvert system makes a good try at fixing your rules, but when it comes to building, a good try just means irritating unexpected rebuilds – gah! Don’t forget that you created the rules file so that you could share it between projects but now each 2008 project you open wants to reconvert it for 2010, so you can’t start fixing it until you’re all done converting your projects.
So where to start. The conversion creates three separate files, a .props, .targets and .xml file. First of all just see if it works – it might! If you have anything more complex than the simplest rules the conversion may have failed in some interesting way. Some things you’ll need to look at are:
- A possible loss of quotes. The command line from one of our rules files looked like “$(toolpath)” arg1 arg2 “$(argfile)” where we’d quoted the arguments that could include paths with spaces. The conversion process spotted that the line started and ended with quotes and left us with $(toolpath)” arg1 arg2 “$(argfile), or fubar’d as it’s better known.
- Any dependencies you had in the rules file (like the tool used by the rules file) may be encoded as an AddedDependencies list which we found VS couldn’t resolve and therefore always considered the output as always needing an update. This seems to be a common problem, missing files causing a rebuild.
- In the .targets file, the Inputs list includes $(MSBuildProjectFile). It’s difficult to work out if this is necessary or not since some of the property values for the converted files could be in the .vcxproj, but it does mean that all your custom stuff rebuilds whenever you make changes to the .vcxproj, such as adding a file or making a setting change.
- Depending on your diagnostics settings, the message you had for your rules when building may appear randomly when building. We had to tweak ours so that the message wasn’t always output for files excluded from the build.
And that’s the first set of problems we had. Since this is much longer than many of my previous posts and I’m not even halfway there yet, I’m going to split this into at least one more post.