There are two types of ampersands that you need to be aware of when playing with SharePoint Taxonomy

Our favorite and most loved

& ASCII Number: 38

And the impostor

＆ ASCII Number: 65286

After reading this article by Nick Hobbs, it became apparent that when you create a term it replaces the 38 ampersand with a 65286 ampersand.

This then becomes a problem if you want to do a comparison with your original source (spreadsheet, database, etc) as they are no longer the same.

As detailed in Nick's article, you can use the TaxonomyItem.NormalizeName method to create a "Taxonomy" version of your string for comparison.

Below is the code I used in the SharePoint 2013 Client Component which is a little different from the server code.

string myString = "This contains &";
using (var context = new ClientContext("http://myurl"))
{
var result = TaxonomyItem.NormalizeName(context, myString);
context.ExecuteQuery();

string normalisedString = result.Value;
}