Panic mode: Missing VMDK files for an ESX guest

At my POE, we have a huge VMWare infrastructure.  About 30% of our servers are actually VMs.  This isn't just test servers, but production as well.   Last night, I was performing some routine maintenance on a VM owned by one of our support and test groups.  I was adding a second NIC to their VM so they'd be able to get on the TAN and start restoring production environments so they could duplicate issues.  A relatively simple task, I powered down the VM, added the NIC and powered back on.

 

Or, should I say...hit the power button.  The machine wouldn't come back up, I kept getting  "File not found" error.  So, I tried the logical thing, I removed the NIC and tried again. Same results.  Hmmm..  So, now, I needed to do some real investigating.  Since it was a test machine and getting very late (and I've got a bad cold), I put it off until this morning.  Taking a closer look at the event log, I found the problem: "VMware server cannot find the virtual disk /vmfs/blahblahblah".  Uh-oh.  That disk is the first image in a group of disks that makes up a 450G volume in the VM.  Did I mention this machine wasn't on the TAN, and so not backed up?

 

 I fired up Putty and did some poking around.  This machine actually has 4 disks in it.   And sure enough, all of my flat files were there, but the descriptor files were missing.  For those unfamiliar with the specifics, in newer versions of VMWare, virtual disks are actually made up of two files: disk.vmdk which is just a text file called a descriptor containing the specifics of the disk.  Size, type, geometry, etc.  There's another vmdk file, disk-flat.vmdk that is the actual flat file that holds your data.  Personally, I think it would make more sense if they used different extensions, but that's just me.

 

I did some poking around and found that disappearing descriptors is a relatively common problem these days.  The solution is relatively easy, but not something *I* could do easily.  I had to create a new virtual disk, copy the descriptor from it and just edit it to point to the right flat file.  The problem is, the first disk that was missing its descriptor is 250G and I've got 50G free on this standalone host.  I'd have to recreate on another host, copy over, etc.  But, at the last moment I found this:

 

esXpress Stub Generator

 

Simply enter the size in bytes of the flat file (copy...paste) and the name of the flat file (copy...paste) and it generates a new stub that you can...you guessed it...copy, paste into VI on the host.  I took a moment to try recreating a couple of vmdks that I did have, just to make sure I got exactly the same results using this tool.  I'm not sure if it makes a huge difference or not, but esXpress generator always populates with the line:

 

ddb.toolsVersion = "0"

 

And, all of my descriptors have this one:

 

ddb.toolsVersion = "7299"

 

I don't think it would've made any difference, but I edited all of mine so the version number was 7299.  Fingers crossed, I powered up the VM and it came right up.  Chkdsk found no errors on the volume, and we're back in business.   Thanks so much to esXpress.  You saved me a ton of work this morning!

 

 

Automated Active Directory Distribution List Creation

One of the skills that I've found to be most helpful in my day-to-day as a Windows admin is knowing how to script well.  One of the running gags from most Unix guys is that Windows is completely unscriptable.  They forget, of course, that any scripting language that runs on their OS of choice most likely works on mine.    That being the case, I generally stick to one of the VB variants when doing what I need to do.  I stick to them for a couple of reasons, most particularly that they're very easy to use and learn.  I've also found that most situations I might encounter have been overcome by someone else and I can count on one hand the number of times I've encountered a solution that wasn't written in VB or VBscript (most often it's C# or Powershell).  

The latest little project to cross my desk was a need to create almost four dozen distribution lists in AD.  I could've done them all by hand, but usually it's easier to write a script.  To that end here's my solution, with a bit of info on how it works. 

' Setup some variables for use later on.
strParentDN  = "ou=DistributionGroups,ou=Exchange,dc=domain,dc=com"
strGroupName = "THE_" & Wscript.Arguments.Item(0) & "_DAEL"
strSMTP = strGroupName & "@domain.com"
Const ADS_GROUP_TYPE_UNIVERSAL_GROUP = &h8

set objOU = GetObject("LDAP://" & strParentDN)
set objGroup = objOU.Create("group","cn=" & strGroupName)
objGroup.Put "groupType", ADS_GROUP_TYPE_UNIVERSAL_GROUP
objGroup.Put "sAMAccountName", strGroupName
objGroup.MailEnable

' Here I'm going to "finalize" some of the settings for the DL.  The reason for this
' is I've found that until a group is created and mail enabled, you can't set an
' SMTP address for it.  Guess it makes sense...
objGroup.SetInfo

' But, since the group's already opened as an object, I can simply continue
' to add attributes to it.
strDN = "cn=DL Management,ou=Groups,dc=domain,dc=com"
objGroup.Put "mail", strSMTP
objGroup.put "targetAddress", strSMTP

' This says who can add/remove members from the group.  But, read on...
objGroup.Put "managedBy", strDN
objGroup.SetInfo

Set objShell = CreateObject("Wscript.Shell")

' If you look on the Managed By tab on the DL's Properties sheet, you'll see a checkbox
' "Manager can update membership list".  Just because you've given a user (or in this case a group)
' management of that list doesn't mean they can manage the membership.  Now, we could do this
' the right and approved way, which involves setting up DACLs and ACLs for the object
' and all kinds of other voodoo.  I find it's easier to just use some of the tools provided by MS.

strCMD = "dsacls " & "cn=" & strGroupName & "," & strParentDN & " /G domain\dlmanage:WP;member"
objShell.Run strCMD

' Let the user know it worked!
Wscript.Echo "Successfully created mail-enabled DL."

Not much to it, but someone might find it useful.  And, it certainly demonstrates how easy it really is to script AD.   19 lines of actual code to create 47 distribution lists.  I guess that's a fair trade!

Can the iPhone cut it as a corporate solution?

Originally, I had wanted this to be a very positive post.  I'm in the process of rebuilding my laptop and as a result needed to get iTunes up and working with my iPhone again.  I thought it would be a relatively easy process.  After all, everything else with the iPhone is.  Not so much.  Two days and three calls to Apple Support later and it's working.  It appears I haven't lost any info, but I was fully confident all along the way that I would. 

 

Since this was a voluntary reinstall, I had the opportunity to back everything up first.  I backed up my music folders as well as a complete copy of my user profile to minimize the chance of losing anything.  After reinstalling, I plugged my phone in and the first thing iTunes told me was "You can only sync with one computer at a time, if you sync your phone will be erased and replaced with what's in this instance of iTunes" (it's been a couple of days and I don't have the exact error message, sorry).  Warning bells!  So, I called Apple Support.  I spoke with a very nice fellow who told me "Oh, it's really easy.  Just plug it in and make a backup.  That'll repopulate iTunes and you'll be good to go."  Sweet!  Nope, wasn't to be.  I did this, but when I looked, none of the music had been put back as had none of the applications. 

So, I did a little research and found information on the "Transfer Purchases" option when you right-click your phone in iTunes.  It'll put any applications or music you bought from the iTunes store back from your phone.  Grand!  That seemed to work, but when I went to sync, I got the same dire warning that everything would be erased.  Hmmmm...now, keep in mind, I use the iPhone primarily to access the Cloud.  The vast majority of the data I use is out there, but I do keep some things on the phone.   Most specifically pictures of my daughter, but I have data in the Notes application, Calendar, To Do, Checklist, etc.  The pictures are easy enough to back up, but the others aren't.  It wouldn't be the end of the world to lose some of this data, but I figured it could wait until the morning when I had a chance to talk to Apple Support again. 

 

The second tech was just as nice, but easily one of the most clueless people I'd ever spoken to.  I had to explain that a monitor going into power save was not the same thing as the computer going to sleep.  She even told me that if I didn't check the boxes that said "Sync Music" or "Sync Applications" that it would still do so.  Apparently, in her mind, checking those boxes meant to do some kind of SPECIAL sync that erases everything and starts over.  As long as you pushed the Sync button on the Summary tab, it'll sync everything everytime regardless of what you told it to sync.   I got off the phone with her, tried a few things and decided to try my luck again with Support...third time being the charmed one and all.  This last tech did have a clue, but was completely unable to help me as well.  In the end, I finally had to just give up and tell iTunes to go ahead and do whatever it planned to do and if I lost data, I lost data.  It looks like I didn't (although I did have to relogin to every app that stored one like Facebook and Linkedin), but the experience has really soured me.

 

It really shouldn't be this hard.  The iPhone is about ease of use and convenience, and this experience didn't match that.  I never told the techs I'd done a manual reinstall as I wanted to avoid the "Oh, of course you reinstalled...you're on Windows" rhetoric.  I told them I'd had a hard drive crash as I figured there must be some kind of procedure available to deal with that situation.  Apparently, though, I'm the first person out of the millions of iPhone users that had a hard drive crash and had to restore.  One of the techs simply shrugged it off "I think most people will just reinstall and not bother to try to keep their data".  Yes, most of the drones who bought the iPhone solely because it has an Apple on the back won't care.   I also acknowledge that perhaps I got a little overanxious about the situation.  But when I have error messages that specifically state "you WILL lose everything on your phone", I can only take those at face value.  Compound that with a string of support techs that don't even have the faintest idea of how the software works and what am I supposed to think?

 

At my workplace, my team supports the messaging environment and therefore has a hand in supporting the mobile workforce.  When management came to us and asked to validate the iPhone with our Exchange environment, the team came back with a "No" because it couldn't be reliably and consistently wiped in the case of a theft or loss.  If Apple wants the iPhone to compete seriously in the business use marketplace, it's going to need to understand that this kind of laissez faire attitude towards people's data doesn't apply in business.  They've spent the last couple of decades complaining that Microsoft makes inferior products and have used their size to "bully" other people out of the marketplace, but these experiences prove to me that Apple's failings are completely due to their own misunderstanding that the needs of business users differ significantly from the needs of home users.   

 

In the end, though, I still love my iPhone.  I'm just glad it's only for personal use.

 

Every journey begins with a first blog entry

As part of my daily news and learning routine, I read a lot of sites.  I read on a wide variety of topics from computers to astronomy, philosophy to finance, comic strips and news.  One of the sites I have subscribed to in my RSS reader of choice (Newsgator) is Get Rich Slowly, "a blog about sensible personal finance".  Today, I read an interesting article called 10 Essential Steps to take BEFORE You're Laid Off.  One suggestion was to start a blog to serve as my "living resume".  So, since I was taking advantage of Microsoft Live's SkyDrive, I might as well kill two birds with one stone.  So, here's my blog where I'll discuss the stuff I DO want employers to find.  :)

Hopefully, it won't be boring for you...

Harry Potter and the Order of the Visa

We're watching a show right now about the "Secrets of Harry Potter" on A&E. A good show, but something's really bothering me about it. Then, after watching the same Visa commercial for the 2000th time tonight, it hits me. It's a training exercise. Seriously, think about it. Who's going to be watching this show right now? 14-16 year olds? The perfect age to start training them that "using cash is bad". You should use your Visa to buy everything. People who pay cash are stinky and make you wait in line. Don't you hate that? Every commercial break so far has lead in with the Visa commercial and lead back with it.

How easy are we swayed?

Powered by ScribeFire.

Performancing for Firefox

So, I've decided to try and spend a little more time with my blogs. My peeps deserve that, afterall. They do love me for a reason, so I have to keep giving it up! :)

But, blogging can be a pain in the ass...enter Performancing for Firefox. It's a nifty little extension for Firefox that allows you to blog pretty easily. Just install it, restart FF and you're off. You then setup your accounts and done! It puts a little button on your statusbar and when you click it, your window splits in half and you blog in the bottom. There's even an addon to publish to Myspace blogs. Pretty cool stuff!

Foxyproxy...nice!

Been a while, huh? Wonder if anyone has me in their RSS feeds to see if I update this again. Hehehehe. Well, I'm going to try and be a bit more regular with all of my blogs, let's see how that works out.

Anyway, wanted to stop in and tell you how cool this Firefox extension is...FoxyProxy. I used to use the SwitchProxy Tool because I was using my laptop both at work and home. SPT allowed me to right-click and use the proxy at work, then right-click and use no proxy at home. Made life easier...until Firefox 1.5 came out and the developer stopped working on SPT. Fortunately, at about the same time I got a real desktop at work, and didn't need to switch proxies anymore, so I wasn't concerned...

...until a couple of weeks back when they "upgraded" the proxies at work. Now they're up, they're down, they're left, they're right. Half the time I can't even get out. Fortunately, there's five of them, so if one's down I might be able to get out on another. Back to needing a proxy switching util...FoxyProxy, how wonderful a tool. See, after upgrading the proxies, the rulesets aren't the same on them anymore. So, for example, the primary proxy I use (which is closest to my location and fastest) won't allow me to go to RPGNow to see what's new. However, the default one used by most folks will allow it. What a PITA to have to switch, no matter how easy FP makes it, back and forth between proxies for one or two sites.

Not so! FoxyProxy allows you to set patterns for URLs, so it can use different proxies depending on where you want to go. So, I use my "close" proxy for all URLs, unless it contains "*.rpgnow.com/*". Then, it uses the user-default proxy. All of this happens behind the scenes and I don't have to even think about it. Nice!

If you need a proxy solution, this is so the way to go, folks.