VMWare - THE AUDITORS ARE COMING!

Hi All,

The Auditors are coming!

Here are 2 handy scripts for you to run against your VMWare database. Written for Oracle, but will probably work for MSSQL too.

This one, returns the OS type and name of a virtual machine, and the host it runs on:

select v.DNS_name, h.dns_name, v.guest_os

from vpx_vm v

inner join VPX_HOST h on h.id = v.host_id

where v.DNS_NAME is not null

order by h.dns_name;

 This one returns build version, name, boot-time and some blank and prefilled columns (because of the spreadsheet we had to use) You will need to replace [VCNAME] with your Virtual Center name. Funny that

select l.product_name, l.edition_name, l.product_version, '', h.boot_time, h.name, '[VCNAME]', '', '', 'Production', 'Location', '', '', '', '', h.cpu_core_count / h.cpu_count, h.cpu_count, h.product_name from vpx_lic_assets a
inner join vpx_lic_context c on a.asset_id = c.asset_id
inner join vpx_lic_licenses l on c.license_id = l.license_id

inner join vpxv_hosts h on a.name = h.dns_name;

This should help with the most basic questions. Other people want more, or less info.
Some useful tables/views:

Tables:
vpx_lic_licenses
vpx_lic_assets
vpx_lic_context
vpx_vm
vpx_host 


Views:
vpxv_hosts



Thanks for reading,

Install and configuration of SRM 4.1

Hi All,

Configuration of SRM:

For the initial config of SRM, I installed it onto the same server as the Virtual Center. I am only supporting 25 VMs at these early stages. I won't cover the installation procedure here as its a piece of cake, what I will say is that you need to set up your database, login and you will also need to go into ODBC and set up your database connection before running setup.

Once this has been done, log into your virtual center server through the client.

Click on plugins, then install the vCenter Site recovery manager extension.

I'm going to take you into the GUI for this so you can see what needs configuring.

Once it has installed, you will have another tree under "Solutions and Applications"
Go into Site recovery.

• You need to provide a login for the local (primary) site and the paired site (DR)
• You also need to create a connection between the 2 sites
• You need to provide a driver and login details for your Storage device.
• Inventory mappings are source folders/datacenters and destination datacenters/resource groups
• Protection Groups are bunches of machines carved up into lumps. The lumps depend on various things like OLAs, applications, server teams and other logical groups

First things first then.

Create a new user on the SRM (live site) server.
I did this simply by creating a local user on the VC server. If you have a seperate SRM server, you'll need to create a domain account. It will need full admin priviledges on VMware though.

Create a new user on the SRM (DR/Failover) server. Same as above.

Go into the site recovery manager, click on "Site Recovery" on the tree view on the left hand side.
In the main pane, under "Protection Setup", click configure.

Follow the prompts. All you are doing is putting the user accounts in that you set up above.
Once that is done, you should notice that the local site and paired site boxes are populated. If they aren't, something has gone wrong.

Next step:

Array managers. Storage, basically.

1. Set up your LUNs and your snapmirrors/data protection transfers/remote clones, whatever your storage vendor wants to call it this week. This needs to be done first as when we get to the next bit it scans your storage to get this information. If you don't do this bit first, you'll have to run the same scan again and depending on the number of luns, you could be sat there for quite some time.
2. You will need to provide a user account for both the live and DR sides of your storage arrays. It will need to ability to snapclone a replicated lun (for testing) and to break a mirrored volume, delete a volume etc... pretty low level stuff.
3. Pop onto the vmware website and search for your storage vendors driver for SRM. Download the right ones for you (either block or CIFS or both) and copy it to your SRM server. They should come as MSI packages, or self-installing executables so run them.

Once they are installed restart the SRM services.

Back to the Vsphere client.

Click the configure button next to array managers
Click Add

fill in the IP address details and the username and password for your primary storage.
It will refresh and you should see the array ID and the device count. (device count is the number of luns or presented storage from that device.)

Add all your storage arrays like that, then hit next.

Same again, this time adding your secondary (DR) sites storage username and password

press next and this should show you a list of all replicated data stores. If you have replication switched on, you should get a list of datastores. Hit Finish.


Nearly there!

Inventory Mappings - hit configure.

Here is where you map you current live directories/networks/clusters etc... to the DR site resources. Easy enough done if you map things the same either side.

Protection Groups.
First thing you need to do, is to go onto your DR storage and present a small LUN where you can put your placeholder data files for all your protected machines. Then go back to the VClient

Click Create to create a protection group
This is volume based VM protection groups, so choose your volume
Choose where to put your placeholder data files (you're new lun would be great)
then click finish.

You're done. Now its time to create a basic plan. Plans are always created on the secondary or DR side.

• Point your VClient at your secondary VC and login.(oh quick stop here....look at all those machines!!! Don't turn them on, they are placeholders)
• Click solutions and Applications -> Site Recovery
• Click recovery plans. (we'll just do a very, very basic one. Although if your secondary and primary sites match exactly, this plan would be amazing for you)
• Click create
• Give it a name click next.
• Choose a Protection Group
• Click next - Leave the timeouts as default as we won't be doing either.

Test networks. Right, if you run the test plan, then SRM will either create "fake" vswitches, or you can make it do the "real" networking. Map the networks to the correct type here. Auto means create a non-uplinked test switch.

Suspend local VMs - if your secondary site hosts test or development machines, they rank below production machines that are going through DR process. You can configure SRM to suspend these machines, releasing those resources.

Click finish.

Want to test it?

Choose a recovery plan in the tree view on the left.
Under the summary tab you can see some buttons.
Click Test.

There you go, all done.
If you want to see what its doing, log into another client session.
On your original session drive SRM from the recovery steps tab.
Click it and click the test button (top left)
Now you can see what steps its got to. On your second session you can see what the machines are doing.

There you have it.

Basic failover in SRM.


Thanks for reading

Solaris and ZFS Including ISCSI target recovery

Hi all,

I had a rough day the other day. I was using Nexentastor to present iscsi storage to my VM Test Environment, and the repository.db file became corrupt. After a reboot the server failed to boot. At all.

I don't like Nexenta. The GUI, its main selling point, keeps crashing and I have a lack of faith in it. So I thought I'd give Solaris 11 a punt. Here is how I recovered my ZFS luns and presented them.

First, install Solaris. This is a piece of cake, just make sure you don't install solaris on disks being used by your ZFS volumes.

Once you have your OS up and running, sort your networking out. I installed the Gnome version so I did this using the GUI tool. I will cover aggregates and command line networking in another post.

If your storage is DAS based, ie its directly attached to your Solaris server, and you have more than 15 SAS disks and you can't see the others, you will need to modify one of the OS files and reboot.

The filename is /kernel/drv/sd.conf
You should see lines like this:
name="sd" class="scsi" target=16 lun=0;

Add more lines for the number of disks you have, incrementing for each line, save and reboot.

Next thing is to rescue your ZFS volumes.

Running zpool import will scan all your disks and report back on the volumes it has found. It won't import them. to actually import them run this command:

zpool import -f [poolname]

To see your imported pool type

zpool list -L

I have 3 pools, 3 volumes, 1 per pool. When I did mine, I only "rescued" one volume or pool at a time, then presented them, then did the next one. You can do yours however you want.

Next thing is to install the iscsi target software. If you are hiding behind a proxy, the easiest way of doing this is to open a terminal, then do this as root:

export http_proxy=http://username:password@yourproxy.fqdn:portnum

Then run the following:

pkg install storage-server SUNWiscsitr

Once it has installed, we can start presenting volumes. First off, we need to identify the shares. You should know what they were called. Run this command:

sbdadm import-lu /dev/zvol/rdsk/[Volumename]/[sharename]

That will import your lun. Then run:

sbdadm list-lu

This will list your luns. The number you will need is the GUID so copy it and get ready to paste.
Run this command:

stmfadm add-view [GUID]

This will share your iscsi lun with every device but also from every IP on your solaris server. This is fine for me, you may need to lock it down. I haven't investigated this as I don't need it.

Next you need to create the target process itself. Run these 2 commands to complete the storage side config:

svcadm -l target
itadm create-target

check it with:

itadm list-target

Thats you done, sort your initiators out now on your other servers. I haven't covered creating new volumes, luns or whatever. I'm hoping you find this article when you are in the shit and need some quick answers!

Thanks for reading,

Trev

Help! Some dick deleted the Author for the default sharepoint site!

Hi All,

While I was doing the below, I had a problem.
Someone had deleted the default author for a site, this stopped the stsadm tool running.

Do this nice DB stuff to get it working.
First:

Select author, siteid from webs where fullurl =
'[yoursite]'

Remember the siteid!

run this one:

Select * from userinfo where tp_siteid='[that really long code you
just remembered]'

Now choose another user who is active and remember the tp_id

Now run this one:

update webs
set author = [that tp_id you remembered for the
user]
where fullurl = '[yoursiteid]'

Thanks for reading

Trev

Migrating from Sharepoint

Hi All,

Sharepoint is a whore to get away from. But here is some useful information and how to do it, quick and dirty.

First things first, do you know where your data is? Nope! It's in the database.

This is just for interest. Run this against your sharepoint DB:

Select d.dirname, d.leafname, d.setuppathuser, a.[content] from docs d
left outer join alldocversions a on a.id = d.id
where dirname = '[your directory name in the URL]'
and a.[content] is not null

What this will do is return all the documents. The important column here is a.[content]. This is your file converted into hex. If you don't believe me copy and paste it into a hex convertor on the web or something. Paste that into notepad, save as the right file type yada yada.

Anyway, get rid of the a.[content] entries in the sql then run it again.

Acceptable loss 1. The only docs with pathsetupuser are ones with multiple versions. Single versions do not have this metadata. That is in another hex encoded column called metainfo (select metainfo from docs).

Unless you want to faff with all that, you will lose the metadata. I couldn't personally be arsed and it was an acceptable loss as we are moving to a new file storage system.

Now you need to rescue your files. You can do this by running a tool here:

c:\program files\common files\microsoft shared\web server
extensions\12\bin

This is a sample command line you will need to run:

stsadm.exe -o export -url http://[servername]/sitename]/ -versions 1 -filename d:\sharepointexport -nofilecompression

Ok, quick overview.

-o export: does what it says
-url: the base url before the /forms/allitems.aspx
-versions 1: only the latest versions of files
-filename: destination directory. This mustn't exist before you run the command.
-nofilecompression: important for us. This tool exports .dat files, with compression on we will just get one big one.

OK, run that commandline and get your files. You should have all sorts of files like:

0000000A.dat
00000004.dat
....

These are your files. You should also get an xml file called manifest. You need this as it has the metadata like the real bloody filename!

At this point, I gave up on windows. I was 3 hours in and I managed to get some help off a friend of mine.

We copied the xml file to a linux box, and used perl to parse the xml file and generate a .BAT file at the end that would rename the files and move them into the right directory. Here is the perl code:

use strict;
print "\@echo off\n";
open (my $FL, "<manifest.xml");
foreach my $line (<$FL>) {
chomp($line);
if ($line =~ /^<file url="'\">)
my $realfilename = $line;
$realfilename =~ s/.*?Name=\"(.*?)\".*/\1/;
my $directory = $line;
$directory =~ s/.*?File Url\"(.*?)\".*/\1/;
$directory =~ s/\/.*?$//;
my $currfilename = $line;
$currfilename =~ s/.*?FileValue=\"(.*?)\".*/\1/;
print "copy $currfilename \"$directory\\$realfilename\"\n";
}

on your linux box run this script and > rescuemyfiles.BAT

Right!
Copy this file to your sharepoint server, and drop it into the same directory as your rescued .dat files.
All the .dat files are, are your files. You can rename one from .dat to .jpg or whatever and they will work. All the .bat file does is grab the file info from manifest.xml and copies them into a directory for you.

So there you have it. How to resuce your files from sharepoint. Have fun!

thanks for reading,

Trev
p.s. I never said this was clean did I? :-)

 

Mythtv - Perl script to email the recordings list.

Hi All,

So, I don't want my new recordings list being available on t'interwebs, but I do have my own email server.

So I wrote some perl to parse the RSS feed and email me the results. I stuck this to a cron job and now I get a weekly email with the new recordings on.

It does need tweaking, I only want the newest recorded programs email to me. Currently this emails the lot to me. Anyway, here's the code:

#!/usr/bin/perl

use LWP::Simple;

#to read html streams

use strict;

use Mail::Sender;

my $emailmsg;

my $sender;

print "Getting Content\n";

my $content = get('http://localhost/mythweb/rss/tv/recorded') || die print "Oh fuck it";


#print $content;


$content =~ s/[^[:ascii:]]+//g; #get rid of weird chars

print 'Splitting atoms...oh LOL :-)';

print "\n";


my @lines = split(//, $content);


print "Parsing some stuff\n";

for my $line (@lines){

#print $line =~ /\(.*)\<\/title\>/;

$line =~ /\(.*)\<\/title\>/;

$emailmsg .= $1;

$line =~ /\(.*)\<\/pubDate\>/;

$emailmsg .= " - " . $1;

$emailmsg .= "\n";

#print $line =~ /\\<\!\[CDATA\[(.*)\]\]\>\<\/description\>/;

$line =~ /\\<\!\[CDATA\[(.*)\]\]\>\<\/description\>/;

$emailmsg .= $1;

$emailmsg .= "\n\n\n";

#print $emailmsg;

}



#(my $headlines) = ($content =~ /type\=\"html\"\>(.*)\<\/title\>/);

#print "$headlines\n";

$sender = new Mail::Sender {

smtp => 'IP or server address',

from => 'address@tosend.from',

auth => 'NTLM',

authid => 'username',

authpwd => 'password',

on_errors => undef,

}

or die "Can't create the Mail::Sender object: $Mail::Sender::Error\n";

$sender->Open({

to => 'who@tosendit.to',

subject => 'Mythtv recordings update'

})

or die "Can't open the message: $sender->{'error_msg'}\n";

$sender->SendLineEnc("$emailmsg");

$sender->Close()

or die "Failed to send the message: $sender->{'error_msg'}\n";


There you go. I've left my commented code in so you can see where I was going with it.

Thanks for reading,

Just upgraded your VMWare vmhost from 3.5 to 4.0 or 4.1? Read on

Hi all,

If you have recently upgraded your VM Host to ESX4.0, 4.1 or 5.0 you might want to run this code (after installing the vmware powershell extensions, this is a powershell script after all):

function reportchangetracking{

$vm_name = Get-VM -location [cluster_name] | get-view

$vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec

foreach ($objitem in $vm_name) {

#Write-host $objitem.name, $objitem.config.changetrackingenabled

if($objitem.config.changetrackingenabled -neq "True"){

$vmConfigSpec.changeTrackingEnabled = $true

$vmView.ReconfigVM($vmConfigSpec)

#punch it Chewy (reloads config spec. better than having to shut the machine down)

sleep 3

Get-VM $objitem.name | New-Snapshot -Name "Temp"

sleep 5

Get-VM $objitem.name | Get-Snapshot | Where {$_.Name -eq "Temp"} | Remove-Snapshot -Confirm:$false

}

}

}

In ESX3.5, VMTools never quiesced the base disk properly when taking snapshots. This meant that when Windows Server 2008 came along with VSS, the tools didn't use it, and 3.5 never added in the setting the code does above.

When you upgraded your host, it never added this setting at the VM level, so this task needs to be done manually. If you have a lot of machines, headaches ensue. After running the script, it might be an idea to update the tools installation on your server anyway.

Note that you do not have to run this if you created a machine on ESX4.0 or greater. Its been done for you.

Use this script at your own risk. It's no fault of mine if you break something.

Thanks for reading,

Trev