Migrating from Sharepoint

Hi All,

Sharepoint is a whore to get away from. But here is some useful information and how to do it, quick and dirty.

First things first, do you know where your data is? Nope! It's in the database.

This is just for interest. Run this against your sharepoint DB:

Select d.dirname, d.leafname, d.setuppathuser, a.[content] from docs d
left outer join alldocversions a on a.id = d.id
where dirname = '[your directory name in the URL]'
and a.[content] is not null

What this will do is return all the documents. The important column here is a.[content]. This is your file converted into hex. If you don't believe me copy and paste it into a hex convertor on the web or something. Paste that into notepad, save as the right file type yada yada.

Anyway, get rid of the a.[content] entries in the sql then run it again.

Acceptable loss 1. The only docs with pathsetupuser are ones with multiple versions. Single versions do not have this metadata. That is in another hex encoded column called metainfo (select metainfo from docs).

Unless you want to faff with all that, you will lose the metadata. I couldn't personally be arsed and it was an acceptable loss as we are moving to a new file storage system.

Now you need to rescue your files. You can do this by running a tool here:

c:\program files\common files\microsoft shared\web server
extensions\12\bin

This is a sample command line you will need to run:

stsadm.exe -o export -url http://[servername]/sitename]/ -versions 1 -filename d:\sharepointexport -nofilecompression

Ok, quick overview.

-o export: does what it says
-url: the base url before the /forms/allitems.aspx
-versions 1: only the latest versions of files
-filename: destination directory. This mustn't exist before you run the command.
-nofilecompression: important for us. This tool exports .dat files, with compression on we will just get one big one.

OK, run that commandline and get your files. You should have all sorts of files like:

0000000A.dat
00000004.dat
....

These are your files. You should also get an xml file called manifest. You need this as it has the metadata like the real bloody filename!

At this point, I gave up on windows. I was 3 hours in and I managed to get some help off a friend of mine.

We copied the xml file to a linux box, and used perl to parse the xml file and generate a .BAT file at the end that would rename the files and move them into the right directory. Here is the perl code:

use strict;
print "\@echo off\n";
open (my $FL, "<manifest.xml");
foreach my $line (<$FL>) {
chomp($line);
if ($line =~ /^<file url="'\">)
my $realfilename = $line;
$realfilename =~ s/.*?Name=\"(.*?)\".*/\1/;
my $directory = $line;
$directory =~ s/.*?File Url\"(.*?)\".*/\1/;
$directory =~ s/\/.*?$//;
my $currfilename = $line;
$currfilename =~ s/.*?FileValue=\"(.*?)\".*/\1/;
print "copy $currfilename \"$directory\\$realfilename\"\n";
}

on your linux box run this script and > rescuemyfiles.BAT

Right!
Copy this file to your sharepoint server, and drop it into the same directory as your rescued .dat files.
All the .dat files are, are your files. You can rename one from .dat to .jpg or whatever and they will work. All the .bat file does is grab the file info from manifest.xml and copies them into a directory for you.

So there you have it. How to resuce your files from sharepoint. Have fun!

thanks for reading,

Trev
p.s. I never said this was clean did I? :-)

 

VMWare VCB backups

Hi all,

When backing up many virtual machines, you may try one of the big backup solutions, such as Symantec, Commvault, Veeam, and the like. Unfortunately in this case they all rely on one thing.

THE VMWARE API

There is a problem with the API. VMWare are aware there is a problem with it (calls were logged late last year about the problem) but so far there has been no response at all.

The problem is that when a backup of a VM happens, it takes a snapshot of the machine.
The machines base disk and other chuff is backed up, then the snapshot is released. What is happening (to many people) is that the snapshot is removed from the snapshot manager, but the snapshot file is not deleted, and the deltas are still written to the snapshot.

Luckily(!) for me, my luns filled up and bombed out after 2 snapshots. There are reports out there that if you have 25 of these "ghost snapshots" then you start running into problems. For example redo log errors. The fix is easy enough, although a pain in the arse. Basically just run the convertor against it, and itll roll all the snaps together. Don't faff with the command line unless you really have to.

Anyhew, there is one option. In version 4.1 VCB still works. So you can use that. I have written a script you can use, usual terms apply. By usual terms I mean, if this fucks your environment its not my fault. Lose data, it's not my fault.

Here is the code:

Dim fso, ts, weight
weight=0
Const ForWriting = 2
set ofso2 = createobject("scripting.filesystemobject")
set ofiletemp = ofso2.opentextfile ("servers.txt", 1)
set cline = createobject("wscript.shell")
'----------------------------------------------------------------------
do while not ofiletemp.atendofstream
weight = weight + 1
servername = ofiletemp.readline
if weight < 6 then
'msgbox servername
fullpath = "vcbmounter -h [yourVCname] -u username -p Password -a name:" & servername & " -r e:\vmbackups\" & servername & " -t fullvm -m nbd"
cline.run fullpath, 1, False 'lets another copy open "NBD!!"
else
fullpath = "vcbmounter -h [yourVCname] -u username -p Password -a name:" & servername & " -r e:\vmbackups\" & servername & " -t fullvm -m nbd"
cline.run fullpath, 1, True 'waits for this one to finish before carrying on
weight = 0 'resets weight to 0
end if
loop

What you need to do is put this file in the VCB directory in c:\program files\vmware\vmware consolidated backup/

Create a servers.txt file that has a list of your servers in, one per line.

If you look in the code there is a file path to e:. That is the destination directory. Change it to a destination you like.

Once this script has finished, you will have a directory in your detination dir, for every machine, filled with the files that make up that machine. Back these machines up however you want, or just have them as a handy copy.

When you are finished, run this script:

Dim fso, ts, weight
weight=1
Const ForWriting = 2
set ofso2 = createobject("scripting.filesystemobject")
set ofiletemp = ofso2.opentextfile ("servers.txt", 1)
set cline = createobject("wscript.shell")
'----------------------------------------------------------------------
do while not ofiletemp.atendofstream
servername = ofiletemp.readline
'msgbox servername
fullpath = "vcbmounter -h [yourVCname] -u userid -p password -U e:\vmbackups\" & servername
'msgbox fullpath
cline.run fullpath, 1
loop

This will remove all those machine directories. Again, don't forget to change the file path beginning "e:\" in the script. This also needs to live in the VCB directory mentioned above.

This works without creating ghost snapshots. It won't work at all against ESX5.0 as far as I know.

Thanks for reading,

Trev