My Nightmare

[Photo-John]

Due to poor archiving workflow (none), and working too fast, I am now scanning a hard drive trying to recover about 20 GB of accidentally deleted original RAW files, JPEGs, and layered TIFF master files. My system was not foolproof. Because this fool done screwed up good

Wish me luck. I will write an article when I'm done.

[mjs1973]

Good luck PJ. I lost a few files a couple years ago and it sucked. I didn't loose much, but what I did loose meant a lot to me. My Grandma at Christmas, my brother in law leaving for Iraq...

I still don't back up my images as soon as I should, but I make sure I have 3 coppies before I delete the originals from my HD.

[SmartWombat]

Ouch.
Made me look, discovered in the NAS software it has a Deleted Files option.
Just enabled that with a 2GB limit on the folder.
Plus I have backups now

[drg]

Due to poor archiving workflow (none), and working too fast, I am now scanning a hard drive trying to recover about 20 GB of accidentally deleted original RAW files, JPEGs, and layered TIFF master files. My system was not foolproof. Because this fool done screwed up good

Wish me luck. I will write an article when I'm done.

I feel your pain. Good luck. Still have to post myself about last fall. A little different issue but a continuing baaaad memory.

[Photo-John]

I feel your pain. Good luck. Still have to post myself about last fall. A little different issue but a continuing baaaad memory.

That others might learn from my mistakes...

[Photo-John]

I pulled my drive, attached it to another machine, and scanned it with R-Studio file recovery software. I recovered thousands of TIFF, JPEG, and Canon RAW files. I don't think I recovered any Olympus RAW files and I shot a whole day of racing a couple of weeks ago with the Olympus. Furthermore, all of the Canon RAW files appear to be corrupted. So far, I haven't been able to open them with any software I have

I have tons of files to sort and view now. And I have to research and buy some network storage with redundancy. No more procrastination.

[jorgemonkey]

Ouch. I sorta went through the same thing last year when my external drive crashed. At that time I had all photos over 2 years on the external drive along with backups of all my current ones. The new files stayed on my internal drive.

Well power outage came and it fried the drive, and I didn't have the $1200+ I was quoted to pull the photos off. Luckily I had 2-3 copies of DVD backups, and those saved my butt. The crappy thing is I lost about 3-4 hours of MTB video footage I shot as well as a bunch of music videos.

Now I have (and need to back up!) my files on my internal drive, and mirrored copies on 2 different external drives. One is always plugged into the computer, the other is not plugged in, and they get rotated each backup. That way in case my comp gets fried I has still have copies.

[SmartWombat]

John, I can offer you some temporary space.
I'll clear off a terabyte for you, access via FTP.
I just got it working.

[readingr]

John,

My heart sank when I read this. I hope you manage to get most of your images back. Best of luck. I did this for a customer some years ago. It's not easy and not guaranteed.

From that experience I implemented a work flow which means a DVD copy as and a HDD copy as soon as they come out of the camera. I also have an automatic incremental backup set on a 750Gb offline disc which I plug in every night.

Will be buying another soon to start a new full backup. So along with all the DVD's and HDD I'm safe.

Roger

[Photo-John]

None of the RAW files will open. And all of my Olympus RAW files are MIA. It may be that the software didn't recognize the .ORF extension, though. So I think I'm screwed for anything but images I edited and resaved as TIFFs. And the layers are gone on every TIFF I've opened. Putting things in perspective, it could have been much, much worse. I only lost two or three events. If a drive had died, I might have lost a whole year. And now I'm really motivated to put a new system in place.

I have not been inclined to backup to DVD or CD because of the lifespan of the plastic. However, it would have saved me in this case. So it may be a good temporary strategy to implement.

What kind of hard drive system are you using?

[readingr]

What kind of hard drive system are you using?

Not sure if your asking me or Paul so I'll answer for now.

For the Incremental backup I have Maxtor 750Gb USB disk
For the photos which I use SyncToy to back up ans synchronise with I have a Western Digital MyBook 500Gb and a 400Gb disk both USB.

I also have another PC in another room which is being built with LINUX and will become a file store as well.

The DVDs are duplicated and one set live with my parents in Wales just in case of a disaster (fire). Their 170 miles away.

Roger

[SmartWombat]

I am using a ReadyNAS NV+ it's a Network Attached Storage box.
I found them at PMA in 2007 and I'm so happy, I bought a second and then a third one.
They are automatic RAID.

Put in one 1TB disk and it's just an ordinary (yes and expensive!) drive with a 1Gbit network interface - plus lots of other features, see later
Add a second drive, and it automatically mirrors the disk, so you have an instant backup.
Add a third drive and it starts building a RAID array.
Add the 4th drive and it's a fully redundant RAID 5 array.
So at every stage, if one drive fails - the unit keeps working and will automatically heal itself when you plug in a new drive.

If you start out with small disks and fit bigger ones, you can exchange them one at a time without turning off the power, it just resynchronises the disks.
Then after you swap the last disk, it will automatically expand the array on the next reboot.

It's simple !

It even has a Deleted Files system on the drive (not something you normally get across a network) which is not operating system dependent. Just configure an amount of disk space it can use, and a number of days for retention.

It supports Apple AFS, Linux with NFS, Windows with CIFS, rsync for backups.
But it's not just storage ... also FTP for file transfers, HTTP so you can put up web galleries, Slimserver, iTunes streaming server, UPnP AV, Home Media Streaming Server.
It also supports UPnP discovery and Bonjour service.
Oh yes and it's a printer server too.
Two USB ports for printer or UPS battery backup notification, and a front panel USB port for backup.


I haven't switched on my stack of 6 USB external hard drives for a year now.

[prophotolife]

What a bummer about the files!

The RAID setup sounds amazing, definitely the way to go. We use a storage box with multiple drives but haven't set up an array. It sounds like we shouldn't keep putting it off.

I started backing up to CD/DVD "back in the day", before hard drives were so inexpensive. Out of habit I still burn RAW files to CD/DVD immediately after a shoot (but backup on hard drive, also). The interesting thing I've found is that we all recognize the plastic will have a limited life span...but I've yet to have one fail on me. A few hard drives have gone down in that time, but no data has been lost or corrupted on the CD/DVDs. I'll keep burning to both, just out of habit.

[Photo-John]

Welcome, prophotolife!

I am still putting off a decision about a better long-term strategy. It's definitely going to involve a separate box and redundancy, though. I've looked hard at the ReadyNAS system that Paul's using. And I've talked to a local builder, who built my laptop, about them building me a RAID server. I've also been looking at the DROBO boxes, since they are relatively inexpensive and I've been hearing lots of good things about them. Part of the reason I was waiting was money. I have my tax refund now and just need to make up my mind about what to do.

[gahspidy]

What a bummer about the files!

The RAID setup sounds amazing, definitely the way to go. We use a storage box with multiple drives but haven't set up an array. It sounds like we shouldn't keep putting it off.

I started backing up to CD/DVD "back in the day", before hard drives were so inexpensive. Out of habit I still burn RAW files to CD/DVD immediately after a shoot (but backup on hard drive, also). The interesting thing I've found is that we all recognize the plastic will have a limited life span...but I've yet to have one fail on me. A few hard drives have gone down in that time, but no data has been lost or corrupted on the CD/DVDs. I'll keep burning to both, just out of habit.

I agree with you here. The plastic has no moving parts or electrical circuits to burn out. I always back up my cards to a dvd/s and then the external drive. Simpler the better.
I will be keeping my steel box of DVD's at another location than my computer and the external

[fx101]

I agree with you here. The plastic has no moving parts or electrical circuits to burn out. I always back up my cards to a dvd/s and then the external drive. Simpler the better.
I will be keeping my steel box of DVD's at another location than my computer and the external

Hard drives are quite prone to failure because of their relative complexity.

Photo-John have you considered a Solid-State-Drive backup system? Solid state memory although slower, tends to be far more reliable and less prone to read/write errors than a traditional HDD. I recently bought 6 64 GB SSD discs for three separate redundant RAID arrays. 192GB is not enough for my entire collection, but I'll consider adding to my backup server if it works well in the next month. Currently I have a system similar to the one SmartWombat described (albeit without the terabyte ) that gives me 500GB of redundant RAID space. Again, if the SSD drives work out I might consider setting them up to work as the redundant drives behind the traditional ones (because the traditional drives are faster). I would worry strongly about keeping all your backups as DVD's because one big scratch and there go your photos .

[Photo-John]

Thanks for jumping in, fx101. I have not considered solid state drives. And based on the capacity you've mentioned here, they wouldn't do the job. I think I will probably be up to about half a terabyte of image files a year this year. I still haven't pulled the trigger on anything. I am very curious about the Drobo, the ReadyNAS system that Paul and my friend Alan are using. But I'm leaning towards having a local computer company that I've used before build me something based on the ReadyNAS specs. I like the idea of solid, local support.

As for DVDs - as a quick and temporary solution, they seem fine. But I am very suspicious about the longevity of the plastic. I think that's the weak link. I would hate to be archiving to CDs or DVDs and then find that the substrate deteriorated leaving me with nothing at all.

[SmartWombat]

Not just the substrate, but the recordable dye media itself is an issue.
If you write on the discs with the wrong sort of pen you can kiss your data goodbye too.
This is why I now only mark the centre clear portion where there's no data.

[fx101]

Thanks for jumping in, fx101. I have not considered solid state drives. And based on the capacity you've mentioned here, they wouldn't do the job. I think I will probably be up to about half a terabyte of image files a year this year. I still haven't pulled the trigger on anything. I am very curious about the Drobo, the ReadyNAS system that Paul and my friend Alan are using. But I'm leaning towards having a local computer company that I've used before build me something based on the ReadyNAS specs. I like the idea of solid, local support.

As for DVDs - as a quick and temporary solution, they seem fine. But I am very suspicious about the longevity of the plastic. I think that's the weak link. I would hate to be archiving to CDs or DVDs and then find that the substrate deteriorated leaving me with nothing at all.

The ReadyNAS system is supposed to be very good, I've never used it but the specs make it out to be very intuitive and foolproof. Your local computer company should be able to make you a nice multi HDD backup server. I only have around 450GB so 1TB of space suffices for me, but if they can make you a nice redundant array server that you can connect to your computer via Fibre Channel (to a controller card) and host on the internet, then you have a nice and easy networked storage system. Local support is nice but overrated. Usually when something breaks it isn't something your local guys could fix.

As for DVD's SmartWombat is right on. The dye media is much more sensitive to light than commercial (non DVD-R) DVD's because of the lower power lasers in your drive. Effectively this means that under light and moisture data loss will begin in less than 10 years, with MTBF times for DVD's usually ranging around 20 years or so. That's IF you're careful. Use your DVD's a lot and the small scratches and finger oils on the surface will drastically reduce their longevity.

Also, have you considered utilising a tape backup system?

[SmartWombat]

I looked at tape backup to backup my NAS and it was about twice the cost !

[Alison]

OMG! John that IS a NIGHTMARE! Good luck!!!

[fx101]

I looked at tape backup to backup my NAS and it was about twice the cost !

Really? The company I'm interning at (a governmental research firm) recently purchased some Dell rack-mount drive libraries for $2500 a pop with each one offering 1TB of storage.

[SmartWombat]

Yes, and in the UK that'll be £2500 which is twice what I paid for a 3TB NAS.

[gahspidy]

As for DVDs - as a quick and temporary solution, they seem fine. But I am very suspicious about the longevity of the plastic. I think that's the weak link. I would hate to be archiving to CDs or DVDs and then find that the substrate deteriorated leaving me with nothing at all.

Absolutely agree. I would not put all my eggs into plastic but for quick immediate backup their cheap and readily available. . . and quick. I keep them around and stored because who knows, one day they may be all I have and I'll be glad they held up if they do.

Just curious, does anyone use those hand held storage/viewer devices that Epson and some other companies have out? They of course are not long term solutions but for immediately backing up cards on the spot from a photo shoot and being able to view them on a larger higher res screen seems like a cool thing to have.Alot of them seem to work with camera RAW files as well, which would be important.

[fx101]

Absolutely agree. I would not put all my eggs into plastic but for quick immediate backup their cheap and readily available. . . and quick. I keep them around and stored because who knows, one day they may be all I have and I'll be glad they held up if they do.

Just curious, does anyone use those hand held storage/viewer devices that Epson and some other companies have out? They of course are not long term solutions but for immediately backing up cards on the spot from a photo shoot and being able to view them on a larger higher res screen seems like a cool thing to have.Alot of them seem to work with camera RAW files as well, which would be important.

I recently bought a Hyperdrive Colorspace 0 (80GB) made by Sanho and I am quite happy with it. If you want ultimate speed then get the HD80 (80GB in 2 minutes) if not this one backup up a 4GB CF card in around 3 minutes but lets you preview the images. Generally I have multiple CF cards so I just let it back up my things in my bag while I shoot with another card so speed isn't a real issue. They are really reliable although they are not really meant for "storage" type backup, just to limit the number of CF cards you have to carry around.