User Panel
Posted: 2/3/2021 8:03:12 PM EDT
My photo archive is ever growing and backing it all up continues to increase in difficulty, I was hoping somebody else has figured out a better solution than I have
|
|
[#3]
|
|
[#5]
I have stuff like that on a file server on my local network. I use iDrive cloud backup service to protect it in case the server crashes. If you really want to keep those pics, use a good cloud backup service.
|
|
[#6]
14 TB USB 3 Hard drives are $250, $300 for 16TB. Stack them deep, never save on just one. Keep short-medium term storage on NAS (RAID Box on network with lots of space), Offload "older once a month to offline USB 3 drives (USB 3 is really speedy for that now). The lower cost external drives are not meant for continual updates, just keep the backups in a safe, two is one, one is none (online backup is also none)
Keep the USB storage unplugged and off network in the event of ransomware. RAID just helps them encrypt/delete more data. I've had good luck with these, raw filesystem dump with Aeomi backup manager is what I use, lots of features and abilities for a lower cost, also can image your system to the online RAID backups. I just posted this in GD and the prices are $50 higher now, but they'll go back down, just make sure a USB 3.0 or 3.1 spinning/platter hard drive for archive storage. Good and common NAS box / media server RAID and 4 Bays, it's a DIY "Home Cloud" is what it amounts to, remote access options are available (not suggested if also doing other backups on it). Get 4 x 10TB drives for above NAS WD Red are built for continual running NAS applications. Speed and cache size matter on these compared to the backup drives (which are Hopefully write only in ideal world ). |
|
[#7]
What kind of data footprint are you talking?
Local NAS are great but without off-site storage (another physical location) it is incomplete. In addition to local storage and removable drives I use a cloud service … and until a few weeks ago I'd have said AWS on a Glacial tier @ $4 per TB a month I am now looking at https://tresorit.com/individuals as a start cuz noBezos when I can. For stuff I shoot on my iPhone I just use Apple Photos and iCloud and it covers all the personal stuff. For all client work (not just photos) I have a separate setup. The stack looks like this: Desktop (only copies if I am editing, not the original) External Volume Disk or SSD Library on a removable dock (https://www.amazon.com/gp/product/B073W4YDFQ/) + Mirror of each External volume (so I buy 2 drives everytime I need new space, same dock as above) + Cloud mirror of External |
|
[#10]
Quoted: Seriously. Delete that shit. Nobody cares about your junior high school yearbook pics or your first dirt bike. You can post your coolest teenage-adventure memory pic to FecesBook and you'll get three likes. All from people who were not even remotely involved in the subject of the pic. As for "family photos," try giving that stuff away to family. They don't give a shit. Ask me how I know. Porn can be replaced. View Quote I don't think he was looking to have a probable hobby/pastime psychoanalyzed. |
|
[#11]
|
|
[#12]
|
|
[#13]
Quoted: I should clarify I think. I'm interested in the complete system. From primary storage to backups. View Quote View All Quotes View All Quotes Quoted: Quoted: Home NAS I should clarify I think. I'm interested in the complete system. From primary storage to backups. Can cost some coin. Synology has a solution that seems pretty simple. Get two NAS boxes from them. Put one at a friend/relative's house. Store your photos on the NAS and you can configure it to auto-backup to the remote NAS. |
|
[#14]
Quoted: What kind of data footprint are you talking? Local NAS are great but without off-site storage (another physical location) it is incomplete. In addition to local storage and removable drives I use a cloud service … and until a few weeks ago I'd have said AWS on a Glacial tier @ $4 per TB a month I am now looking at https://tresorit.com/individuals as a start cuz noBezos when I can. For stuff I shoot on my iPhone I just use Apple Photos and iCloud and it covers all the personal stuff. For all client work (not just photos) I have a separate setup. The stack looks like this: Desktop (only copies if I am editing, not the original) External Volume Disk or SSD Library on a removable dock (https://www.amazon.com/gp/product/B073W4YDFQ/) + Mirror of each External volume (so I buy 2 drives everytime I need new space, same dock as above) + Cloud mirror of External View Quote About 4 TB of photos currently. I'm using a NAS for storage but backup is really where I'm struggling. Cloud storage pricing for 4 TB isn't insane, but it's not cheap either, and I only expect it to increase. Having backups on additional hardware seems more cost effective, but from my understanding cold storage of hard drives isn't reliable. A secondary off site NAS seems like the best option, but that requires roping somebody into letting me keep my stuff on their property and the inconvenience of that. |
|
[#16]
I'm on a Mac, and use an external enclosure that holds 4 drives (either HDD or SSD). Basically, the only thing on the internal Mac drive is the OS, and programs. The internal drive is cloned (bootable) to one of the external drives each week. All images go to the external drive, and are backed up with Time Machine.
If the house burns down.....so be it. |
|
[#17]
I have HDD and SSDs. But I came across a Google search that SSD do wear out, so a note of warning and caution. I store the same in my house. IF I get hit by a meteor, I am in trouble. Actually, I should look into cloud storage though, in additon to my on-site backups
|
|
[#18]
I have a PCIE SATA III 8-port expansion card, which in conjunction with my motherboard, gives me a total 16 SATA drive connections. So, I have 2 5-bay hot swap cages sitting below my desktop where mirrored drives and additional storage drives reside. 5 drives in the desktop, and 10 hotswap bays below it (4 currently occupied). I have to leave at least one hotswap bay empty and do period backups to that bay, and pull the drive for safe keeping. Just prior to a month long trip out west this fall I pulled my backups and stopped by my neighbor's house to drop off the Pelican case, but they weren't home. Drove past the wife while she was jogging, but handing off a Pelican case to her out the window of the truck just seemed, well, sketchy. So I just roadtripped with my backups in the truck. One of these days, I need to move to cloud backup ... it's not the expense ... they can do it for about what it costs me to add my own physical backup drive space ... it's the transfer time that puts me off. Can't imagine how long it would take to do the initial transfer of 12TB. I dunno. One day, I'll cloud the backups. Until then, I'll be squirreling a Pelican case with backup drives. |
|
[#19]
Quoted: I have a PCIE SATA III 8-port expansion card, which in conjunction with my motherboard, gives me a total 16 SATA drive connections. So, I have 2 5-bay hot swap cages sitting below my desktop where mirrored drives and additional storage drives reside. 5 drives in the desktop, and 10 hotswap bays below it (4 currently occupied). I have to leave at least one hotswap bay empty and do period backups to that bay, and pull the drive for safe keeping. Just prior to a month long trip out west this fall I pulled my backups and stopped by my neighbor's house to drop off the Pelican case, but they weren't home. Drove past the wife while she was jogging, but handing off a Pelican case to her out the window of the truck just seemed, well, sketchy. So I just roadtripped with my backups in the truck. One of these days, I need to move to cloud backup ... it's not the expense ... they can do it for about what it costs me to add my own physical backup drive space ... it's the transfer time that puts me off. Can't imagine how long it would take to do the initial transfer of 12TB. I dunno. One day, I'll cloud the backups. Until then, I'll be squirreling a Pelican case with backup drives. View Quote I hear ya on the cloud stuff. I was testing speeds with Amazon Photos and Crashplan Pro and it would take forever to get everything uploaded. Download would be faster, but it would still be awhile. I know Backblaze offers to send you your data on a hard drive (only 3.5TB at a time though) if you need to recover data, but I'm not a fan of their app for the personal backup and their B2 backup (which would be necessary for NAS use) gets expensive with lots of data. At your 12TB of photos B2 would be $60 a month. You could just buy a 12TB drive within a handful of months paying that. My solution for awhile has been a Synology 2-bay NAS with two 6TB HDD's in raid 1 as my primary storage and a spare external hard drive as my backup. At about 4TB it has some legs left on it before I would need to upgrade the drives, but I decided to be proactive and figure out a better upgrade path. I put together an Unraid server from my old desktop and bought a pair of 10TB drives to put it in it for now. One parity drive and one data drive. I have 6 more 3.5" drive bays to add more HDD's down the line, two 2.5" bays as well. I have a PCIE SAS/SATA card in it as well to actually be able to hook up that many drives at some point. The motherboard had 8 SATA ports but only 4 were SATA 3.0. I can set it up to automatically backup to Crashplan, but it almost seems useless if it would take months to get everything back. Not that I would necessarily need my photos back ASAP, it would just suck having your internet bandwidth being occupied for months. I'd be willing to just buy hard drives and make copies, or Blurays and make copies, but everything I've read seems to indicate that just storing those away for safekeeping is still fairly unreliable. It makes me think my safest and possibly most cost efficient solution would to just be building a 2nd Unraid server that I can put offsite and mirror my local server to. Based in the B2 subscription costs I'd pay off my hardware before the warranty on the drives ran out, I'd just rather not inconvenience a family member or friend by having them store my tower and have it leach their internet. |
|
[#20]
Quoted: I should clarify I think. I'm interested in the complete system. From primary storage to backups. View Quote
If you've kept count, I use 5 drives:
This is probably more comprehensive (complex?) than most non-professional photographers may need. Most of the complexity comes from the "3-2-1" backup strategy and I started doing that religiously after having to explain to my wife how we lost years' worth of photos of her son because the cloud backup option I used at the time failed miserably when attempting to restore. (That's a story worthy of its own thread.) |
|
[#21]
Quoted: OK, here's my entire data flow - and remember that you asked for it...
If you've kept count, I use 5 drives:
This is probably more comprehensive (complex?) than most non-professional photographers may need. Most of the complexity comes from the "3-2-1" backup strategy and I started doing that religiously after having to explain to my wife how we lost years' worth of photos of her son because the cloud backup option I used at the time failed miserably when attempting to restore. (That's a story worthy of its own thread.) View Quote Thanks for spending the time to type all that up, I appreciate it. What are you using for your archive drive? Just another 14TB drive? What's the plan for when you fill that 14TB of space? I've looked into the M-Discs, but at $12 per 100GB I'm not really sure they're the best method for me. Financially it seems to make more sense to buy HDD's, even if that means having to purchase hardware and setup a second NAS/unraid server. |
|
[#22]
Quoted: Thanks for spending the time to type all that up, I appreciate it.
View Quote
|
|
[#23]
Photos stored in Apple 'photos' app.
Synced thru iCloud with masters kept on local iMac hard drive. Backed up to Carbonite Time machine local backups to 4 rotated 4TB drives. 3 oldest in offsite safe deposit box. 45,000+ photos, about 500GB |
|
[#24]
Quoted:
View Quote Yeah, the rough part about HDD's is they only go so big. I think it's going to be hard to keep getting big gains out of the format as well. It's kind of amazing how far they've come, but look at 2.5" drives, they had to go from 9.5mm to 15mm to get above 2TB. That was my reasoning on going with a NAS/server with a lot of HDD bays. Building it myself also made sense from a hardware replacement standpoint. |
|
[#25]
Quoted: Yeah, the rough part about HDD's is they only go so big. I think it's going to be hard to keep getting big gains out of the format as well. It's kind of amazing how far they've come, but look at 2.5" drives, they had to go from 9.5mm to 15mm to get above 2TB. That was my reasoning on going with a NAS/server with a lot of HDD bays. Building it myself also made sense from a hardware replacement standpoint. View Quote |
|
[#26]
What would be the ideal setup for a part time drone photography business?
I plan on focusing on agriculture and construction industry’s. |
|
[#27]
I'm currently about to redo my whole setup. Considering my internet is obscenely slow, I'm looking at alternative methods.
Medium term storage will be onto redundant drives/NAS. Long term storage of important files will be archived to M-Disc and stored offsite. Namely the images I sell the most of, important shots, etc. Ideally once I get better internet, I'll be doing active file backups to backblaze or similar. That said, I need to go through my archive and massively cull things first. That's the huge time sink since I haven't culled files in a year or two. |
|
[#30]
NAS server at home with RAID and weekly dumps to a NAS backup drive, portable drives in the safe, then portable offsite at a family members house about once a quarter.
I did the online storage for a couple years, don't exactly trust it anymore and I don't think both my house and my dad's house are going to burn down at the same time, not to mention I find it a hassle. I try and keep it low effort, or I'll find a reason not to mess with it. |
|
[#31]
Mine’s going to go exponential once I get out and start really shooting with the new camera. But that’ll be a good problem to have.
|
|
[#32]
My system tends to be old school.
Primary files on internal and external USB hard drives. Backups on external USB hard drives. Backups done by windows CMD scripts using XCOPY and ROBOCOPY. My personal files stay on the hard drives and their backup drives. My "pro" files are on the hard drives and their backups until the images are uploaded for their online viewing/ordering. Then those original files and their edits are burned to DVD and removed from the hard drives and from the backup drives. Yes, I know that if the house burns down, I have lost everything. I am willing to take that risk. One important part of any workflow is to properly triage the images and delete the bad ones that are not worth keeping in the first place. |
|
[#33]
I didn't mention, I use Aoemi backup manager for disk image backups, but for photo folders, I use FreeFileSync which only updates the new files from source to destination, or can overwrite all, tons of filter options and overwrite/save new copy, etc. It's free and very flexible to do backps quick and easy rather than waiting to copy entire folders at once when there's only a dozen now ones with several dozen that were already saved.
It runs nightly on the computer to update my documents, downloads, photos, data folders to the external USB while I sleep and in the morning I disconnect the USB and then rotate through them for a week or more deep worth of backups since they're not super reliable. The 3.1 drives are way faster than 2.0 and a must have, but still really slow once spoiled by an SSD. |
|
[#34]
|
|
Sign up for the ARFCOM weekly newsletter and be entered to win a free ARFCOM membership. One new winner* is announced every week!
You will receive an email every Friday morning featuring the latest chatter from the hottest topics, breaking news surrounding legislation, as well as exclusive deals only available to ARFCOM email subscribers.
AR15.COM is the world's largest firearm community and is a gathering place for firearm enthusiasts of all types.
From hunters and military members, to competition shooters and general firearm enthusiasts, we welcome anyone who values and respects the way of the firearm.
Subscribe to our monthly Newsletter to receive firearm news, product discounts from your favorite Industry Partners, and more.
Copyright © 1996-2024 AR15.COM LLC. All Rights Reserved.
Any use of this content without express written consent is prohibited.
AR15.Com reserves the right to overwrite or replace any affiliate, commercial, or monetizable links, posted by users, with our own.