{"id":4635,"date":"2015-05-18T13:26:50","date_gmt":"2015-05-18T12:26:50","guid":{"rendered":"http:\/\/dev.adambowie.com\/?p=4635"},"modified":"2015-11-03T11:21:37","modified_gmt":"2015-11-03T11:21:37","slug":"storage-and-backup-some-personal-experiences","status":"publish","type":"post","link":"https:\/\/dev.adambowie.com\/blog\/2015\/05\/storage-and-backup-some-personal-experiences\/","title":{"rendered":"Storage and Backup: Some Personal Experiences"},"content":{"rendered":"[Note that this is likely to be duller than usual! It&#8217;s mostly written up so that anyone who searches for similar problems might find it useful.]\n<p>This all began last Wednesday when I noticed an email from Synology inviting me to update the firmware that runs that two NAS drives I own &#8211; a DS210j and DS214se. They&#8217;d just released DSM 5.2, their proprietary software that allows you to do all sorts of clever  things  with their products.<\/p>\n<p>I&#8217;ve had nothing but good experiences with Synology. They&#8217;re reasonably priced, and they offer a safer way to store data than simply using an external hard drive.<\/p>\n<p>So I ran the updates. My new DS214se updated no problem, but the older DS210j had a problem. It got to about 20% of the process and then stalled. And it wouldn&#8217;t restart. The status light was flashing orange which is not a good sign.<\/p>\n<p>Now I was unable to get into the volume at all. I was concerned. <\/p>\n<p>I had previously put in place a monthly &#8220;off-site&#8221; back-up. The NAS had 2 x 2TB drives in a RAID 1 array. That gave me a 2TB volume safely mirrored on the two drives alleviating most hardware failure. Additionally, I would monthly plug a 2TB external hard drive into my computer and ensure that all the contents were backed up. This 2TB volume would sit safely in my desk drawer at work &#8211; &#8220;off site&#8221;. Unfortunately, I left my old job, and hadn&#8217;t carried on with my back-ups, so my most recent back-up was over a year old. Furthermore, I&#8217;d since bought a second NAS, and that had no back-up at all. But that was working fine, so a problem for another day&#8230;<\/p>\n<p>At one point in the procedure, I&#8217;d attempted to reinstall DSM using the software provided. But then I&#8217;d been presented with the five stages the software would take &#8211; including re-partitioning the drives! I pulled the plug &#8211; literally &#8211; when I saw that. I didn&#8217;t want my drives to wiped!<\/p>\n<p>My workaround after a bit of Googling was as follows. I bought another 2TB hard drive and a SATA dock for a hard drive from my NAS. Because I&#8217;d used RAID 1, using one of the drives should be fine. I then downloaded a bootable version of Ubuntu onto a memory stick, and followed instructions to boot into the OS on my PC and get access to the drive. Ubuntu us seemingly better for reading the RAID file format.<\/p>\n<p>My files were safe! Ubuntu wasn&#8217;t reading my external HD though. I put this down to buying a special kind of HD that I&#8217;d my eye on anyway. It&#8217;s WD WiFi model that can deliver files to portable devices easily. More usefully to me in photography was the built in SD-Card reader which can hoover up any files on a card while you&#8217;re in the field. While SD cards are cheap enough, it&#8217;s good to have a backup on the road, and reading the resulting images into Lightroom afterwards via USB will be quicker. However Ubuntu couldn&#8217;t read this new purchase, so I had to get <em>another<\/em> cheap 2TB drive which it could see.<\/p>\n<p>I then used Ubunutu to copy down the files to my external hard drive. This obviously took many hours. But now I had at least the safety of knowing that they were backed up before I went about rebuilding my NAS.<\/p>\n<p>Then it was a question of reinstalling DSM 5.2 onto my Synology. I was expecting the software to reformat the drives before I had to copy them back to the NAS. However, when it came to the re-partitioning step&#8230; it skipped it! It took a while, but it reinstalled DSM onto my NAS, and my files were safely sitting there without me having to copy anything back. I hadn&#8217;t actually needed to buy either of the two 2TB drives I now had, boot into Ubuntu or buy a HD dock. I felt much safer for having a backup though.<\/p>\n<p>There was another problem now &#8211; one of the hard disks was reported as physically degraded. That was because I&#8217;d foolishly yanked the HD out of the dock before making sure it was switched off. You know instantly if you&#8217;ve done this because the spin speed imparts some serious gravitational forces on the drive and you can feel them in your hand. I really can&#8217;t explain how weird this feels. I don&#8217;t suggest you try it unless you don&#8217;t care about your hard disk.<\/p>\n<p>Had I completely messed up my drive in this process? Was I going to have to buy yet another 2TB drive? <\/p>\n<p>Possibly. <\/p>\n<p>I went through the Synology repair process &#8211; it took a painfully long 18 hours &#8211; and all seemed fine. The drive is no longer reported as degraded (although I will keep a close eye on it), and once I&#8217;d reinstalled Plex and Download Station onto my NAS, I was back up and running. <\/p>\n<p>Searching my records, it appears that I bought this NAS in 2010, and the drives at the same time. I suspect that I need to replace them anyway in the near future. So a couple of 3TB or 4TB drives is on the cards. Storage is so expensive, yet so dull.<\/p>\n<p>Do I get a new NAS at the same time? Getting a Synology NAS that supports Plex is a more expensive proposition these days, despite the advances in processing power. I&#8217;ll see depending on what comes out. <\/p>\n<p>In the meantime, I&#8217;m now faced with another dilemma. What do I do about my &#8220;off-site&#8221; back-ups? <\/p>\n<p>I reckon I currently have between 3 and 4TB of data stored across various devices that I&#8217;d like safely backed up. But what&#8217;s the most cost effective way of doing this? I could revert to portable hard-drives sitting in a locker at work again. But cloud computing costs are coming down, and my fibre connectivity is unlimited. Even then, I realise that it&#8217;s likely to initially take a couple of weeks to fully back-up  my current file usage. But having files in the cloud would be massively useful. And much safer.<\/p>\n<p>I currently have a Google account with a bit over 1TB of included storage (a Chromebook promotion), of which I&#8217;m using a grand total of 7%. They charge $9.99 per TB per month for storage additionally. So that&#8217;s an option, although getting on for $500 (\u00a3320).<\/p>\n<p>I also have 1TB of storage with Microsoft&#8217;s OneDrive because I have an Office 365 Personal subscription that came with a computer. They came out as Labs winners in the most recent PC Pro magazine. I&#8217;m using even less of that 1TB, but I&#8217;m a bit confused about OneDrive&#8217;s options. <\/p>\n<p>Logging onto OneDrive to look at additional storage options suggests that I have to pay either \u00a31.99 a month for another 100GB or \u00a33.99 a month for another 200GB. Yet <a href=\"https:\/\/blog.onedrive.com\/office-365-onedrive-unlimited-storage\/\">this piece<\/a> from their blog in October last year says that they are rolling out unlimited storage! It still seems to be in <a href=\"https:\/\/preview.onedrive.com\/?wt.mc_id=oo_blog_onedrive_onedriveunlimitedstorage\">beta<\/a>, so I&#8217;ve added myself to the list. \u00a360 a year for Office plus unlimited cloud space would be a great deal (or even \u00a380 for 5 PCs). [Update &#8211; November 2015: I never got my &#8220;unlimited&#8221; storage. I got the 1TB when I later bought an MS product that came with 12 months&#8217; of Office 365. Microsoft has <a href=\"https:\/\/blog.onedrive.com\/onedrive_changes\/\">now pulled unlimited<\/a>, and is sticking with 1TB for Office 365 users, and 5GB (down from 15GB) for free users.]\n<p>The other option seems to be Amazon where I pay for Prime. But again the situation is a bit confusing. Logging into my account I seemingly have 10GB of total storage space, of which I&#8217;ve got 9.7GB free. The remainder is mostly made up of documents I&#8217;ve sent to my Kindle in the past.<\/p>\n<p>Amazon does allow free photos to Prime members in the UK. But there are question marks about some file formats. Amazon recognises <a href=\"https:\/\/www.amazon.co.uk\/gp\/help\/customer\/display.html?nodeId=201649930\">some RAW file formats<\/a> but not others. Video is excluded though, and although I mostly keep it out of Lightroom, there&#8217;s a little in there. Perhaps Amazon could take care of my Lightroom photo library though for no additional cost.<\/p>\n<p>What&#8217;s odd is that <a href=\"https:\/\/www.amazon.com\/clouddrive\/pricing\">in the US<\/a> you seem to be able to pay $60 a year for unlimited storage of any kind of file. This just doesn&#8217;t seem to be an option in the UK. <\/p>\n<p>Otherwise, there&#8217;s DropBox, but it&#8217;s too expensive on current price plans. There are smaller companies, but it feels safer to go with a big company. If Microsoft does properly roll out a full unlimited OneDrive offering, then I&#8217;m in. Otherwise it could be Amazon as a partial solution.<\/p>\n<p>Decisions, decisions.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>[Note that this is likely to be duller than usual! It&#8217;s mostly written up so that anyone who searches for similar problems might find it useful.] This all began last Wednesday when I noticed an email from Synology inviting me to update the firmware that runs that two NAS drives I own &#8211; a DS210j [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11],"tags":[479,189,377],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/posts\/4635"}],"collection":[{"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/comments?post=4635"}],"version-history":[{"count":3,"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/posts\/4635\/revisions"}],"predecessor-version":[{"id":5076,"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/posts\/4635\/revisions\/5076"}],"wp:attachment":[{"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/media?parent=4635"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/categories?post=4635"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dev.adambowie.com\/blog\/wp-json\/wp\/v2\/tags?post=4635"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}