As one might expect, running web development and photography-related businesses generates massive volumes of digital data that needs to be backed-up and stored for both daily access and safekeeping.
We have considered and tried a lot of options, almost all of which have their advantages and disadvantages.
What we have learnt along the way is that our business, and I’m sure most small to medium-sized operations, require most of the following in terms of data backup or a file server:
Daily access to the file server – We need to read and write large image files and video files daily.
Scalability – The system needs to be easily upgradable with additional drives.
Pooling drives – We have spare hard drives lying around and want to use them even if they are different sizes and from different manufacturers.
One drive – We have no interest in searching multiple backup drives for client files. The system must have one drive like N: – not N: M: and O: drives.
Speed – The system needs to be accessible permanently and offer a reasonable read and write speed. Standard network speeds are fine for us, but waiting for weird database stuff happening behind the scenes is a major annoyance when you are trying to get something done.
3TB Drives – Ability to handle large 3tb and larger drives!
Dependable – Stable, robust and secure.
Ability to easily make backups of backups – Break-ins are a major problem in Durban, Pietermaritzburg and Johannesburg – or any other major centre in our country South Africa.
Affordable – Sorry, there is no budget for the latest and greatest file server solutions. It’s got to be data storage on a reasonable budget.
Our experiences over the past few years has also led us to understand that our requirement is not actually for a backup device. It’s for a functioning file server that itself needs to be backed up.
There is quite a difference in the thinking around that point. A backup drive can simply be a 2tb drive in a USB drive enclosure. The required files are then sitting in two locations – the original computer and the backup drive. The problem with this process is two fold. The first is that backups are not automated. That means someone has to go the trouble of plugging everything in and running backups. And in a small business, I know who that “someone” is. The second problem is that we have outgrown our data capacity on any given computer and that we also need to work on the same files.
So our needs have rapidly changed from existing computer backup into a file server – which can be seen as just one large external data device. The advantage of a dedicated file server is that it’s always online via the network and instead of storing files locally on each computer, we can dump all our working files on the file server, freeing up local storage space and easily sharing the same files.
The disadvantage of a file server is that in crime ravaged South Africa, the server itself needs to be stored in an ultra secure location or it needs to be backed up frequently too.
If you are in a hurry, here is the condensed version of our experiences and recommendations based on the past 12 months of trial and error in trying to find a good scalable file server solution on a reasonable budget:
Up to 4TB server requirements and below – Look no further than Server Elements and their NasLite 2 file server system. You will need to be a bit handy with computers and computer builds or enlist the help of someone who has those skills. It’s not rocket science, but could be tricky if you don’t know anything about installing operating systems, drives or network configuration. Simply install the drive of your choice. Then either back your server up internally with another drive of the same size and/or a USB drive of the same capacity. Take the USB drive home at night and you have a fully redundant file server solution.
4TB server requirements and above – Buy a second-hand Drobo drive. At least a second gen Drobo 4 unit will be required because apparently the first gen units do not handle drives larger than 2TB. Load your Drobo up with all available spare drives. The unit will handle up to 4, 4TB drives that will provide around 10TB of backed up data. If one drive fails, just replace it out with a new drive and, apparently, you will suffer no data loss.
Drobo units can fail themselves, so you need to make sure that you have a backup of your file server somewhere and somehow. The only logical solution I can see for this one or a number of 4TB backup drives in separate enclosures. Use software like Second Copy and daily, weekly or monthly backups to high-capacity USB drives to ensure the Drobo is backed up too.
The Bad – Windows Home Server
From the start we failed really badly with our file server decisions when we installed Microsoft’s Windows Home Server 2003. We got sold on some really clever features of the software. The advantage of the system was that it would “pool” different drives together to form one large volume using some software tricks. The bad news was that the system did not read 3TB drives!
Like not at all.
And there was no work-around either. Turns out that someone at Microsoft thought 2TB would be the largest drive anyone might need! And it’s not just Microsoft either, a lot of hard drive enclosures are limited to 2tb drives too.
In then end though, it was the system’s dog slow speed and complete instability that turned the solution into a total fail. The clever software that allowed for drives to be pooled spent all its time indexing and re-indexing. The result was that the server ground to a halt most of the time.
When the operating system crashed for the second time in a month, we realised that the solution was never going to work for our needs. Worse, because the operating system failed, there was no way of obtaining our data from the hard drives again because the OS was running all that clever drive pooling technology!
The “solution” proved itself to be a total fail for us and even someone at Microsoft must have agreed – because last I heard the product had been discontinued.
The Good – NasLite 2
A company with a funny-looking website called Server Elements essentially solved our initial file server problems with their product NasLite 2.
This little Linux-based product has proved itself to be rock solid and reliable, fast and dependable. I can’t deny that setup was bit of a challenge, but help from company’s forum was excellent.
Key good points of this product are:
Linux-based – so 3tb drives and larger and formatted natively without so much as a splutter.
Linux-based – so the system is robust, stable and secure.
Operating System is on USB – Initially I thought they were mad . . . but I now see the logic! The
NasLite 2 operating system takes up such a small footprint that it can be booted from a small USB thumb drive plugged into one of the server’s USB port. The point is that the solid state thumb drive is far less likely to fail than a standard hard drive with moving parts. Also, because the OS is being run from a USB device, all the SATA ports and drive bays are free for 100% dedicated data drives.
Affordable – $29-95 is almost free for such a great little product.
Drive to drive backup – NasLite 2 does not pool drives or have a native RAID system. Instead, each drive is a stand-alone drive available via the existing network. The one drive can also be set to be a replica of another. I’ve had a 3tb drive being backed up by a second 2tb drive automatically overnight. This system has worked flawlessly – although I’m now 95% full on the 2tb drive.
Server to server backup – NasLite 2 has the ability to clone another computer running NasLite 2 at another location via IP address. So, if you have a small office or work from a garage at home, you can back your first NasLite 2 file sever up with a second located in your house via a wireless network. I assume that since this is an IP address, with enough available (and affordable?) bandwidth, location becomes irrelevant.
This is not to say that the NasLite 2 solution is flawless. Here is the downside:
No native RAID or drive pooling – All drives added to the system are seen as individual drives. There is no way to pool the drives. This has stability advantages, but can also be seen as a disadvantage. There is an obvious work-around – simply install hardware RAID on the server itself. Unfortunately, hardware RAID cards are still quite pricey (the cheapest I know of go for around R2 000) and I was personally concerned that we would have insufficient skill set to develop a hardware RAID and NasLite 2 solution all in one. Even if you do get RAID right, you are back to the same old story – needing drives of the same make and capacity for optimum results.
The main point, however, is that if you want grow your server to beyond the maximum capacity of any one drive (at the moment that would be 3tb or 4tb) you will end up with having to have multiple file server drives. That may be okay if you can, say, split your files between photo and video or clients and personal. But it not ideal.
Interface can be confusing – The setup can be a little bewildering. Even adding additional data disks can be confusing since initial settings need to be selected and then the system rebooted before the drive will mount. It’s not such a problem since careful reading of instructions usually solves any problem. That said, not every step is intuitive and the solution is not as polished as it could be.
Drives not natively read by Microsoft – Should there be some kind of drive failure and the whole system falls on it’s head, one will want to recover the data easily. The good news is that all the drives used by NasLite 2 can be easily read by any Linux-based computer.
The bad news is that they can’t be easily mounted by Microsoft-based computers. Yes, there are workarounds. Yes, it’s pretty easy to load Ubuntu Linux on just about any computer. But when disaster does strike, my guess is that the least number of steps required to get your data back will always be best.
Network only (no USB option) – I may be wrong on this point, but as far as I’m aware, there is no way to connect a NasLite 2 server to a computer via USB and expect it to run like a USB data drive.
The system runs on a network at network speeds. So, if you are like us and your network router snails along at the standard 100 Mbit/s and not the faster 1000 Mbit/s, read and write times to the server will hardly be snappy.
The mere thought that a solution like NasLite 2 could be connected via USB to another computer may even be laughable to the initiated, but the reality is that other backup offerings do and there may be an expectation from some that such a solution should take advantage of faster USB data speeds.
As mentioned, NasLite 2 has performed extremely well as a file server over the past 12 months or so. Our latest configuration has been a 3tb drive being backed up by a 2tb drive in the server itself. Another 2tb drive in a drive enclosure has then been used on a more or less monthly basis to back the whole server up and is stored off-site.
This solution is probably as good as it gets in terms of data backup. The most likely problem we will encounter is a hard drive failure on the server itself. So primary redundancy is covered with a backup of the 3tb drive to the 2tb drive. The second most likely problem is that the server gets fried by lightning or is stolen from our office. Both of these eventualities are covered by the USB enclosure backup.
Excessive data needs
The problem we have now encountered, and any business is likely to encounter, is that our data needs are outstripping individual hard drive capacity. They simply won’t make an individual drive large enough to accommodate all our storage needs.
Our 2tb drive is which backs up the main 3tb drive is now 95% full and needs to be upgraded.
Initially I thought the most logical thing to do would be to simply buy a new 3tb drive to back up the existing 3tb drive in the NasLite 2 server. But then, of course, I’d need to buy another 3tb drive for backup. Then, there is the realisation that my hard drive enclosure does not support drives more than 2tb! So, I’d need to replace the hard drive enclosure too.
When you do the maths, it works out that to upgrade the existing NasLite 2 server backup solution from 2tb to 3tb – just one additional TB in data storage would actually cost somewhere between R3 500 and R4 000 because of the additional drives and devices that would be required.
In addition, 2 x 2tb drives (the backup drive in the server and the additional backup drive in the hard drive enclosure) immediately become redundant and of no further value to the system.
Worse, in another 6 months time when we are 95% full with the 3tb drive and 4tb drives are arguably more affordable, we will need to buy three 4tb drives just to gain an additional 1tb of data storage!
In a way, our solution has been to head back to the beginning again with a proprietary drive format and a solution with ability to pool drives – only this time it’s offered by a dedicated backup solutions company, Drobo and not Microsoft.
Drobo has always been on my mind as a solution, but it’s not a cheap offering for what is essentially little more than a pooled hard drive enclosure.
We found a second hand unit for sale in Durban for the same as a new 3tb drive.
The second generation 4 bay Drobo unit is great for us because:
1. It can take 3tb and larger drives.
2. It it able to pool drives – and drives of any configuration
3. Drives are hot swappable – the system backs its own data up and if a drive failure occurs you simply take the failed drive out and replace it with a new drive. Well, that’s what they say. Let’s just hope that actually works when a failure actually occurs.
4. The system is small and relatively easily transportable.
On the downside:
1. Adding 2 x 2tb drives does not give you 4tb of network storage. Because Drobo backs your data up across drives, it only gives you 1.8tb storage capacity. That said, in order to back up the NasLite 2 solution you would need similar additional capacity.
2. The Drobo units themselves are prone to failure according to users like Scott Kelby. He blogged about his frustrations with Drobo and this bad PR has probably cost the company plenty of lost sales.
It’s actually probably a little unfair since the power someone like Scott Kelby wields on Google is quite extraordinary. There is probably another blog post in that 😉
3. If the Drobo dies – we will need another Drobo unit to read the drives. Apparently you can’t take a drive out of the Drobo and expect to read it natively by plugging into an avialble SATA port.
Our solution until 4TB
Moving forward, our solution will be simple. The existing NasLite 2 server is, sadly to be de-commissioned. All available large hard drives will be pooled in the four bay Drobo and a 4TB drive will be added to a USB enclosure for off-site backups.
After 4TB I actually have no idea what to do. The Drobo unit itself can carry on growing with 4TB drives until it offers a little over 10TB of server space. At that point the most likely solution would probably be to split Drobo backups into more or less equal parts and back them up separately with more than one 4TB USB external drive. But 4TB thankfully still in the distance for us for now!