A New Challenge
Recently my team and I were presented with a technical need from the business to solve a lingering problem. The challenge was how to share a large file with external vendors and other third parties. The department involved came to IT seeking a solution as they were consistently experiencing issues with trying to share these large media files (among others) online. To provide a quick summary, here are the difficulties they were encountering which we were asked to solve:
- Large media files (>20 GB) being shared externally had long upload/download times
- Inconsistent uploading methods
- Incomplete or interrupted uploads
- Tying up the machine of the person doing the uploading
Due to the size of the files being shared externally (20GB or more), they obviously could not be emailed to a vendor and therefore had to be shared by other means.
Existing Sharing Methods
Since we store a ton of data in the cloud as part of our overall collaboration efforts, the folks needing to share their large media files naturally attempted to use the tools already available to them. We are talking pretty mainstream solutions here like Box and Google Drive. Immediately at least two bottlenecks became readily apparent – maximum single file size upload limit and the time it typically took to upload to either of these services. Even getting the individual file size limit increased for Box only raised it to 15 GB, still too short for the size of typical media files being shared.
And, of course, the issue of just getting the files uploaded to the cloud was how long it took to complete. I would not claim that our Internet connection could not be faster to help with these uploads and downloads, but it is certainly no slouch by any means. And on the receiving end, the vendor’s Internet pipe would also determine how quickly they could pull down any files being shared with them, which we obviously did not have any control over. Then, when they were done with them, the process would start over again from their end getting the updated files back to us. Imagine doing this on a regular basis over the course of various projects, and you can quickly come to appreciate the inefficiency and frequent frustration involved.
To make matters worse, these uploads from our side were frequently inconsistent such that an upload did not always complete successfully. Sometimes it would get interrupted and could not be resumed, which meant starting from the beginning again. Using FTP alleviated this somewhat but could only be used with Box as getting this set up with Google Drive was not feasible at the time.
But the really obvious problem after these collaboration projects started to gain steam was the fact that it took more hours to upload these files than most people had available to them. Also, since our workforce is highly mobile, being on the go even inside the building (going to meetings, etc.) meant that they were frequently switching connections from wired Ethernet to Wi-Fi and vice versa. This, of course, contributed greatly to both inconsistent and/or incomplete data transfers as well as significantly slower uploads while being on a wireless connection.
Not having dedicated hardware for accomplishing these tasks was also part of the challenge since the uploads would be usually started from someone’s laptop. Well, guess what happened when their machine either went to sleep or they closed the lid getting ready to go somewhere. Wham! There goes that upload, NOT!
Solutions We Looked At
Over the course of many meetings and several months, we considered a number of options such as getting single file size limit raised (still insufficient) by our storage providers, increasing our Internet speed, compressing or splitting up files into smaller chunks, etc. Unfortunately, none of the options being considered proved to be satisfactory as they lacked in one area or another or were too inflexible to be practical.
One other solution being used at the time since the others were not working well were portable hard drives. These sometimes contained terabytes worth of data which were being shuttled between between our folks and external vendors. This presented yet another wrinkle to an already thorny problem – ensuring the safety of the data on these drives, especially in transit, since they would contain intellectual property that had to be preserved.
Two typical threat scenarios when it came to protecting the data were:
- Data loss due to a natural disaster such as a fire, flood, etc.
- Loss or theft of the hard drives while en route to a vendor
The first thing that comes to mind, of course, is encrypting the drives in order to safeguard the data they contained. All fine and good until you come to realize that each vendor working on these projects was using different systems running either Windows or MacOS. This meant that each hard drive had to be encrypted in a way that would allow access based on which OS was being used. Can you see the inefficiency in doing that? Arghhh!
Proposed Solution
Since it became quickly obvious that sharing these large video files online was not very practical for the various reasons outlined above, another solution had to be developed. Fortunately, we did not have to start completely from scratch and were able to build on what was already being used by the teams needing to share these huge files.
So, as you may have already guessed, we chose to improve on the portable hard drive shuttle process to address the shortcomings identified earlier. Namely, protecting the data on these devices both at rest and in transit with sufficiently strong encryption that would be also OS agnostic.
Obviously, there are many similar products on the market that would potentially fulfill the above requirements, so we had to also consider such factors as cost and usability. With all of that in mind, we finally settled on portable encrypted drives from Apricorn. They have a nice product range with a rich feature set to protect our data that is also cross-platform compatible.
Some of the features making these drives an attractive and cost-effective solution for us:
- ON-THE-FLY 256-BIT AES-XTS HARDWARE ENCRYPTION
- SOFTWARE-FREE AUTHENTICATION & OPERATION
- COMPLETELY CROSS-PLATFORM COMPATIBLE
- COMPATIBLE WITH ANY OS: Windows®, Mac®, Linux, etc.
- PROGRAMMABLE MINIMUM PIN LENGTH
- RECOVERY PINS IN CASE OF FORGOTTEN OR LOST PIN
- INTERNAL EPOXY FILLING FOR PHYSICAL-ATTACK PROTECTION
- PROGRAMMABLE BRUTE-FORCE PROTECTION
- SELF-DESTRUCT PIN
The drives come in various capacities ranging anywhere from 500GB to 16TB of storage space. 
For our needs we started out with the Aegis Fortress L3. It is a very rugged portable drive with a built-in PIN pad providing on-the-fly encryption without requiring any special software to use it. So if the drive was ever lost in transit or stolen, any data it contained would be inaccessible without the PIN. And after a number of failed attempts to brute-force the drive, it would perform what essentially amounts to a self-destruct process (queue Mission Impossible music).
“… if the device comes under a physical brute force attack, your data is protected with a programmable “Brute Force Hack Defense Mechanism” which, if the programmed number (between 4 and 20) of consecutive incorrect password entries has been attempted, the device will delete its own encryption key and destroy the ability to decrypt its stored data.”
Also, Separate Admin and User Mode allow for the universal programmable settings of the device and can only be accessed with the Admin PIN. The User mode is limited to basic external drive functions like read /write, unlock / lock, etc. There are many other additional features which makes this a very versatile solution:
- FIPS 140-2 level 3 Validated
- Aircraft Grade 6061 Aluminum Alloy Enclosure
- Admin Forced Enrollment
- User Forced Enrollment
- Data Recovery PINs
- Polymer-Coated Wear-Resistant Onboard membrane Type Keypad:
- Programmable PIN Lengths
- Brute-Force Defense
- Unattended Auto Lock
- Lock Override
- Drive Reset Feature
- Self-Destruct PIN
- LED Key Press Indicator
- Dust and Water Resistant Durable Aluminum Alloy Housing and Membrane Keypad
- Two Interchangeable USB 3.1 / 3.2 Cables
Overall, this was the most acceptable and reliable solution meeting two of the most important requirements – security of the data at rest and in transit while providing reliable sharing with vendors and other parties. I would love to hear from any of you who had to deal with a similar problem and what you ended up using as your solution, so please comment below.
