Shawmut did not attempt to have IT resources automate the file synchronization task, but other companies have—normally with unsatisfactory results. Getting bi- or multi-directional file synchronization to work well is not a trivial endeavor. Indeed, successfully navigating the different “file logistics” of multiple incompatible storage systems can become a Tower of Babel that is fraught with potential peril. Making a mistake when comparing just one of the file’s properties involving the last accessed/modified date, user/group access permissions or locking can result in a file becoming corrupt or over-written by an older version. And if the custom integration application lacks robust error detection and reporting (something that is deceptively difficult), the mistake will remain undetected—until a user complains.
For a one-time migration or a one-way backup, a custom integration effort, consisting of a combination of manual and automated procedures, may work well enough. This is especially true if the differences among the storage systems involved are relatively minor and manageable.
But in most cases, the answer to the question asked in the title is: Yes, it will take an army to successfully and securely synchronize files in a hybrid storage environment. Fortunately, there are three alternatives to custom integration.
Using familiar, proven and low-tech “brute force” bulk copy commands, such as xcopy in Windows/DOS and rsync in Linux, is certainly simple and, therefore, might seem to be fairly foolproof. Applications like the File Explorer in Windows and the file management applications offered with most EFSS services also provide bulk file and folder copying capabilities.
For brute force bulk copy to work well, though, the storage systems involved either need to be compatible or must be made interoperable at their “lowest common denominator.” For example, more lenient file naming conventions and more generous file size capabilities might need to be abandoned in order to accommodate the most restrictive storage system, but doing so will minimize the complexity involved. Unless all systems can be made fully interoperable, however, challenges are certain to remain, especially involving file locking and security context via properties like user and group permissions for read/write/delete access.
As with custom integration, rudimentary copy can work well for a one-time migration or as a one-way backup solution. But because basic bulk copy commands and utilities lack robust file comparison capabilities, this approach is risky as a file synchronization solution in a hybrid storage environment.
Various forms of import services are available with virtually all EFSS platforms. Each has its own file management application with an online file import function, and some providers recommend using a physical disk drive when importing more than 100GB of data.
While these online applications and services shift responsibility to the EFSS provider, they can suffer from the same potential complexities and/or limitations such as lost permission models and structures, user-defined metadata, file ownership, and versions as encountered in custom integrations and rudimentary copy mechanisms. So if the import service fails to adequately accommodate the underlying file property differences between or among the different storage systems, the results are destined to be less than satisfactory. And it is for this reason that EFSS providers—just like a growing number of enterprise IT departments—are starting to use purpose-built third-party file migration and synchronization tools.
Sign up for CIO Asia eNewsletters.