Tips for transferring large data files

Sift Media
Share this content

Several members of our UK sister site,, have posted questions in the Q&A forum about data management and transferring large files. AccountingWEB UK tech editor, John Stokdyk, delves into some of the answers suggested by AccountingWEB members.

The questions raised by Ian Main, Michelle Woods and Kevin Ringer share a common concern - how can they move around the large data files that are associated with accounting applications?

Ian Main's query was based on a need to find a better way to pass 10Mb QuickBooks back-up files to and from his client - a round-trip that typically took up two hours for both parties. Michelle had a similar query about how she could send Sage backups to a client for whom she was working remotely. In her case, one member suggested setting the Sage back-up option to "Data files only" rather than "All files", which results in a much smaller file that e-mails easier.

Kevin Ringer's data challenge was on a different scale and concerned now to back up 150GB of company data. Kevin's IT team run a DOS Xcopy routine to write the data from its server across the local network to a backup drive located in another part of the building. The process achieves the desired result, but is horrendously slow and takes up to two days to complete.

What Ian is trying to do with his client's data is conceptually pretty close to Kevin's approach, as they both effectively deal with taking back-ups of data files, which need to be moved between different machines. To maintain proper information security, the files need to remain accessible to both machines in case either one fails. In the event of such a failure, having all the data (and operating software) immediately to hand is the vital first step of any business recovery operation.

So what are the best solutions to transferring and backing up large quantities of sensitive financial data? While relying on contributions from AccountingWEB members, this article also explains some of the key principles and technologies that underpin their answers.

E-mail options and restrictions

Although it is commonly used, e-mail is not the ideal mechanism for big file transfers or back-ups. Many e-mail providers and office administrators set limits on the file sizes that can be sent; a 10MB data file, for example, could hold up other users' mail while it chugs through the system. Hosted e-mail services can get around some of these blocks, for example the unlimited amount of e-mail storage you get with AOL, or Googlemail's 5GB (gigabytes) should cater for the Sage/QuickBooks back-up scenarios described by Ian and Michelle., suggested by Richard Langner, is a free service that provides 5GB of capacity you can access from any PC or POP3 e-mail account. It can also handle attachments up to 20MB. This was the option initially chosen by Ian Main, who thanked the community for all the assistance. He also said he would take a more considered look at the other options suggested. Other members put forward services such as, and that are a cross between e-mail and file transfer protocol (see below) sites. As Emily Coltman explained, "Basically you upload a file to the yousendit site and tell the site your client's e-mail address. The site then sends a link to your client to tell them where to pick the file up. As James Pearce pointed out, however, the efficiency of all these services will depend on the upload and download speeds supported by you and your clients' ISPs. JS recommended's speed test to check whether your net connections can realistically support these online approaches.

File transfer protocol sites

Using a file transfer protocol (FTP) server as a staging post is an effective way to handle large file transfers. Data travels half way for each party and resides on the server until the recipient is ready to download it. FTP is conceptually very simple and a low cost option, but requires access to an FTP server and some administrative set-up. Ian Main was worried about how easy this would be for a non-literate IT client to operate, but once you know how to do them, FTP transfers are pretty easy. But you will need to put a robust security system in place and then take the time to train one or two people at the client's office to use it. If you already have a website, Robert Brook recommended creating a file upload page on your web server. He operates a password-controlled area for clients and says it doesn't suffer from the delays that often hold up e-mails. James Pearce agreed that this was the most professional approach. Your IT partner should be able to configure FTP on one of your servers with a web front end, he said, so clients can access the download area from their browser by typing in a URL such as: http://[yourdomainname].com/clients.

Online storage

E-mail and FTP can help you pick up and deliver data to clients, but neither is particularly robust or efficient as a permanent back-up mechanism for your business data. An increasingly popular way to handle off-site back-ups is to store and access the information on the internet. Most services increase in cost the more data you store, but can relieve you from the need to administer and physically store the back-up data. You simply rent storage space from the hosting company by the gigabyte on a monthly basis, so issues such as disk size and type, and operating system upgrades are no longer your problem. You could take particular care, however, to check the hosting company's own security and backup facilities, and see how easy it is to restore data from the service. Jill Spiers uses Attix 5, provided by Non Stop IT Security, which backs up automatically at a chosen time, or can carry out a manual back-up at the click of a mouse. "For us the back-up takes about a minute, and it keeps whatever you want, as long as you want," she said. To give client access, you can either set up a secure access area on the storage site, or FTP the specific back-up file from your online database to a separate download site, she added.

Remote tools for QuickBooks and Sage users

As Nicholas Myles noted, since 2002 QuickBooks Premier has included a Remote Access facility, which also has drag-and-drop support for the remote access service. The system lets the accountant work on QuickBooks files stored on a client's PC via an open Internet connection - so there is no data transfer or synchronization, as you are working on the target PC's hard disk. Sage 50 and Instant Accounts have a slightly different utility called the Sage Accountant Link, which compresses and encrypts files ready for e-mail or upload transfers to and from clients.

Online accounting and joint storage areas

The built-in links and remote access options available with desktop accounting programs such as QuickBooks and Sage are old hat to champions of software as a service. Alan Wright of Liberty Accounts, e-comomic's Mark Davies and Dennis Howlett on behalf of FreeAgent all urged readers to look into hosted accounting systems that could make the concept of back-up file transfers redundant. As well as handling the accounts processing online - where the accountant can have more immediate sight and control over the client's bookkeeping - these hosted, "on demand" services also have built-in data back-ups. Davies explained: "A fully online system means that you will never need to go to the client to collect data or see their accounts, nor will they ever need to send you data, because you'll always have access to it and be able to provide valuable advice and guidance whenever it suits you."

External hard drive/network-attached storage

Perhaps you are not psychologically ready to entrust your sensitive financial information to the internet "cloud". Disk technology has advanced so quickly that multi-terabyte capacities are now feasible, which is how AOL, Google and other online services are able to offer such generous storage allowances. As well as being inexpensive, hard drives provide reasonably quick read and write access to your data. For a bit of extra investment of time and money, Kevin Ringer's IT colleagues could install an array of cheap disk drives to "mirror" the firm's primary business database, and then schedule automated back-ups to be taken from the second server, so as not to affect the speed of the main network. To allow for easy off-site storage, or to transfer large amounts of data to and from a client site, external network attached storage (NAS) is also a convenient, low cost way to take back-ups. Currently £310 will get you a 1TB Buffalo LinkStation Mini NAS device that measures 40 x 80 x 132mm, roughly the size of a clunky digital camera.

USB flash drives

Even if they cannot match hard disks for capacity, USB memory sticks, iPods, mobile phones/PDAs and even digital camera memory cards can provide a quick, low preparation medium for data transfers - a technique known back in the days of floppy disks as "sneaker net". Because they are solid state electronic devices, they offer fast data transfer speeds and can hold up to a gigabyte or two of data. If the machines are relatively close to each other, the information can be moved in discrete chunks. But flash drives can be temperamental and are easy to lose. While they may be fine for passing data to an accountant, do not rely on them as your primary back-up device.


The age of the CD may be on the wane, but a "normal" write-once CD-R still has a capacity of 700MB and can handle more data if files are compressed - which should be enough for smaller business back-ups and data transfers. Some of the newer DVD formats can hold up to 9.4GB of data, which should be more than adequate for most AccountingWEB members. Optical disks can be relatively slow to write, but offer very quick access and transfer times, and are frequently used for back-ups in smaller organizations. But with such a variety of formats, many of which are nearing obsolescence, watch out for compatibility problems. A DVD written on your PC may not be readable on another machine, so test carefully before committing yourself to this route.

Magnetic tape

Bringing up the rear in this cavalcade of storage technologies is the magnetic tape drive - a mainstay of the computing that has been storing back-up data since the 1960s. Tape is still widely used a high-capacity, low-cost back-up option, but can take a lot of time if you are trying to restore an entire system, or to locate an individual file.

Further reading

IT Zone guide to data storage
Any Answers: Online backup services
Back-ups - The secret to surviving IT system failures
Focus on data protection
Focus on disaster recovery and business continuity planning
Information Security Expert Guides

More information is available in the Sage whitepaper on data protection, published in January 2008.


Please login or register to join the discussion.

There are currently no replies, be the first to post a reply.