Quantcast

Maximum PC

It is currently Thu Sep 18, 2014 12:06 am

All times are UTC - 8 hours




Post new topic Reply to topic  [ 45 posts ]  Go to page Previous  1, 2, 3
Author Message
 Post subject: Re: Has v. slow file copy/transfer in Win7 ever been fixed?
PostPosted: Thu Jan 17, 2013 2:51 pm 
Smithfield
Smithfield

Joined: Sun Jun 18, 2006 7:37 pm
Posts: 5247
You know, just to prove a goddamn point (a point, not belonging to anyone here in particular) on an internet argument, I'll burn as much as I can from a folder that has over 100K files onto a DVD, then read back from it.

Using Windows 8, which should be as bad amirite?


Top
  Profile  
 
 Post subject: Re: Has v. slow file copy/transfer in Win7 ever been fixed?
PostPosted: Thu Jan 17, 2013 4:29 pm 
[Team Member]
[Team Member]

Joined: Sat Jun 26, 2004 4:31 am
Posts: 11090
Location: Home Sweet Home
It is worth a try, just to see if they gimped the OS on purpose.

Nasty


Top
  Profile  
 
 Post subject: Re: Has v. slow file copy/transfer in Win7 ever been fixed?
PostPosted: Thu Jan 17, 2013 10:03 pm 
Smithfield
Smithfield

Joined: Sun Jun 18, 2006 7:37 pm
Posts: 5247
So I don't have Windows XP to test this against, but let's just go ahead with this. Keep in mind this isn't really much of a scientific experiment, so you'll just have to take whatever I give.

This is my setup. Please note that I'm using binary prefixes.:
  • Computer is a Dell XPS 15z. I'd do this on my desktop, but it's packed up at the moment.
    • For the curious: Core i5, 8GB of RAM, 128GB Samsung 830 SSD
  • Source of files is from a 7200RPM 2.5" USB 3.0 external hard drive.
    • Before you go cry at me that I'm using an external, yes there may be a possibility of the drive being nerfed, but I have no reason to believe so considering USB3.0 should provide more than enough power to run the drive at full speed. And there's plenty of bandwidth.
  • What I'm copying is 70,072 files totaling 4,199,083,765 bytes, or about 3.91GiB. Most of it are small sound files ranging from 8KB to 20KB. There are a few larger files here and there.
    • For a comparison, I transferred about 13 video files totaling 4,180,306,029 bytes (3.89GiB)
  • I have not defragged the external drive or ran any optimizers lately on the SSD. So we can consider this "average" case.

These are the results.

Transferring the 70K files
  • Transferring from HDD to DVD using Windows' built in burner: 58 minutes 30 seconds, an average rate of 1.17MiB/sec. It spiked up to 3-4MiB/sec at times when writing one of the larger files.
  • Transferring from DVD: Didn't bother. It took about 8 minutes to discover 20K files.
  • Transferring from HDD to SSD: Forgot to time it, but about 2 minutes. It averaged around 35MiB/sec according to the file copy dialog
  • Transferring from SSD to HDD: 9 minutes, 17 seconds, an average rate of 7.19MiB/Sec. It had spikes of about 15MiB/sec

Transferring the 13 files
  • Transferring from HDD to DVD: 16 minutes, 51 seconds, for an average rate of 3.94MiB/sec
  • Transferring from DVD to SSD: Operation didn't complete, but it started at 5MiB/sec and climbed up to about 10MiB/sec before stopping at 68% with 6 minutes, 41 seconds on the clock. So this is about 6.76MiB/sec.
  • Transferring from HDD to SSD: Didn't time it, again. Averaged about 105MiB/sec to 110MiB/sec though.
  • Transferring from SSD to HDD: 41 seconds, for an average rate of 97MiB/sec

So what can I take from this? The number of files that need to be transferred will impact the average transfer rate. To start, the SSD, HDD, and DVD transfer rates improved dramatically with a handful of files. Also writing has a heavier impact than reading, as writing isn't just writing the file down onto the storage device, but updating the file system as well, especially for HDDs/SSDs which have to update the NTFS system files.

This is why when doing the 70K transfer from the HDD to the SSD, it was a lot faster than the other way around. The SSD is better in orders of magnitude than the HDD when it comes to one thing: seek time. Since the NTFS system files are nowhere near where these files would've been written, the HDD had to constantly go back and forth. Contrast to the 13 file transfer, where both scenarios had almost the same speed (HDD to SSD was probably limited to from the HDD's read performance).

As for the DVD, I'm really not familiar with the low-level operations of DVD file systems. However, Windows did ask me if I wanted to format it like a USB drive or a mastered disk (I chose USB drive style). Chances are if the way this formatting works, it's table of contents is actually all over the place. Which would explain why it took forever to discover 20K of those files on read.

Moral of the story: cut your file quantity down.

EDIT: Just for fun considering Nasty thinks DVD movies are nerfed for transfer speed...

Unfortunately all of my actual DVD movies are packed up... except some that were those special bonus features from games. DVD movies have to have a standard file structure, because if they didn't, you can't guarantee any DVD movie would work in any DVD player. So under that assumption and Nasty's paranoia, we can infer that any DVD treated as a movie will have to trigger any "nerfing". Oh, the DVD in question is the Final Fantasy XII bonus feature.

And all I did was drag the AUDIO_TS and VIDEO_TS folders to my SSD... And for 6,469,423,104 bytes in 28 minutes 52 seconds is about 3.56MiB/sec.


Last edited by LatiosXT on Thu Jan 17, 2013 10:40 pm, edited 1 time in total.

Top
  Profile  
 
 Post subject: Re: Has v. slow file copy/transfer in Win7 ever been fixed?
PostPosted: Thu Jan 17, 2013 10:35 pm 
Million Club - 5 Plus
Million Club - 5 Plus
User avatar

Joined: Sat Jul 08, 2006 6:23 am
Posts: 2629
Location: Folding as BlackSun59
Latios, I want to thank you for all that you've done.
"Transferring from DVD: Didn't bother. It took about 8 minutes to discover 20K files."
I am so glad that you saw this and that it's not my imagination. This is the gist of my entire situation... and with my backups on DVD+RW, the slow discovery and calculation times for lots of small files make backup from optical disc - for me, anyway - a proposition to reconsider with Win7.
"Moral of the story: cut your file quantity down."
Which is why when I put the huge number of files into a TrueCrypt container, Win 7 sees the optical disc as having only one massive file, and resulting fast transfer speed. A single massive Zip file would probably have the same results.

Nasty, as far as thoughts of a gimped OS go, I'll throw out a conspiracy theory... the possibility that Microsoft wanted to perhaps quietly or insidiously "encourage" users to store their data backups on the cloud, accessible from anywhere a user might be instead of on a stack of old-fashioned optical discs. M$ had everything right with XP. Since Vista and 7, however, there's been this push to store our data on the cloud. Who uses optical storage anymore, right? Even MPC has built new rigs without optical drives recently. A lot of people think optical storage is "obsolete". So when you get people like me who prefer to do things the old-fashioned way - keep my hands on my data and not pay M$ or anyone else for 80, 90 GB of cloud storage - how else to get us to do as they want by discouraging what we've always done? Gimp the OS and wrap it around eye candy and useless "Discovering", "Calculating" and "Time Remaining" info.


Top
  Profile  
 
 Post subject: Re: Has v. slow file copy/transfer in Win7 ever been fixed?
PostPosted: Thu Jan 17, 2013 10:45 pm 
Smithfield
Smithfield

Joined: Sun Jun 18, 2006 7:37 pm
Posts: 5247
I don't want to blame Microsoft for conspiracy theories but I don't know on the low level how DVD file systems work. Naive assumptions would say that it would put a TOC up front, then write the files. If you write more onto it, it would update the TOC and continue on. But considering that the DVD was formatted when I tried to drag and drop something in there suggests that Windows' driver for DVD-R's as a "USB thumb stick" means either the TOC is updated as it goes, or it's written in pieces. It really depends.

And it may not be a Windows thing, as Wikipedia's article on UDF formats has a mode called VAT mode, which tries to emulate a traditional storage device. If Windows is using it, it explains a lot.


Top
  Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 45 posts ]  Go to page Previous  1, 2, 3

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group