Quantcast

Maximum PC

It is currently Tue Aug 19, 2014 10:21 pm

All times are UTC - 8 hours




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Gb Teaming & Transfer Expectations
PostPosted: Sat Jun 04, 2011 12:11 pm 
8086
8086

Joined: Sat Feb 13, 2010 12:12 am
Posts: 8
I've been chasing better LAN speeds for about 6 months now and have finally come to the conclusion that I am just plain stumped.

I keep all of my media and home movies on a dedicated server that in turn file shares out to 5 workstations 2 laptops and 2 media PCs as well as a secondary server (for Backup). I store all of my DVDs and Blue-Ray as pure ISOs on the main server and mount the images at the media and end PCs. With that method the LAN stays very consistent with no noticeable impact with several machines pulling at once. However, I've noticed that multiple regular file transfers can eat the LAN alive.

The entire system is Gb and wired using Cat6.
Router: Net Gear WNDR4000
Switch 1: D-Link DGS-1224T 24port
Switch 2: NetGear JGS524 24port
Server : Xeon E3-1245 on a ASUS P8B WS /16GB DDR3 1333
Server OS is on a OCZ Revo X2 120Gb /Win Server 2008 R2 Ent
Array: Areca 1260 w/12 WD black 7200 1.5TB drives / Raid 5
The 2 port Intel server NICs are Teamed

Prior, the server was built around a ASUS Rampage Formula and a 9650 using the same Areca and drives. My primary system is an ASUS P6T deluxe/Intel 940/12GB DDR3 1600/OCZ Revo X2 240. And 3x 500GB WD 7200 -Raid 0/ Intel ET Dual port server NIC

Under the original config my transfer speed between the old server and my main machine for a 5GB ISO is around 105-130 MB/s. When I added in the multi-port NICs to my machine and the quad-port on the server I was hoping I see a significant increase. But no matter which drive arraignment I move a file to-or-from the same 105-130MB is still consistent. But this speed is for all the machines, the teaming seems to make no difference. To make matters worse if I happen to move another file transfer anywhere on the LAN then the speeds still decreases down into the 40-50MB/s. I've gone through the switches and router looking for an option I may have over looked but nothing. Jumbo packet on, Corresponding Ports are teamed...etc.

So over the last few months I've now upgraded my entire backbone but with little to show for it. All-in-all the entire system works great and I have no real problems with it, but I just keep thinking that the last upgrades to the switches and servers were for nothing. So here I am hat in hand hoping you guys might gleam from this something I've overlooked.

By the way MPC, my wife blames you for our electric bill........


Last edited by Bad-Karma on Thu Jun 23, 2011 6:36 am, edited 2 times in total.

Top
  Profile  
 
 Post subject: Re: Gb Teaming & Transfer Expectations
PostPosted: Fri Jun 10, 2011 7:23 am 
Team Member Top 50
Team Member Top 50
User avatar

Joined: Tue May 24, 2011 7:51 am
Posts: 249
That's a ton of hardware, I can only imagine the electric bill! Since I started folding we haven't gotten our first one yet, but between that and air conditioning, I'm sure it will be rough.

Are the files you're transferring all coming from the RAID5? Do you have any way to test the raw transfer rate of the array on the server itself, without hitting the LAN?

I have a lowly software RAID5 in my Debian Linux server, with 3 1.5TB drives. The best I can do over my single gigabit link between the server and my desktop (writing to SSD to minimize write overhead at the receiving end) is around 40MB/s

I would think your RAID performance would be much greater than mine, but I'm wondering if you might still be running up against disk performance walls with multiple simultaneous transfers. Do you have NCQ enabled?

It does seem a bit strange that your speeds top out almost exactly at theoretical gigabit throughput, but I suppose it's possible that it's just a strange coincidence that the fastest the RAID can pump out is also roughly equivalent to a single, non-teamed gigabit pipe.

Edit: I would be VERY surprised if your RAID couldn't push far past 130MB/s read speeds, but when you experience unexpected behavior, I feel like it can't hurt to ask!


Top
  Profile  
 
 Post subject: Re: Gb Teaming & Transfer Expectations
PostPosted: Fri Jun 10, 2011 5:26 pm 
8086
8086

Joined: Sat Feb 13, 2010 12:12 am
Posts: 8
Ran the test using a 4.97 GB ISO completely internal on the server.
The Areca is set to Sata 300 +NCQ

Two Tests:
1: Raid 5 transfer to C: on Revo X2.
Saw a 643MB bursts for about 3-4 seconds then it slows to around 425-460MB , Total about 9 seconds to transfer between drives.

2: Reverse back to the Raid 5 locks in about 370-372.


So the arrays are quite capable of pushing well over what the NICs can pump out. ie...I have a serious bottleneck either in the OS or at the NICs themselves. I've been scanning through Server 2008's features and roles but I don't see anything related to throttling the output to the LAN.

Don't get me wrong, I realize that I'm getting some great LAN throughput, and I am happy overall. I'm just really frustrated that the last upgrades didn't get me anywhere when they should of.

I've actually been hoping MPC would eventually do an article on high end LAN layouts.


Top
  Profile  
 
 Post subject: Re: Gb Teaming & Transfer Expectations
PostPosted: Sat Jun 11, 2011 6:16 am 
Team Member Top 50
Team Member Top 50
User avatar

Joined: Tue May 24, 2011 7:51 am
Posts: 249
Thanks for running tests on it, definitely monster storage performance! I figure it never hurts to eliminate possible causes. I second the request for an article on high performance LAN!

It seems like you've got pretty hard proof that you are capped at gigabit. Wonder if MS would provide any support on this for Server 2008? Not sure what kind of support options they have nowadays


Top
  Profile  
 
 Post subject: Re: Gb Teaming & Transfer Expectations
PostPosted: Sun Jun 12, 2011 9:06 pm 
8086
8086

Joined: Tue Aug 28, 2007 6:55 pm
Posts: 37
Two things I can see here. One, on the netgear, you will be topped out at the gigabit max since it's a non managed switch. Two, unless you can do some kind of link aggregation on the d-link switch, you won't see any LAN performance increases. The only thing you're going to do from this point is pull your hair out on trying to get any more speed out of it. 10G ethernet is out there, but i don't know how close it is to being priced for the consumer market.


Top
  Profile  
 
 Post subject: Re: Gb Teaming & Transfer Expectations
PostPosted: Sun Jun 12, 2011 10:26 pm 
8086
8086

Joined: Sat Feb 13, 2010 12:12 am
Posts: 8
Yeah, the Netgear was a "previous purchase" before I started really studying the problem. I needed quite a few ports and figured that a 24 port would do it plus give me future expansions. I have now relegated it to peripherals use like printers and KVM links, basically anything other than the PCs.

Unfortunately the D-link isn't really a managed switch either. But I do have it set up with Link aggregation (x 4 on the server & x2 on my main rig and 1 other machine). It is configurable from the web interface but it isn't full time. I snapped a picture of my setup if you guys are curious: https://docs.google.com/document/d/1frMXe9Yw6yPNGVZAnjDuP5vqYJpCLxjegf_g1WamPTU/edit?hl=en_US&authkey=CPzA-7YH
Just don't make too much fun of the cabling mess!

The real kicker is that a few months back I had a chance to get a free second hand OC96, that was being replaced for upgrade, from work but I figured that might be way out of my league. I shot myself in the foot and let it pass to someone else in my shop. Man do I regret that now!


Last edited by Bad-Karma on Sun Jun 12, 2011 10:47 pm, edited 3 times in total.

Top
  Profile  
 
 Post subject: Re: Gb Teaming & Transfer Expectations
PostPosted: Sun Jun 12, 2011 10:43 pm 
8086
8086

Joined: Sat Feb 13, 2010 12:12 am
Posts: 8
AliusBU,

I reran the same tests today with a few different programs and got some really surprising results.

This time I also transferred by both pushing from the server to the workstation as well as workstation to server. No noticeable change on LAN transfers for either direction or arrays vs. SSDs; no matter the combination.

Before my readings were taken using the built in Explorer (Win 7 x64). Today I also did the same transfers using Directory Opus and QuickSyncPro. Going across the LAN, DO reported back 65MB/s on average and QuickSyncPro usually threw out 108-120MB/s. So it might be something in the transfer algorithm used by each software vendor. Do you guys know of any programs out there that may have better performance to throw at the test?

I'm going to grab a dedicated "LAN speed tester" and try again tomorrow to see if something comes up different. Any recommendations?


Top
  Profile  
 
 Post subject: Re: Gb Teaming & Transfer Expectations
PostPosted: Mon Jun 13, 2011 6:21 am 
Team Member Top 50
Team Member Top 50
User avatar

Joined: Tue May 24, 2011 7:51 am
Posts: 249
Interesting! I believe that Sandra might have a network speed test, think it might rely on Windows files shares though. If you feel like running an FTP server, even temporarily, that would at least be one other (pretty lightweight) protocol you could try to push some large transfers through and see if there are any differences there. I have 0 experience with Windows home server, but if they didn't take out IIS from WHS, you should be able to run an FTP server through that, or try one of the free 3rd party FTP daemons that are kicking around.

Thanks for the update! Just an aside, but isn't that file transfer details feature in the file copy window in Win7 such a nice change of pace? Goodbye progress bar that says 30 seconds remaining for 10 minutes! :)


Top
  Profile  
 
 Post subject: Re: Gb Teaming & Transfer Expectations
PostPosted: Thu Jun 23, 2011 6:18 am 
8086
8086

Joined: Sat Feb 13, 2010 12:12 am
Posts: 8
Took me a bit longer than expected to fit some time in and test this.

I bought a pair pf programs from tutusoft to get my measurements. The first was the actual "LAN Speed Test 2.0.5 " And then "LST Server 1.1"

Ran several test that stayed pretty consistent. Here what it spit out:

---Writing--- | ---Reading---
Packet length : 1,048,576 × 100 | 1,048,576 × 100
Avg time/packet : 0.0094122 | 0.0101820
Time to complete: 0.9412213 | 1.0181999
Bytes per second: 111,405,891 | 102,983,317
Bits per second : 891,247,128 | 823,866,536
------------- | -------------
MBps (Mega-Bytes): 106.2449369 | 98.2125444

Not sure what's up with the 98.2MB read time but it seems odd considering. Also the 106MB speed has also occurred frequently using Teracopy.

AliusBU, I do like Win7's progress bar, it is light-years beyond what we had back under XP. Only thing I'm wondering about now is that Win7's reported highs up in the 130MB. No other program I've used gets near that close. Makes me wonder which one of these programs isn't accurate.

I just can't win with this one. To bad Gordon couldn't shed some light on this one.


Top
  Profile  
 
 Post subject: Re: Gb Teaming & Transfer Expectations
PostPosted: Mon Jul 11, 2011 4:44 pm 
8086
8086

Joined: Wed Sep 10, 2008 3:33 pm
Posts: 30
Location: Covington, La.
NIC teaming is generally more or less a farce. It looks really good on paper but its limitations are great. On a home network this may not be an issue but usually as soon as your traffic crosses a layer 3 boundary or gets routed receive teaming becomes totally useless as it depends on ARP to share the load and that is only a layer 2 protocol.

Also and you may know this, but if you team 2 one gig connections that is not a 2gb pipe. It is 2 one gig pipes. So any one data stream will only go down one pipe it is not split between the 2 so you only get 1gb.. Linux can split packets between connections but it does it in such a way that it really violates Ethernet to some degree so that's not good.

Another thing I noticed about your post. And again you may know this but network speeds are shown in mb (megabits) or gb (gigabits) whereas I think most of the stuff you are looking at is in MB which is megabytes. That is an 8x difference.


Top
  Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group