{"id":679,"date":"2012-08-28T15:47:56","date_gmt":"2012-08-28T13:47:56","guid":{"rendered":"http:\/\/www.shukko.com\/x3\/?p=679"},"modified":"2012-08-28T15:47:56","modified_gmt":"2012-08-28T13:47:56","slug":"resolved-intel-e1000e-driver-bug-on-82574l-ethernet-controller-causing-network-blipping","status":"publish","type":"post","link":"https:\/\/www.shukko.com\/x3\/2012\/08\/28\/resolved-intel-e1000e-driver-bug-on-82574l-ethernet-controller-causing-network-blipping\/","title":{"rendered":"Resolved Intel e1000e driver bug on 82574L Ethernet controller causing network blipping ???"},"content":{"rendered":"<p>taken from: http:\/\/www.doxer.org\/learn-linux\/resolved-intel-e1000e-driver-bug-on-82574l-ethernet-controller-causing-network-blipping\/<\/p>\n<p>Earlier I posted a question about centos 6.2 lost internet connections intermittently. Now finally I got the right way to fix this.<\/p>\n<p>Firstly, this is a known bug on Intel e1000e driver on linux platforms. This is a driver problem with the Intel 82574L(MSI\/MSI-X interrupts issue). The internet connection lost itself now and then and there\u2019s nothing logged about this which is very bad for troubleshooting.<br \/>\nYou can see more bug reporting about this at https:\/\/bugzilla.redhat.com\/show_bug.cgi?id=632650<\/p>\n<p>Fortunately, we can resolve this by install kmod-e1000e package from ELrepo.org. To solve this, you need do as the following(ignore lines with strikeouts):<\/p>\n<p>    Install kmod-e1000e offered by Elrepo<\/p>\n<p>    Import the public key:<br \/>\n    rpm \u2013import http:\/\/elrepo.org\/RPM-GPG-KEY-elrepo.org<\/p>\n<p>    To install ELRepo for RHEL-5, SL-5 or CentOS-5:<br \/>\n    rpm -Uvh http:\/\/elrepo.org\/elrepo-release-5-3.el5.elrepo.noarch.rpm<\/p>\n<p>    To install ELRepo for RHEL-6, SL-6 or CentOS-6:<br \/>\n    rpm -Uvh http:\/\/elrepo.org\/elrepo-release-6-4.el6.elrepo.noarch.rpm<\/p>\n<p>    Before installing the new driver, let\u2019s see our old one:<br \/>\n    [root@doxer sites]# lspci |grep -i ethernet<br \/>\n    02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection<br \/>\n    03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection<\/p>\n<p>    [root@doxer modprobe.d]# lsmod|grep e100<br \/>\n    e1000e 219500 0<\/p>\n<p>    [root@doxer modprobe.d]# modinfo e1000e<br \/>\n    filename: \/lib\/modules\/2.6.32-220.7.1.el6.x86_64\/kernel\/drivers\/net\/e1000e\/e1000e.ko<br \/>\n    version: 1.4.4-k<br \/>\n    license: GPL<br \/>\n    description: Intel(R) PRO\/1000 Network Driver<br \/>\n    author: Intel Corporation, <linux.nics@intel.com>\n    srcversion: 6BD7BCA22E0864D9C8B756A<\/p>\n<p>    Now let\u2019s install the new kmod-e1000e offered by elrepo:<br \/>\n    [root@doxer yum.repos.d]# yum list|grep -i e1000<br \/>\n    kmod-e1000.x86_64 8.0.35-1.el6.elrepo elrepo<br \/>\n    kmod-e1000e.x86_64 1.9.5-1.el6.elrepo elrepo<\/p>\n<p>    [root@doxer yum.repos.d]# yum -y install kmod-e1000e.x86_64<\/p>\n<p>    After installation, reboot your machine, and you\u2019ll find driver updated:<br \/>\n    [root@doxer ~]# modinfo e1000e<br \/>\n    filename: \/lib\/modules\/2.6.32-220.7.1.el6.x86_64\/weak-updates\/e1000e\/e1000e.ko<br \/>\n    version: 1.9.5-NAPI<br \/>\n    license: GPL<br \/>\n    description: Intel(R) PRO\/1000 Network Driver<br \/>\n    author: Intel Corporation, <linux.nics@intel.com>\n    srcversion: 16A9E37B9207620F5453F5E<\/p>\n<p>    [root@doxer ~]# lsmod|grep e100<br \/>\n    e1000e 229197 0<\/p>\n<p>    change kernel parameter<\/p>\n<p>Append the following parameters to grub.conf kernel line:<\/p>\n<p>    pcie_aspm=off e1000e.IntMode=1,1 e1000e.InterruptThrottleRate=10000,10000 acpi=off<\/p>\n<p>    change NIC parameters(you should add these lines to \/etc\/rc.local)<\/p>\n<p>    #disable pause autonegotiate<br \/>\n    \/sbin\/ethtool -A eth0 autoneg off<br \/>\n    \/sbin\/ethtool -s eth0 autoneg off<br \/>\n    #change tx ring buffer<br \/>\n    \/sbin\/ethtool -G eth0 tx 4096 #maybe too large(consider 512). To increase interrupt rate, ethtool -C eth0 rx-usecs 10<10000 interrupts per second><br \/>\n    #change rx ring buffer<br \/>\n    \/sbin\/ethtool -G eth0 rx 128<br \/>\n    #disable wake on line<br \/>\n    \/sbin\/ethtool -s eth0 wol d<br \/>\n    #turn off offload<br \/>\n    \/sbin\/ethtool -K eth0 tx off rx off sg off tso off gso off gro off<br \/>\n    #enable TX pause<br \/>\n    \/sbin\/ethtool -A eth0 tx on<br \/>\n    #disable ASPM<br \/>\n    \/sbin\/setpci -s 02:00.0 CAP_EXP+10.b=40<br \/>\n    \/sbin\/setpci -s 00:19.0 CAP_EXP+10.b=40<\/p>\n<p>PS:<\/p>\n<p>    pcie_aspm is abbr for Active-State Power Management. This is somehow related to powersaving mechanism, you can get more info here.<br \/>\n    acpi is abbr for Advanced Configuration and Power Interface, you can refer to here<br \/>\n    apic is abbr for Advanced Programmable Interrupt Controller, it\u2019s somehow related to IRQ<Interrupt Request>. apic is one kind of many PICs, intel and some other NICs have this feature. You can read more info about this here.<\/p>\n<p>Now reboot your machine and you\u2019re expected to have a more steady networking!<\/p>\n<p>PS2:<\/p>\n<p>The reason why there\u2019s so much strikeouts in this article is that I\u2019ve struggled a lot with this kernel bug. Firstly, I thought it\u2019s caused by kernel bug of e1000e driver, and after some searching, I installed kmod-e1000e driver and modified the kernel parameter. Things turned better for a short time. Later, I found the issue was still there, so I tried compile the latest e1000e driver from intel. But neither this worked.<\/p>\n<p>Later, I tried a script which monitored the networking of the time NIC went down. After the NIC failed for several times, I found that Tx traffic was so high each time NIC went to failure(TX bytes went up like 5Gb at a very short time). Based on this, I realized that there may be some DoS attack on the server. Using ntop &#038; tcpdump, I found that DNS traffic was very large, but actually my host was not providing DNS services at all!<\/p>\n<p>Then I wrote some iptable rules to disallow DNS queries etc, and after that, the host now is becoming steady again! Traffic went down as per normal, and everything is now on the track. I\u2019m so happy and so excited about this as this is the first time I\u2019ve stopped an DoS attack!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>taken from: http:\/\/www.doxer.org\/learn-linux\/resolved-intel-e1000e-driver-bug-on-82574l-ethernet-controller-causing-network-blipping\/ Earlier I posted a question about centos 6.2 lost internet connections intermittently. Now finally I got the right way to fix this. Firstly, this is a known bug on Intel e1000e driver on linux platforms. This is a driver problem with the Intel 82574L(MSI\/MSI-X interrupts issue). The internet connection lost itself now [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-679","post","type-post","status-publish","format-standard","hentry","category-kategerisiz"],"_links":{"self":[{"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/posts\/679","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/comments?post=679"}],"version-history":[{"count":1,"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/posts\/679\/revisions"}],"predecessor-version":[{"id":680,"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/posts\/679\/revisions\/680"}],"wp:attachment":[{"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/media?parent=679"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/categories?post=679"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shukko.com\/x3\/wp-json\/wp\/v2\/tags?post=679"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}