Linux performance tuning howto!

[JP83]

Now it's here JP's Linux tuning howto, but maybe *not* for first week newbies. Why this howto: I want that my system runs at maximum speed and i haven't seen much good threads about this.

These are only my suggestions and i warn that if your system goes to not working state when doing these things it's NOT my fault!

Here is the list (i have made all these steps except 1 because Debian doesn't start that much services):

1. Turn off some services that are loaded when booting (in this forum has some threads how to do that to your distro).

2. Use hdparm program to check and change some hd settings (could do much).

3. Make your own "small" kernel from tarball (this could replace step2 like it did to me).

4. Compile/install slow program or program that you are using lot from source code = use tar packages.

5. If you have Nvidia video card install Nvidia driver for that.

6. Updating/reinstalling XFree86 server using tar packages (did huge difference to kde and apps speed for me).

7. Installing new gcc from tar package affects to step 4 performance, but depends on application = (like) multimedia apps difference is bigger.

8. Reinstalling glibc could do something to step 4 (or may not), but i DON'T suggest that if you are new in Linux.



Step 1. I suggest it.

Step 2. Could do big difference to performance.

Step 3. System usually boots faster, (maybe) faster hd usage=replaces step2, could affect that stability of Linux is better. Bad thing is you need to know something about your hardware but lears a lot about your system and Linux. Related to this step you should check some patches check this site:
http://www.linuxdj.com/audio/lad/resourceslatency.php3

Step 4. Can do something (and learns how to install programs).

Step 5. Do if you have Nvidia card (does much if you play 3d games).

Step 6. I must say that in my Debian this was best thing that i have made programs starts up 2-3 times faster in kde2 seriously! Bad thing this can be tricky and could be that no affect to performance.

Steps 7-8 If you are new in Linux leave these alone (atleast glibc it's tricky to get working and install at wrong way = Linux reinstall needed).

And those steps are not in best --> bad affect order.
If you have more let me know it.

[Joe_Atlanta]

Hey JP, thanks! Since my install went fine (no real problems, really easy with Libra's adminmenu) I've been searching for speed tweaks. Yours is the first I 've seen laid out this way.

Question about number three. By "small" kernel" do you mean have all the support stuff load as modules rather than be compiled into the kernel?

[JP83]

That question is difficult personnally i build lots of modules like cdrom support, mouse, agp, usb, vfat, ethernet card support and i build in only needed things in the kernel like xfs filesystem support
network protocol supports that can't be builded safely to modules. And that "small" kernel was that you try to make kernel image about so small that you can but that size depends on your configuration. Like in my Debian custom kernel 2.4.19 + xfs patch image size is "huge" 844KB when in LFS kernel using ext2 filesystem is about 650KB but i'm not happy with that size (i make it to 600KB someday when i have time).

More tunings to Nvidia driver (i haven't tested this):
http://linuxnewbie.org/forum/showthread.php?s=&threadid=77129

[Joe_Atlanta]

Well, obviously this will take some studying. How do I tell how large the kernel is now? I did a search for kernel and found a bunch of stuff in /usr/ including the kernel source and a readme on installing a new kernel.

Fortunately my boot time is not too bad and my hdd support is complete except for 32 bit transfers. Right now I'm using the flag in hdparm to turn it on. If it works ok for a week, I'll figure out how to turn it on permanently. I read over on Debianhelp that I need to edit the hwtools file.

I guess I'll start working my way through the next steps, thanks again!

[JP83]

Well kernel image locates in /boot dir and usually that is named like vmlinuz.... If your distro uses Debian kernel is quite big around 1200KB but boots up is quite fast (unlike RH default kernel). Why i always use custom kernel reason is that i was having small hd performance problems with default kernel and i always use custom kernels because one time i have did that (first week in Linux) and after that i have always used custom kernels because it's easy now to make good kernel. What type hardware you are running and what speed you get if you run hdparm -Tt /dev/hd? (i'm just curious).

[Joe_Atlanta]

Hmmm, more good news about Libranet. Along with their automatic setup, the basic kernel compile (and recompile) options are set to use modules. So my vmlinuz file is only 855k (has Reiserfs and I think usb support compiled in).

Wow, I just found out about rcconf so turning off unneded services permanently is going to be much easier.

Ahhh, hardware. :D :D I'm a total newbie at tweaking Linux, but an old hand at tweaking hardware. I built my new box about 4 months ago after dragging along with an old K62 socket 7 based system for 4 years.

My new system:
MSI 745Ultra (SIS 745 chipset)
AMDXP1600 overclocked to 1775mhz (10.5X169, XP2100 equivalent)
1 gig (2 X 512) Samsung PC2700 memory
WDJB80 hdd - 80meg w/ 8mb cache

/dev/hda:
Timing buffer-cache reads: 128 MB in 0.36 seconds =355.56 MB/sec
Timing buffered disk reads: 64 MB in 1.37 seconds = 46.72 MB/sec


I don't know if this is good or bad. Hdd performance feels a bit slow compared to W98. This maybe because Linux is installed on the slowest part of the disk. When I resized my partitions, I took the space from my last extended partition. I plan at this point to reinstall over the holidays, putting my Linux partitions next to my Win boot partition. It should help.

[JP83]

/dev/hda:
Timing buffer-cache reads: 128 MB in 0.36 seconds =355.56 MB/sec
Timing buffered disk reads: 64 MB in 1.37 seconds = 46.72 MB/sec

Wow... :)

I usully get:

/dev/hda:
Timing buffer-cache reads: 128 MB in 0.84 seconds =152.38 MB/sec
Timing buffered disk reads: 64 MB in 1.96 seconds = 32.65 MB/sec


/dev/hdb:
Timing buffer-cache reads: 128 MB in 0.87 seconds =147.13 MB/sec
Timing buffered disk reads: 64 MB in 2.39 seconds = 26.78 MB/sec

But my hardware is older AMD Athlon 600MHz (slot A) and ATA66. My hda is same as yours WD800JB, hdb is old Seagate.

[Joe_Atlanta]

Originally posted by JP83
Wow... :) I'm glad to hear that these are good numbers. One of the main things I was aiming for when I built this box was good hardrive performance without going to RAID or SCSI. I do a lot of audio editing and the program I use saves the entire file (30-100mbs)everytime I make a change.

BTW, I got the Nvidia drivers installed, so I'm still working my way down your list. http://www.gamers-forums.com/smilies/otn/realhappy/biggthumpup.gif

[fancypiper]

Another speed tweak you might want to investigate is the preemptable kernel patch. See: Linux kernel preemption project (http://kpreempt.sourceforge.net/)

[Joe_Atlanta]

Thanks for the link FP. I had read a little about that patch and it seems worthwhile to check out. Can't find a whole lot of newbie friendly info on it ("step 1, download patch" :p ). I started reading up on patching kernels and of course I have to a little more digging to find the "Debian way".

Doesn't Gentoo use that as the default in it's build?

[fancypiper]

Originally posted by Joe_Atlanta
Doesn't Gentoo use that as the default in it's build? It's included in the gentoo-sources kernel along with some other nice patches :cool: I don't know about the other kernel sources as I haven't had time to try them out. uml-sources looks very interesting.

I never could figure it out in Redhat :mad:

[JP83]

fancypiper actually there was that (almost same) link in my list (in comment list) but good that you posted it now it shows better. I use preemptable kernel patch.

[Durph]

Will this only work for Debian? How about Mandrake? I have and AMD Athlon 850 and a meare 96mb of memory. As most people my goal is to have my computer run as fast as possible. Do any linux versions run faster than others? Oh yeah. You probally smell that too. Im a newb. Well anything that you say can help. Thanks.

[JP83]

Originally posted by Durph
Will this only work for Debian? How about Mandrake? I have and AMD Athlon 850 and a meare 96mb of memory. As most people my goal is to have my computer run as fast as possible. Do any linux versions run faster than others? Oh yeah. You probally smell that too. Im a newb. Well anything that you say can help. Thanks.

It will work with all distros Debian, Mandrake, Red Hat... Some distros will run faster = source based distros like LFS, Gentoo... because software is compiled to that computer --> less compatibility but faster running. Note that if you rebuild some areas in your distro that will be like source based distro. I suggest that you start with closing services that you don't need. Everyone Linux user has been a newbie. Actually i think that doing these thing learns how Linux works and how to use that and if/when you manage to do these steps you are fastly moving to guru...
You probably know this but generally would be good idea to buy litle more memory (normal memory is quite cheap) if you are KDE or Gnome user.

[Durph]

Im looking at getting some memory for Christmas. Hopefully. I run Gnome and as soon as I can fully understand what you are talking about I might give it a try.

[Fryguy8]

here's the best optimization hint.

Recompile your operating system with gcc optimizations :):)

[The Ennead IX]

Not too sure what the performance gain is since i've not done any comparatives but the old "ditch kde and use fluxbox" maxim is certainly one that fits in with turning off services. I'm messing about in those files at the moment so booted into both kde & fluxbox out of interest.

Running Gentoo with gkrellm2 giving the readouts:
KDE-3.0.4 weighs in @ 48 running processes
Fluxbox? A rather lighter weight 30 processes.

[Joe_Atlanta]

Originally posted by JP83
I use preemptable kernel patch. Well, I got my kernel source patched and recompiled. I'm not seeing a huge difference in day to day tasks. However, my client for Distributed Folding (http://www.distributedfolding.org/index.html) seems to be crunching a few thousand more structures per day. :cool:

What about the lock break patch? Have ya'll found it necessary to install that along with the other?

[Joe_Atlanta]

What about the lock break patch? No problems so far, so I guess I don't need the lock break patch.

[sharth]

Originally posted by JP83
/dev/hda:
Timing buffer-cache reads: 128 MB in 0.36 seconds =355.56 MB/sec
Timing buffered disk reads: 64 MB in 1.37 seconds = 46.72 MB/sec

Wow... :)

I usully get:

/dev/hda:
Timing buffer-cache reads: 128 MB in 0.84 seconds =152.38 MB/sec
Timing buffered disk reads: 64 MB in 1.96 seconds = 32.65 MB/sec


/dev/hdb:
Timing buffer-cache reads: 128 MB in 0.87 seconds =147.13 MB/sec
Timing buffered disk reads: 64 MB in 2.39 seconds = 26.78 MB/sec

But my hardware is older AMD Athlon 600MHz (slot A) and ATA66. My hda is same as yours WD800JB, hdb is old Seagate. so that means I get to cry right? :)
debian:/boot# hdparm -Tt /dev/hda

/dev/hda:
Timing buffer-cache reads: 128 MB in 2.05 seconds = 62.44 MB/sec
Timing buffered disk reads: 64 MB in 26.27 seconds = 2.44 MB/sec
debian:/boot# hdparm -Tt /dev/hdb

/dev/hdb:
Timing buffer-cache reads: 128 MB in 1.51 seconds = 84.77 MB/sec
Timing buffered disk reads: 64 MB in 23.96 seconds = 2.67 MB/sec
debian:/boot#

[Fryguy8]

what settings are you usign to get

355 and 46? I'm getting 278 and 37 on my ATA100

[fancypiper]

Originally posted by Fryguy8
what settings are you usign to get

355 and 46? I'm getting 278 and 37 on my ATA100 You could probably use these settings:

hdparm -X69 -d1 -u1 -m16 -c3 /dev/hda

I would do this first:

hdparm /dev/hda

This tells you the parameters you have for defaults. Then run a timing test several times:

hdparm -Tt /dev/hd{a,a,a}

The most accurate results would be by running this test with the drive having no running processes on it (linux single is good).

This command runs the timing test 3 times. You can adjust the number of tests by the drive letter repeats in the {}.

Adjust your flags, one at a time and run the timing test to see the results. If your system doesn't freeze with any of these flags, you can use them if they increase performance.

[Joe_Atlanta]

what settings are you usign to get 355 and 46? Pretty much as Fancy Piper outlined above. Libranet set up everything for me except 32 bit .

The pci/ide controller on my mobo (SIS 745 chipset) is really great. It averages 20-30% faster than the VIA chipsets. Plus I'm overclocking the pci bus just a tad, running at 34mhz.

You never hear much about these boards because the memory bus is a little bit slower than the VIA boards. So they are not the first choice for extreme overclocking and gaming. However, for video and audio editing (heavy hdd use), they are great.

[Fryguy8]

fancypiper, those are the settings that I am using, and still I'm around 22% slower than that number. What else am I missing?

[fancypiper]

Originally posted by Fryguy8
fancypiper, those are the settings that I am using, and still I'm around 22% slower than that number. What else am I missing? That may be the limit of your drive/mobo combination. When I switched from a 5400 RPM drive with 2mb cache to a 7200 with 8mb cache, my numbers went way up.

[Fryguy8]

I'm on a 7200RPM ATA100 2mb cache VIA KT266A mobo.

[fancypiper]

You are using the 80 conductor ide drive cables, aren't you?

My timings

ECS K7S5A mobo and Western digital 7200rpm 8mb cache on an unshared ide bus:

/dev/hda:
Timing buffer-cache reads: 128 MB in 0.91 seconds =140.66 MB/sec
Timing buffered disk reads: 64 MB in 1.37 seconds = 46.72 MB/sec

[Fryguy8]

Yes I'm using ATA100 cables, not the ATA 33/66 cables, and it's an unshared bus.