2.6.X Kernel Stability

[DSwain]

Well this is just something I picked up personally, and I don't know if its universal or not. While using the 2.6.X series of kernels, I've noticed a major difference in stability from the 2.4.X series. Not that it freezes or anything of the sort, but I've noticed for example, when I get say a terminal compiling software in the background, and I attempt to run something else, it will almost skip if you will, and just seems to bog on a lot of system resources, whereas the 2.4 kernel never did that for me. I'm just curious to know if there's something I'm doing wrong with the kernel, or something I should build into the kernel which may resolve this issue or what the problem be. Just for a little info, this has been on every version of the 2.6 kernel, and every distro I've built the 2.6 kernel for (Gentoo, Slackware, Red Hat.)

[Dark Ninja]

I'm currently running Gentoo with the 2.6.6 kernel, and I'm having absolutely no problems. (In fact, I've definitely noticed an improvement over 2.4.x kernels.)

Not sure what to tell you, though...you may want to check what you're compiling into your 2.6 kernel when you create it -- that could definitely have an adverse effect if you include some of the experimental/new "stuff."

[DSwain]

For the most part its all been improvement for me also. faster, easy to use, things like that, but this little problem has been annoying me a bit. I don't think i'm adding anything strange to it... or maybe it's what I'm not adding. Not sure i'll play around unless somebody has a specific idea for me to try.

[Dark Ninja]

Well, when your machine "skips" -- is it always the same pieces of software that you are booting? Are you always doing the same things?

Look for commonalities between the systems to see if the same stuff is causing this skip.

The other possible problem (which seems more likely to me for some reason -- maybe 'cause a lot of people would have noticed a major bug like this with the kernel) is that you have installed a program on your different machines (the same program) and it causes this flaw. That's why you should take note of what is running whenever you notice your system skipping around.

Hope you figure it out!

[DSwain]

well no its really any software at all. I'm not exactly sure how to explain it, but i'll try with this example. Say i have XMMS running playing a song. Whenever i (say on gentoo) start building software, while it unpacks and things like that, the music will stop and then continue when resources come back to it. I never had any problem like this on the 2.4 kernel. I highly doubt its a major bug either, but I just need to know what i'm doing wrong. I doubt its from any software because it can happen with anything.

[Dark Ninja]

That is odd. I'm gonna do some research into it ('cause it makes me curious -- especially 'cause it has happened to you on multiple systems) and see what I can come up with.

In the meantime, do you mind pasting your .config file in here for your kernel configuration? Oh yes...and what is the exact version number of the kernel you are using? (2.6.?)


Thanks

[DSwain]

generally i use the 2.6.4 kernel, but I am trying out the 2.6.5-gentoo kernel now to see what happens with it. I will post when i finish installing it which should be within the next few hours (i pray) long time to install stage1's sheeesh.

Oh yeah... um in make.conf, within the CFlags, can -O4 be an optimisation flag? I usually us -O3 but i wanted to see if -O4 would do any different. This is probably dangerous lol.

[squeegy]

iirc, anything above -O3 is just brought back down to -O3. You do realize that using very aggressive CLFAGS can lead to system instability? I've also found 2.6.6 to be the best 2.6.x kernel I've used. At the moment I'm using love-sources, search the gentoo forums for more information on these. It's a kernel patchset built off of mm-sources. It is very "bleeding edge" but it makes your system ROCK LIKE NINJA (as they say). Being you're using gentoo you can just download the ebuild, put it in your portage overlay directory, ebuild <file> digest, emerge <file>. You can find more information on installing them on the gentoo forums as well...

You may have luck with some other more "conventional" patchsets, such as ck or mm. What I think really helps performance that is in love, is Nicks scheduler work, the cfq io scheduler, and some other virtual memory work.

[Dark Ninja]

Common flags
The -O (the letter, not a zero) flag is used extensively when compiling anything. The 'O' stands for optimize, which is exactly what this flag does. By including a number or letter after it, you can tell GCC the type of generic optimization you would like. Here are some examples on how to use this flag:

-O3
means you would like level 3(the highest level) optimization. Each level of optimization builds on the previous level by including more specific options. Effectively, the -O# flag encapsulates many other options.

Note: Optimizing requires more compiling time, and resources. In the case of level 3 optimization, there is a space-speed trade-off, and you will usually end up with a larger binary.


The rest can be found here: http://gentoo-wiki.com/CFLAGS

It's an interesting read.

[Dark Ninja]

You do realize that using very aggressive CLFAGS can lead to system instability?

Actually, yeah, I was thinking the same thing. Sometimes the performance gain really isn't a performance gain when you push your system just a little too far. (Heh heh...certain hdparm settings come to mind here...)

it makes your system ROCK LIKE NINJA

Yeah. Like me. Remember that. :D

[DSwain]

hmm i understand. I realise pushing too too far is bad, but i wanted to see if pushing above O4 was possible. Not a big deal though. Thanks for the tips. I may look into the love-sources you are talking about.

[Dark Ninja]

Well, just for reference, -O3 is sometimes pushing too far. Depends on the rest of your system setup.

(Now, of course, if this is what you used before with the 2.4 kernel, then I don't think that would affect it.)

[DSwain]

i never used gentoo with a 2.4 kernel, ever. I'm a fairly new Gentoo user, so i haven't used it very long. 2.6 is overall preferred for me anyways, so it makes more sense to me.

[Dark Ninja]

Ah, okay. Well, you may want to drop that -O setting down a notch...see if that fixes your problem.

I'll keep checking back for the .config file whenever you post it. Good luck.

[DSwain]

thanks. Normally I ran it at -O3, and i've run it at -O2 also without resolution. I know its something to do with the kernel, because its happened to me on all the 2.6 kernels i've built on and Linux distro.

[DSwain]

question, where can i find the kernel config file? I have no idea where to look lol. sorry never had to deal with the kernel.conf file ever.

Oh, and btw, with my -O4 flag, it hasn't been very unstable, at all. As a matter of fact, its only faster now, a good deal too. Nothing but improvements so far. It doesn't seem unstable, yet.

[Dark Ninja]

The .config file (after you compile the kernel) can be found in

/usr/src/linux assuming that you made the link to point the version of Linux you were compiling. (AKA - Follow the Gentoo setup.)

Well, now that it seems okay, keep an eye on it. If you see something start to jump after you install an application...then...DEFINITELY check that out.

[ghostwalker]

I notice you don't show how much ram you have in your system config? Also with the 2.6.6+ version there are three different type of schedulers, as (default), cfq and deadline. cfq is for mostly multmedia apps.
To use it just pass the parameter in you lilo.conf or grub.
append=cfq .. for lilo.conf

[DSwain]

no it still happens, i'm just saying that i have no software problems with the -O4 flag. So far its been building the software faster and running them faster. overall operation has only improved. I still have the skippingness of it.

I'm going to try to clarify what i'm asking, in case i didn't enough, it seems like i haven't. When i'm doing a good load of work, say emerging some packages, and i attempt to run something else, say mozilla, it tends to kind of give a "skipping" feeling to it. Like it will be building a package, and the mouse won't move and everything seems like it stopped, and then it comes back. Its kind of like with old versions of Windows (ME, 98,95 old) where you would be doing something, and another process would take up a lot of cpu power and cause the system to almost pause in a sense, like music in the background would stop, the mouse would stop, etc.

As for the kernel config, i do not see it in either my symlink or the original directory. I will take another examination for it though.

512mb of RAM, 1gb swap. Like i said this problem never occured with the 2.4 kernels, just 2.6's on any Linux Distro.

[ghostwalker]

I messed up guys it is append elevator=cfq....

[cybertron]

I think I've actually run into the same thing. Every once in a while with a lot of stuff running the mouse starts to jump and sometimes MPlayer blips off for a second. The blip also happens when the screen comes back on after it's been in powersave mode. I'm pretty sure it's happened on both the 2.6.3 and 2.6.6 kernels.

Of course, I only used the 2.4 kernel for a few months and I was coming straight from Windows at that time, so Linux seemed so much faster and smoother that I wouldn't have noticed this (except booting, Mandrake 9.1 + KDE was really slow:)). Plus I've only got 256 meg in my laptop, so performance in general isn't spectacular:)

Anyway, just wanted to let you know that you're not the only one who has noticed something like this.

[ghostwalker]

Also what is the nice level are you running X, it is probably -10 try running it as 0. renice X or top and select r and you PID of X

[DSwain]

i have no idea what i'm running X at... i didn't know there was a level to run X at. I have no idea honestly lol, how would I go about checking something like this?

[squeegy]

Unless something changed that I don't know about, -O4 just gets interpreted as -O3. Just man gcc, search for the O3 flag, it is the highest optimization flag there is. If you read there you'll find out what CFLAGS each -O level implies.

[DSwain]

i've read that myself too, so i'm not denying that its true. But watching the code compile, none of it changed, it all stayed at -O4, and also it seems to be working much faster, like building software seems to be faster, and apps seem to be running quicker also.

I don't know, its going faster as far as i can tell.

[Dark Ninja]

Mmm...actually...there are things better than O3. Yeah...I prefer Smirnoff Twist myself...

:: sits ::
:: blinks ::

OKAY...now that I have that out of my system, squeegy is correct, -O3 is the best you can do. However, your speed increase may be due strictly to the fact that you...are...using...GENTOO.

:: crowd roars with enthusiasm ::

Welcome to our side. :D


[filed under goofy mood]

[DSwain]

okay, well I am still yet to find the kernel.conf file. I've done "whereis kernel.conf" and it showed all of my kernel source (and symlink) directories, and looking in there for hidden files and normal files, none of them happen to be kernel.conf so is there something I did wrong in the configuration of my kernel to not have it generate the file?

[squeegy]

the kernel config file is .config located under the /usr/src/ directory pertaining to that kernel.

[bwkaz]

find / -xdev -name .config

If you have any other directories on separate partitions, follow this up with:

find /that-directory -xdev -name .config

for each of the relevant directories.

(The -xdev directive makes it so that find does not recurse into other filesystems, like /proc and /dev/pts -- which it shouldn't recurse into. It also makes it not recurse into any other partitions that are mounted in places like /home, which is bad, so you need to do the second group of find's to get it to search each of those.)

[rocketpcguy]

i haven't notcied such a thing, the mouse never ever skipped, fedora 1, linux 2.6.3

[DSwain]

alright I've got the config file now, thanks for the help on that, here it is (i'll try and attach it if it isn't too large)

[Dark Ninja]

Okay...well, I just started going through your config file to see what you had set/not set whatever. There are a couple things that you may want to change. Of course, I am no kernel expert, so if there is someone here who disagrees with my opinion, please let me know (and let me know why, as well).

BTW -- Haha, I am recommending these settings based on the recommended kernel settings that are located in the kernel's help file. So...if you are curious as to why, check that out in the 'make menuconfig'

Things to Set (Turn On)
------------------------------
CONFIG_BSD_PROCESS_ACCT


Things to Unset (Turn Off)
---------------------------------
CONFIG_AUDIT
CONFIG_MICROCODE -- Intel Processors Only
CONFIG_X86_MSR
CONFIG_X86_CUPID
CONFIG_MATH_EMULATION

Questionable?
-------------------
CONFIG_PM -- Personally, I don't need Power Management and I think it's sort of useless if you don't use a laptop. But, that's just a matter of personal preference.

CONFIG_BINFMT_AOUT -- Make sure you really don't need this. The a.out binary is not in much use anymore, and is generally not needed.

CONFIG_LBD - You have a RAID or Loopback that is > 2 terabytes? I'm thinking you probably don't need this.

CONFIG_SCSI_PROC_FS - There's a few applications that use this still. I actually asked about it in the Gentoo IRC at one point and I was told it's a good idea to keep it in for now.



Those are the things I noticed. I'm sure I could get even more picky if I was able to sit down at your system and start looking through it. But, that's my recommendations. Again, make sure you don't overwrite your currently working kernel if you decide to play around. Otherwise...you will be starting back at square one. :p

Good luck.

[squeegy]

try adding elevator=cfq to your kernel append line in either lilo.conf or grub.conf, this will set your IO scheduler to cfq, it works really well for desktops... cfq == complete fairness queueing if you were curious. See if that improves your lagging issues.

[DSwain]

alright i will try alll of those things. The only reason i use ACPI at all is for powering down my box. For some reason without it, I can't shutdown the actual computer and it just reboots itself, so I just leave that there. But like I said I will attempt both of those things. Thanks for the tips

[Dark Ninja]

:: confused look ::

The 'halt' command (as root) always worked for me. The computer doesn't automatically power down, though...I do have to press the power button after it says it's safe to do so.

[DSwain]

i dunno but I don't think it causes the skipping i'm suffering from. I would doubt it anyways.

alright well I just tried the editing the grub config line. It is using the cfq scheduler but not seeing improvements yet. I will take an attempt to fix up the kernel next.

[hard candy]

Add "apm=power-off" to your bootloader config file. Lilo to the "append" line and grub to the kenel line. Enable apm in the kernel config (not as module). No other options for apm are needed. Your computer will shut down completely then.

[squeegy]

Originally posted by DSwain
i dunno but I don't think it causes the skipping i'm suffering from. I would doubt it anyways.

alright well I just tried the editing the grub config line. It is using the cfq scheduler but not seeing improvements yet. I will take an attempt to fix up the kernel next.


squeegy@mars ~/ $ dmesg | grep scheduler
Using cfq io scheduler


try that command and make sure it says you're using cfq.

[Dark Ninja]

try that command and make sure it says you're using cfq.

What if it doesn't say anything?

(Note: I didn't make any of the changes you said yet. I just wanted to see what scheduler I was currently using.)

[squeegy]

My guess would be the anticipatory scheduler.

[blackhawk714]

Hey guys. I have the same problem. It doesn't really bother me that much, but I thought I would let you guys know just the same. I did get a tip from roger55 that said something about adding the line PORTAGE_NICENESS=20 in the /etc/make.conf I think. Just givin you guys something to go with. Later .. if ya fix it post back lol. :D ;)

[Dark Ninja]

squeegy - You wouldn't know of any good articles that describe the advantages/disadvantages between the different types of schedulers? I searched online, but really couldn't find anything.

[squeegy]

Originally posted by Dark Ninja
squeegy - You wouldn't know of any good articles that describe the advantages/disadvantages between the different types of schedulers? I searched online, but really couldn't find anything.

I used to have a few links describing them, but I can't seem to find them right now. You might wanna take a look around www.lkml.org. I'll look around and try and find the links.

[squeegy]

http://lwn.net/Articles/49966/

http://linuxtoday.com/developer/2003022502126OSKNDV

just some random bits from google.com/linux theres lots of stuff to be read.. search for IO schedulers.

[DSwain]

alright just took another look at the scheduler, and as i saw during the boot, it is the cfq scheduler. I am still yet to play with the kernel config though

[Dark Ninja]

DSwain --

I was reading this link here: http://kerneltrap.org/node/view/3081#comment

Go down a bit on the messages. Apparently other people are having the same problem you are. Hmmm...well, at least we know you haven't lost it and there really is a problem! ;) :p

I'll keep searching around and see if there's a fix or an explanation of why the problem occurs.

[DSwain]

hmm, according to most of those posts it seems like a large amount of memory would do better... but nonetheless that doesn't resolve the issue. At least we have it pretty much down to the kenel though. And no, I don't have that severe problems. It gets sluggish for a few moments, but then responds fine after its finished a certain task. I can stop X very quickly almost all the time and I rarely need to anyway.

[Dark Ninja]

Okay...so I'm stealing the thread a bit, but just a quick question.

Squeegy, you suggested the use of the CFQ scheduler over the Anticipatory scheduler. Any reason? The only reason I ask is it seems some of the info on KernelTrap is saying CFQ isn't ready for primetime yet.

Haha...I'm gonna test it out anyway. ;-) See if things seem to move quicker. If not, it's easy enough to go back.

Hmm...wish there was like a list of all of 'em and why one is better than the other. Something like that would be nice. I may do it myself if I can find enough info.

Back to Google!

[squeegy]

I read somewhere that the CFQ IO scheduler is good for desktops for some reason or another. In regards to it not being ready for primetime yet, I don't think anything on my system is "conventional" by any means. If you look at my sig i'm currently running 2.6.7_rc2-love2, which is based off of mm-sources, plus a bunch of other patches, here's fhe list


v30g.gz | Nick's scheduler v30g (v31 was dropped by him)
nickvm-267r2m2.gz | Nick's VM work
2.6.x-bring-up-compiler-option.patch.bz2 | extra CFLAGS hack
acpi-dsdt-initrd-patch-v0.5-2.6.5-eric.patch.bz2 | Custom acpi dsdt patch
bootsplash-3.1.4-2.6.7-rc2-r2.diff.bz2 | bootsplash
acx100-0.2.0pre7-linux2.6.2.diff.bz2 | ACX100 Driver (upped to _pre8 by hand)
orinoco-cvs-20040527.patch.bz2 | orinoco patch
squashfs2.0-linux-2.6.6.patch.bz2 | SquishFS v2
subfs-genstef.bz2 | SubFS (death to supermount!)
acerhk.patch.bz2 | acer hotkeys
BadRAM-2.6.5.2.patch.bz2 | a workaround bad ram
bd-claim.patch.bz2 | EVMS fix
config-nr-tty-devices.patch.bz2 | for a cleaner /dev
force-config_regparm-to-y.patch.bz2 | reversed for binary drivers' sake
ipw2100-0.45 | new ipw2100
genpatches/2110_bcm5700_broadcom_gigabit_drvr_11272003.patch | bcm5700 driver
genpatches/4105_lirc_infrared-2.6.5-rc2.patch | lirc patch
genpatches/4710_lufs-0.9.7-2.6.0-test9.patch | lufs
genpatches/4905_speakup_accessibility.patch | speakup patch
iriver.patch.bz2 | iriver
iteraid_1.44.patch.bz2 | Giga Raid
linux-2.6.0.dxr3.20031229.diff.bz2 | dxr (is there's a new patch somewhere?)
linux-2.6.4-Walken.diff.bz2 | the Walken bootlogo
love_ppm.patch.bz2 | the love bootlogo
packet-2.6.5.patch.bz2 | Packet CD/DVD writing
*can't find* linux-2.6.6-mppe-mppc-1.0.patch.gz | searching for it, the old one is broken
patch-2.4.x-vesafb-rrc.bz2 | vesafb refresh hack, couldn't find the new one
patch-2.6.4-amd76x_pm.bz2 | AMD Power NOW!
sii-3512-update.patch.bz2 | siimage patch
viafb_02.diff.bz2 | via framebuffer
via-v4l-1.4a-drm.patch.bz2 | via v4l stuff
menuconfig-NAME-v1.0.patch.bz2 | name in menuconfig patch
win4lin.patch.bz2 | send your hugs and kisses to PickledOnion


I don't think my system has ever been so responsive since using the love-sources. I think the NickScheduler v30g and Nick's VM work in addition to mm-sources is the source of the improvements. Keep in mind this all for my desktop machine, I still use 2.4.26 for my server.

I am also using udev, which you using Gentoo should think about using, it's really easy to switch over. In addition to udev, i'm using hal+dbus+ivman to handle the automounting of removable media, ie. my cdrom drive. I've also heard of hal+dbus working with gnome-volume-manger too.

I guess my point is my system is far from the "typical" system, I just think it's fun to experiment with new things like this, I guess this is what happens when I can't break my gentoo system no matter how hard I try. :D Sometimes I have to spend some time fixing things, but I am always up for the challenge.

[Dark Ninja]

For anybody who is curious as to what squeegy is talking about (and is still following this crazy long thread) head here to check out udev:

http://www.gentoo.org/doc/en/udev-guide.xml

[Dark Ninja]

Hmmmm...squeegy -- according to this UDEV article, if I use udev, I don't need to have

/dev file system support (EXPERIMENTAL)
Automatically mount at boot

...those two items selected in my kernel anymore. Is this true? (I am considering using udev -- but I really don't want to screw things up. Haha.)

[squeegy]

Yeah, you can remove that stuff if you plan on using udev. In addition to the howto on gentoo's site, hayl also wrote one here (http://www.justlinux.com/forum/showthread.php?s=&threadid=123721&highlight=udev)
It's really pretty easy to change over. Only thing I had issues with was the /dev/null and /dev/console stuff, but it tells you how to fix that in almost every guide out there now.

[lxmusic_man]

Originally posted by squeegy
try adding elevator=cfq to your kernel append line in either lilo.conf or grub.conf, this will set your IO scheduler to cfq, it works really well for desktops... cfq == complete fairness queueing if you were curious. See if that improves your lagging issues.

This is a bit of an offtopic...

I have been following this thread, and see references to the love sources for the kernel. I'm running FC2 with a 2.6.6 kernel I built myself with the realtime-lsm patch applied to get low latency as a module.

I don't know how to apply the love patches or where to get them. I dowloaded the -mm patches and tried to apply it to the clean 2.6.6 kernel tree but got a lot of rejects, so I don't know if I'm doing something wrong or what...

lxmusic_man

[blackhawk714]

There is a line that you have to add in the /etc/make.conf.

PORTAGE_NICENESS="19"

That puts every application in front of portage in priority so that the other programs don't skip. Hope that helps all of you. :D ;)

[miko]

Just try to switch off Preemptible Kernel option in the kernel configuration.