Compile into kernel vs modules

[tmcG]

This is a question that has probably been asked many times by many people but I have not found much info on Google that answers it to my satisfaction.

When compiling a kernel, is it better to compile modules, drivers, etc directly into the kernel or run them as modules so they load as needed?

I have done both on my Slack system and have not found a significant performance increase in either situation so I am proably not doing something correctly.

I have only compiled my first kernel recently so I am still getting the hang of it, but was hoping JL members could inform me.

Any advice/comments would be greatly appreciated.:)

[ph34r]

Some things obviously have to be compiled into the kernel - file system support for your / filesys, stuff like that. Most everything else can be modules, if you desire. Used to be that the kernel couldn't be more than a certain size, mostly due to being able to use a floppy to boot (DOS system disk, loadlin.exe, kernel, perhaps an initrd file, etc all had to fit in 1.44mb).

On older hardware, you may be able to tell a difference (like on 386/486 systems with less than 16mb ram) but on anything else I doubt you'd notice a difference.

[XiaoKJ]

I have a similar problem.

I tried to compile everything I need as a built-in, not as modules.

At first everything worked fine, till my old lan router broke and I reverted to the alcatel speedtouch usb.

It was only then did I realise that the computer cannot recognise the modem!!! how can I make the speedtouch modem be used and appear in ifconfig -a?

I have to compile it as a module?

Can anyone tell me this stupid phenomena? the device is detected but not used. I have the 2.6.6 kernel but compiled in slackware.

[Pafnoutios]

I compile everything I know I need into my kernel from the kernel tarball: my sound drivers, network drivers, pci drivers, filesystem modules, etc... I only have module loadability for my dxr3 card and nvidia driver.
I don't hot-swap any usb peripherals, though. If you hot-swap different usb peripherals then I would recommend compiling those drivers as modules. I don't change my hardware often, either. If you're frequently changing hardware or swapping it between different machines, then I would recommend comiling those drivers as modules. When I do change something, I plan to recompile the kernel with module support for my present and new hardware, then after verifying the new hardware works properly, compile the drivers directly in.
I, personally, see little use to compiling modules that will need to be loaded every time you boot and used the whole time the machine is on. I think using modules is best for stock distribution kernels which get installed on many different computers, or on a machine with frequently changing hardware configurations. Unless you think you have a reason to compile something as a separate module, just compile it in.

[bwkaz]

Originally posted by Pafnoutios
I, personally, see little use to compiling modules that will need to be loaded every time you boot and used the whole time the machine is on. Exactly. There is NO advantage to using a module when it's going to be loaded the entire time the machine's up anyway.

For example, making usbcore a module is extremely pointless -- you need it loaded to mount /proc/bus/usb (because that directory doesn't exist before it gets loaded), but once you mount /proc/bus/usb, you can't unload usbcore until you umount it. The module is pinned into memory the entire time the machine is on; at that point, you'll actually save a little bit of kernel memory by compiling it directly in (the functions, data structures, etc. that are marked with __init can be removed by the kernel after it's done booting if the driver is built-in, but I don't think they can if it's a module).

If you have hardware that's a PCI card, you're going to want it built-in, for the same reason (unless you've spent a bunch of money on a hot-pluggable PCI bus; most people haven't). You'll save a little bit of kernel memory by not making them modules.

The only thing modules are good for is, like you've said, when the kernel is going to be used on different hardware (kernels that distros compile), or when the hardware isn't always going to be there (USB devices, for example), or when the kernel module is not distributed with the kernel (nVidia/ATI video cards).

[micro]

The concept of extending a kernel's capabilities at runtime with the help of detachable code pieces is not to be taken lightly.

This feature is practical in development and debug times as well as in regular use.

A module can be detached. Code inside the kernel can not.

In development time, a device driver or a network protocol can be compiled, attached and tested and if something (not fatal) goes wrong, can be detached, reprogrammed and so on, without the need to reconstruct a whole kernel and reboot.
This feature turns our Linux boxes to excellent development platforms.

In regular use, I find it very practical also. Here is an example:

I use to work in the ttys when I do not need a graphical environment or setup a new installed Linux OS in my computers. One night my wife opened the door annoyed by the pc speaker sound events that the tab generated, but i could not use a graphical environment yet.

So, I recompiled the kernel with the pcspkr as a module and used it only when I needed the sound of it.

In another case, I did a full upgrade to my computer. In other words, I took the disks out, put them in the new machine and the kernel behaved as if nothing happened, despite the fact that even the sound and network cards were totally different (via82xx and rtl8139 as opposed to sbpci128 and via rhyneII)

In general, I compile a most minimal kernel (pci ide scsi-emulation ext2 ext3) and ALL non experimental code as modules. In my fanatic times, I even have the parallel and serial ports as modules.

After this compilation, trying to put the system to work with all my peripherals can be a very educational procedure.

The module concept does not make slower kernels. Only debugging options and frame pointers do this.

[tmcG]

Thank you to all who have replied and offered their advice/opinion, it has helped me understand the role of modules and their benefit much more accurately!

I will now use this advice to recompile my kernel in a more efficient way :)

[Fryguy8]

I loaded everything that's going to be basically full-time use (sound, net, mouse, keyboard, etc etc). And then, things like floppy and cd-rom were so minor, I just ended up compiling those in too, it's like 1k of memory if I'm not using them.

[bwkaz]

Originally posted by micro
annoyed by the pc speaker sound events that the tab generated, but i could not use a graphical environment yet. So do a set show-all-if-ambiguous in your inputrc file (/etc/inputrc or ~/.inputrc), to turn the beeping off. It's pointless anyhow: if you only want to see the list of files instead of the beep, make readline not beep. Don't recompile the entire kernel just to disable the speaker. ;)

The module concept does not make slower kernels. I didn't say slower, I said bigger. ;) There is more (non-pageable, because it's kernel space!) memory in use when you've got the serial port module loaded than when you have the serial driver compiled in. It may not be noticeable, but this is memory that can never be swapped out, and will never be used again! It's a complete waste.

Yes, modules are helpful when debugging kernel drivers. But who of us is doing that? OK, I'm sure a couple of people here do (actually, I remember a couple of threads in the programming forum about them), but I don't, and I doubt that most people do.

Yes, modules are helpful when you want to disable something. But wasn't that posted already? Hot-swap?

Yes, modules are helpful when a kernel gets run on different hardware -- but why not recompile the kernel at that point? Why have modules installed (not inserted, just sitting in /lib/modules) that you're very likely never going to use? Unless you're a distro -- then that makes sense.

But when I upgrade, I know the hardware I'm upgrading to; it makes a lot more sense to rebuild a kernel for that specific hardware, and skip the time it takes to build all possible drivers when I won't ever use most of them. I don't know right now the next video card I'm going to get, so how can I decide which framebuffer driver to compile? I'd have to do them all, or just wait until I get the card, and at that point, it makes more sense to build the driver into the kernel.

[micro]

Well, it is better to have a module detached and reattached in runtime than to edit an /etc configuration file and reedit it later on to get the beep luxury back on.

I have walked the non-module path of course and I have to say that it is not very practical anyway.

One day a friend came with a flash disk and I had to return to the standard distro kernel, because I didn't ever guess that a peripheral like this would ever be in my "practical" all-in kernel environment.

And quess what? This didn't make a good promotion of Linux to my Windows friend.

You never know what is coming next. Suppose you see an astounding usb joystick with 12 buttons and 4 axis and you want it into your system.

Would you have the hid device previously compiled in just in case? No, you would have to rebuild the kernel.

Well, I didn't even have to insert the module. It is done automatically.

When you make all peripherals as modules, you have to make this step once and it will work everywhere. Having a LAN in my home, I find this very practical to make a kernel+modules once and distribute it in my hosts.

When you change hardware, the kernel will definetly work, and seeing the modules loaded will better inform about the new system, without having to read every manual page of your new motherboard.

If you choose the non-module path, you will have to remake a kernel before the actual hardware change, if you want it to be fully functional of course.

But haven't you found till now that the manuals do not always have the details you actually need?

The funny thing is that even choosing to make a kernel with all necessary drivers built in, first making them as modules and seeing what's loaded helps a lot.

[Pafnoutios]

Originally posted by micro
...
One day a friend came with a flash disk and I had to return to the standard distro kernel, because I didn't ever guess that a peripheral like this would ever be in my "practical" all-in kernel environment.

And quess what? This didn't make a good promotion of Linux to my Windows friend.

You never know what is coming next. Suppose you see an astounding usb joystick with 12 buttons and 4 axis and you want it into your system.

Would you have the hid device previously compiled in just in case? No, you would have to rebuild the kernel.

Well, I didn't even have to insert the module. It is done automatically.

...

The funny thing is that even choosing to make a kernel with all necessary drivers built in, first making them as modules and seeing what's loaded helps a lot.

Yes, modules would be good in those cases. I would classify those as one of the "If you thing you have a reason for it" times. The same goes for driver development.

I haven't upgraded my computer in over a year. Back then, I used the lsmod output from my redhat kernel to know what was being used. When I started compiling my own kernels and then switched to gentoo I've compiled everything into my kernel that I use. I don't even compile parallel port support at all, since I don't use it. I would be vulnerable to the 'friend comes over with a device' situation if someone had a parallel port device. I think the same of USB flash drives, and now that you mention that, I think I see that as a reason to compile that as a module the next time I build a kernel.

My HID drivers are compiled right in because I have two USB joysticks/gamepads I use for old nintendo games. It seems to autodetect my joysticks, I don't remember selecting them by name in the kernel configuration. Would any USB joystick work if I plug it in?

[Ludootje]

I thought using modules also had the advantage that it makes the system more stable.
I could be totally wrong about this, but here's my understanding of it. Feel free to correct me if needed :)

Imagine there's a bug in your sound card driver.
If it's loaded as a module, you can just unload the module when the problem appears, and your system will continue to work fine (except you won't have sound anymore, of course:)).
However, if it's compiled directly into the kernel, the kernel's ****ed up and you need to use sys-rq stuff etc in the hope to fix it. The only thing left to do might be a hard reboot.
This is the main reason why I compile trivial things like sound, usb, networking, secondary filesystems,... as modules.

[micro]

I think that any joystick will make the kernel call the hid module, if not compiled as a built-in.

What annoys me is that not all modules are detected correctly or loaded. Of course the ones that fail to are much less than the ones that succeed.

I think that to completely disable the parport and parport-pc code is something I wouldn't do, but since you definetly don't use it, why not?

But this indicates that you plan your new printer to be a usb one, so put also the other usb device classes into your kernel, not only the hid.

Here are some modules that might help you, apart ftom usbcore, soundcore, input and the alsa or oss modules, or peripheral modules that are auxiliary to others.

apm (slackware distros do not load this automatically)
sidewinder (sidewinder 3d pro in standard gameports)
pcspkr (pc speaker, rarely a module)
8139too (rtl 8139 based network cards)
via-rhine (via rhine II based network cards - needs auxiliary modules)
floppy (if module)
parport (if module)
parport_pc (if module)
psmouse (if module)
bttv (bt 848 tv tuner chipset)
serial (com ports if module)
ehci-hcd (if module) usb 2.0
ohci-hcd (if module) usb 1.1
uhci-hcd (if module) usb generic
usblp (usb printer)
usb-storage (usb mass storage)
hid (Usb human interface device - mouse, keyboard,tablet etc.)
scanner (usb scanner)

The mass storage is needed for digital cameras that work with this generic protocol and for usb flash disks too.

I do not recall the firewire module, but I didn't need it yet.

[Pafnoutios]

I have usb-storage compiled in for my usb CF card reader. Is that the same code for the usb flash drives (keychain things)?
I have a lot of the usb code compiled in because I have half a dozen usb peripherals: keyboard (with built-in 2-port hub), mouse, 2 joysticks, CF card reader, Zaurus dock (which I don't use for communications but is plugged in and the usb ethernet and NATting code are compiled in), and a color printer.
My parallel printer (HP Laserjet 4) is connected to my server and hosted to my two workstations.
The firewire kernel is ieee1394 or i1394. Firewire is an IEEE (International Electrical and Electronic Engineers?) communication specification numbered 1394. I have that compiled in for my video camera. I make video CDs of my son for my inlaws which live far away. This one could be done as a module since I don't use it all the time, and in 2.4 it is experimental.

[micro]

>> I have usb-storage compiled in for my usb
>> CF card reader. Is that the same code
>> for the usb flash drives (keychain things)?

I think yes.At least as far as I can see through dmesg.

Here's a tip: have a terminal (tty, xterminal whatever) and issue the command:
watch 'dmesg | tail'

Then insert the flash drive or whatever else and the change in the kernel messages will be shown.

watch updates every 2 secs.

[bwkaz]

Originally posted by micro
You never know what is coming next. Suppose you see an astounding usb joystick with 12 buttons and 4 axis and you want it into your system.

Would you have the hid device previously compiled in just in case? No, you would have to rebuild the kernel. Yes, I would. Any USB input device is going to use HID, including my USB mouse.

and seeing the modules loaded will better inform about the new system, without having to read every manual page of your new motherboard. Huh? You don't have to read the motherboard manual to figure out what's on it -- all you have to do is look for a few seconds before you buy it to see what kinds of devices it has, and use lspci to find out what exactly they are...

[GaryJones32]

Originally posted by bwkaz

I didn't say slower, I said bigger. ;) There is more (non-pageable, because it's kernel space!) memory in use when you've got the serial port module loaded than when you have the serial driver compiled in. It may not be noticeable, but this is memory that can never be swapped out, and will never be used again! It's a complete waste.


as i understand it the only thing that exists in the kernel memory space that is not paged as virtual is the memMap[] and the paging tables themselves..
everything else (RAM) that's left over is available to every single process as virtual kernel space.. this is handled by the processors MMU. So while modules are a little larger with some info about processes being used and init() and cleanup()
this stuff is not permanently using hardware RAM !

another issue worth considering is how really insecure it is to have loadable modules enabled in your kernel cause it's an open door to modifying sys_call_table[] and whatever else.