Modern Filesystems Reviewed

[trilarian]

This thread is mostly sparked by cybertron giving me this response to a previous post:

Also, I know you didn't want links, but I'm going to post one anyway. http://www.debian-administration.org/articles/388

I already have all my systems setup and am not planning on making any changes, so this is strictly for personal information acquisition. I don't want this to turn into a flame war of MY file system is better than YOURS. However, every article I pick up seems to be slighted towards one FS being better than the other. Yes, testing conditions and expected results can skew the final reporting, so I was hoping to hit a mature crowd here for personal experience.

Just to take an example, the article cybertron gave me seems to suggest that XFS is the winner in almost all test. Then I do some research and see issues with XFS like this (http://oss.sgi.com/projects/xfs/faq.html#nulls) where file corruption with power hiccups can be a major issue by having NULL bits in your files from the way XFS writes data! Furthermore, I found a graph on SourceForge that suggest XFS is outstanding in Single-User mode, but suffers greatly under multiple threads running at the same time - check this (http://ext2.sourceforge.net/2005-ols/ols-presentation-html/img38.html).

My point here is not to bash XFS at all, in fact I use it on my MythTV box. However, I'm having trouble finding a reliable source for information on true comparison of file systems. I could care less how long it takes for a file system to be created (I'm doing it once!!!!). I'm not concerned if it takes an extra half second to mount - the box I'm typing this thread on has uptime of 168D 2H 3M(and happens to be Ext3) haha...

I would just like to find an unbiased review of file systems in terms on I/O speed, both for single dump of small and large files and for multiple threads dumps (server simulation) without any bias. Also, integrity is important! If a file system can write 30x faster than any other but the moment it loses power it hoses your entire disk - NO THANKS.

Anyway, any thoughts, comments, links, etc. are welcome. I really want this to be kept an open documentation of modern experiences, not what <insert file system here> did to you in 1999. And one last thought, cross-platform compatibility is not such an issue for me. I'm considering this to be internal drives, so if need be I can run SAMBA or NFS to share the partition to other computers. I'm simply looking for performance/stability/disaster recovery/etc.

Thanks in advance!

[JohnT]

You've got hardware parameters that can figure in to the mix.
Here are some links with some links and IRC channels to some knowledgable people and groups on file systems.
HERE (http://209.85.135.104/search?q=cache:NyCzqhBwOpcJ:infohost.nmt.edu/~val/review/repair.pdf+review+of+modern+file+systems&hl=en&ct=clnk&cd=1)
HERE (http://linuxfs.pbwiki.com/)
HERE (http://linuxfs.pbwiki.com/ReadingList)

[cybertron]

It looks like you're discovering the same thing I did when I found that link. It's very hard to get a straight answer on what is better. The link I posted seemed to be the most real-world benchmarks that I could find, although there was another site that had some more theoretical ones that showed slightly different results. Plus there's the whole issue of stability, but it seems that everyone has a brother who know someone that had a horrible crash on a given filesystem, so it's nearly impossible to find out which ones are actually the most stable.

In case you're interested, here (http://www.nemebean.com/tiddly.html#%5B%5BFilesystems%20Comparison%5D%5D) are my notes from when I was looking into this (note that I haven't actually implemented any of the suggestions I made to myself - I'm still running reiser for basically everything and I haven't split my partitions as indicated on that page, so take everything with a grain of salt). Also note that my "painfully slow mounting" comment about reiser seems to have been fixed somewhere along the line. Might have been a problem with my kernel or something.

Disclaimer: What I wrote are basically my opinions based on what I read. It was over a year ago, and I'm certainly no expert on filesystems so it could be completely wrong.

[trilarian]

I started off with a mix of ext2(/boot) and ReiserFS back in the day. My main problem with ReiserFS was the binary tree structure. This was before the days I could afford a UPS and even a slight hiccup in power(which was frequent) would cause part of my ogg file to overlap part of my email file, etc., etc. It was always the fastest FS(in terms of desktop use - lots of small files), but even though I won't go so far to say stability is so much more important than speed that I'll take a 100% stable FS that takes 10 minutes to write a 1 Meg file, I would lean toward stability over speed. RAID has helped make this a good tradeoff. Set up a journal, that yes sacrifices speed, but run multiple drives at once to gain a speed boost to balance.

At some point, long ago, so this may have changed by now - I ran a series of disaster test on old hardware. Basically setup, have a machine write a large file then pull the plug out the back. Have it copy many files 1k or under in size and pull the plug(in retrospect virtualization may of been better, but meh.. hardware was junk anyway). At that point, ReiserFS kept hosing my FS, while ext3 seemed to recover from most of the crashes. So I went ext3 as my FS of choice.

Then came along MythTV and the emergence of both files that can be many gigs in size and drives that can be in TBs - something, at least at the time, ext3 wasn't prepared for. Mostly, trying to just delete a file that I accidentally setup the tuner to record 5+ hours of video meant I couldn't use that PC for awhile. XFS seemed like the savior. My problem with XFS though, is the way it writes to the FS itself that leaves a BIG, MAJOR problem if certain conditions are met. It is possible for XFS to zero fill your files by waiting to write then losing power before initiating the write. This has always giving me chills when setting up anything critical. Yes, in single to few member mode, it not only deals with large files well but is very fast on large partition sizes. BUT WHO WANTS A 0 FILLED MOVIE/MUSIC/etc file? It just seems at some point a best of all worlds FS would of been created.

Of all FS I have played with, none seem to have a clear advantage when the FS is getting near to full capacity. It seems that all the anti-fragmentation algorithms require some disk space(maybe why ext3 reservation for root is actually an overlooked benefit?). Anyway, here are my personal feelings(not facts of life) - even though at this point I still hope for either perfecting of and existing FS or an emergence of a new one that is a good balance of everything.

Ext2 - Very fast as no journal, but very unsafe in terms of disaster recovery. So it is ideal for partitions that are temp in nature or scratch area where files being loss is not of concern.

Ext3 - Seems to have the best disaster recovery of all FS that I've personally seen. However, this comes at a cost of some speed and not having 100% of the disc available to write too(reserved blocks).

ReiserFS - Seems to be the fastest there is for small file usage/writing/swapping/etc. It is a stable FS, but I personally feel its like playing with fire. If your binary tree gets hosed, so does your entire partition unless you are good and have the patience to use grep and dd.

XFS - Optimized for large file support and large partition size. If you are going to have partitions in TBs and files in GBs, FS of choice. HOWEVER, be aware that this FS more than any other demands you have a functioning UPS.

JIS - Similar to XFS, but I find it to be a bit slower. The advantage is very, very low CPU usage. So this would be ideal if you have an ancient PC with large drives. This would keep your overhead down, but if you have the horsepower consider one of the above OS's.

[MkIII_Supra]

Been using it since 2001. Haven't had many issues with it. Seems to be reliable and fast. But I also have a UPS that will cleanly shut my system down in the event of a power failure and if there is a glitch, as in we lose power and the e-generator at work starts, I lose nothing.

I have had the root partition get whacked in a power failure but never my home partition. So I guess I have been luck in that aspect.

ReiserFS seems to recover pretty fast as well. And I get hideously long up-times without any corruption or headaches.