Compressed ROM filesystem

Cromfs is a compressed read-only filesystem for Linux. Cromfs is intended for permanently archiving gigabytes of big files that have lots of redundancy.

In terms of compression it is much similar to 7-zip files, except that practical realtime access (albeit much slower than on most other filesystems) can be provided for the whole archive contents; the user does not need to launch a program to decompress a single file, nor does he need to wait while the system decompresses 500 files from a 1000-file archive to get him the 1 file he wanted to open.

The creation of cromfs was inspired from Squashfs and Cramfs.

• Data, inodes, directories and block lists are stored compressed
• Duplicate inodes, files and even duplicate file portions are detected and stored only once
  • Especially suitable for gigabyte-class archives of thousands of nearly-identical megabyte-class files.
• Files are stored in solid blocks, meaning that parts of different files are compressed together for effective compression
• Most of inode types recognized by Linux are supported (see comparisons).
• The LZMA compression is used. In the general case, LZMA compresses better than gzip and bzip2.
• As with usual filesystems, the files on a cromfs volume can be accessed in arbitrary order; the waits to open a specific file are small, despite the files being semisolidly archived.

See the documentation of the cromfs format for technical details (also included in the source package as doc/FORMAT).

• Filesystem is write-once, read-only. It is not possible to append to a previously-created filesystem, nor it is to mount it read-write.
• Max filesize: 2^64 bytes (16777216 TB), but 256 TB with default settings.
• Max number of files in a directory: 2^30 (smaller if filenames are longer, but still more than 100000 in almost all cases)
• Max number of inodes (all files, dirs etc combined): 2^60, but depends on file sizes
• Max filesystem size: 2^64 bytes (16777216 TB)
• There are no "." and ".." entries in directories.
• mkcromfs is slow. You must be patient.
• The cromfs-driver has a large memory footprint. It is not suitable for very size-constrained systems.
• Ownerships are not saved.
• Maximum filename length: 4095 bytes

Development status: Pre-beta. The Cromfs project has been created very recently and it hasn't been yet tested extensively. There is no warranty against data loss or anything else, so use at your own risk.

This is all very biased, hypothetical, and by no means a scientific study, but here goes:

Feature	Cromfs	Cramfs	Squashfs (3.0)
Compression unit	adjustable (1 MB default)	4 kB	adjustable (64 kB max)
Files are compressed	together (up to block limit)	individually	individually
Maximum file size	16 EB	16 MB	16 EB
Duplicate whole file detection	Yes	No (but hardlinks are detected)	Yes
Near-identical file detection	Yes (identical blocks)	No	No
Compression method	LZMA	gzip	gzip
Ownerships	None (uses the mounter's uid and gid)	uid,gid (but gid truncated to 8 bits)	uid,gid
Timestamps	mtime only	None	mtime only
Endianess-safety	Works on little-endian only	Safe, but not exchangeable	Safe
Kernelspace/userspace	User (fuse)	Kernel	Kernel
Appending to a previously created filesystem	No	No	Yes
Supported inode types	reg,dir,chrdev,blkdev,fifo,link,sock	reg,dir,chrdev,blkdev,fifo,link,sock	reg,dir,chrdev,blkdev,fifo,link,sock

1. Install the development requirements: make, gcc-c++, fuse and openssl-dev
   • Remember that for fuse to work, the kernel must also contain the fuse support. Do "modprobe fuse", and check if you have "/dev/fuse" and check if it works.
     • If an attempt to read from "/dev/fuse" (as root) gives "no such device", it does not work. If it gives "operation not permitted", it might work.
2. Build "cromfs-driver" and "util/mkcromfs", i.e. command "make":

   $ make

3. Create a sample filesystem:

   $ util/mkcromfs . sample.cromfs

4. Mount the sample filesystem:

   $ mkdir sample
   $ ./cromfs-driver sample.cromfs sample &

5. Observe the sample filesystem:

   $ cd sample
   $ ls -al

6. Unmounting the filesystem:

   $ cd ..
   $ fusermount -u sample

   or, type "fg" and press ctrl-c to terminate the driver.

To improve the compression, try these tips:

• Adjust the block size (--bsize) in mkcromfs. If your files have a lot identical content, aligned at a certain boundary, use that boundary as the block size value.
  Note: The value does not need to be a power of two.
• Adjust the fblock size (--fsize) in mkcromfs. Larger values cause almost always better compression. However, the filesystem must decompress each fblock as a single unit into the RAM. Larger fblock sizes increase the RAM usage of the filesystem.
  Conversely, you can decrease the fblock size in order to save RAM.
• Sort your files. Sometimes the order in which the files are compressed has effect on the compressibility.

cromfs has been written by Joel Yliluoma, a.k.a. Bisqwit,
and is distributed under the terms of the General Public License (GPL).

Patches and other related material can be submitted by e-mail at:0i3/f@JoelypiOe3e Yliqy@vhhVeluom0Qnjuh.a <bieguesqwikt7/.@kt@ikeWjVnVcpfi.fi>

• GNU make and gcc-c++ are required to recompile the source code.
• The openssl development library is required for MD5 calculation.
• The filesystem works under the Fuse user-space filesystem framework. You need to install both the Fuse kernel module and the userspace programs before mounting Cromfs volumes.

The official home page of cromfs is at http://iki.fi/bisqwit/source/cromfs.html.
Check there for new versions.