[OpenWrt Wiki] Filesystems (2024)

This article is about file systems used by OpenWrt for device built-in flash.

For installing additional file systems, including partitioning and mounting, see this page for general storage as well as this page to other common filesystems.

Please read about the flash.layout as well. Also, note that there are two types of flash memory: NOR flash and NAND flash. See also mtd.

Used to merge two filesystems, one read-only and the other writable. flash.layout explains how this is used in OpenWrt.

• OverlayFS

• Overlayfs documentation

• https://dev.openwrt.org/browser/trunk/target/linux/generic/patches-2.6.38/209-overlayfs.patch?rev=26213

• Debating overlayfs on LWN.net

• Was mainlined in Linux kernel 3.18

• Overlayfs's support for inotify mechanisms is not complete yet. Events like IN_CLOSE_WRITE cannot be notified to listening process.

tmpfs is implemented on many Unix-like operating systems (including OpenWrt). It operates similar to a RAM-Disk, without writing files to disk. In OpenWrt, /tmp resides on a tmpfs-partition and /var is a symlink to it; /dev resides on a little tmpfs partition of its own.

• Kernel documentation on tmpfs

• (+) doesn't directly use space on non-volatile storage

• (-) no wear leveling

• (-) volatile (doesn't survive a reboot)

SquashFS is a read only compressed filesystem. While gzip is available, at OpenWrt it uses LZMA for the compression. Since SquashFS is a read only filesystem, it doesn't need to align the data, allowing it to pack the files tighter thus taking up significantly less space than JFFS2 (20-30% savings over a JFFS2 filesystem)!

• (+) taking up as little space as possible

• (+) allowing the implementation of an idiot proof FailSafe for recovery, since it is not possible to write to it

  See Also

  [OpenWrt Wiki] Warning about 4/32 devicesDD-WRT vs. OpenWrt: Which open-source router firmware should you pick?Understanding the value of OpenWRT | IOPSYSIs it worth building a router for stability & reliability?

• (-) read only

• (-) waste space, since each time a file contained on it is modified, actually a copy of it is being copied to the second (JFFS2) partition

• Kernel documentation on SquashFS

• SquashFs Performance Comparisons

There is a generic problem when running SquashFS on NAND: The issue is that SquashFS has no bad block management at all and requires all blocks on order; but for proper NAND bad block management you also need to be able to skip bad blocks and occasionally relocate blocks (see squashfs and NAND flash). That's why raw SquashFS is a bad idea on NAND (it works if you use a FTL like UBIFS).

JFFS2 is a writable compressed filesystem with journaling and wear leveling using LZMA for the compression.

• (+) is writable, has journaling and wear leveling

• (+) is cool

• (-) is compressed, so a program (opkg in particular) cannot know in advance how much space a package will occupy

• (+) is compressed, so a program (which is preinstalled) takes much less space, so effectively you have more space

For NAND-flash targets, it was replaced with UBIFS.

• UBIFS is a file system for raw flash. It is used in OpenWrt NAND targets since :: around r40364

• Kernel documentation on UBIFS

• UBIFS File Encryption how does UBIFS understand what a “file” is? Isn't a file

• UBIFS File Encryption v1 on LWN.net

• UBIFS File Encryption v2 on LWN.net

• UBIFS Supports OverlayFS In Linux 4.9, Readying UBI For MLC Support

• ext2

• Ext2/3/4 is used on x86, x86-64 and for some arch with SD-card rootfs

• Kernel documentation on ext2

• (+) a program (opkg in particularly) knows how much space is left!

• (+) good ol' veteran FOSS file system

• (-) no journaling (ext3, ext4 support journaling)

• (-) no wear leveling

• (-) no transparent compression

• was used by older OpenWrt version and thus there are still references to this in the Wiki

• replaced by OverlayFS now.

• The mini_fo filesystem on LWN.net

• mini_fo: The mini fanout overlay file system official site

System bootup is as follows: →process.boot

1. kernel boots from a known raw partition (without a FS), scans mtd partition rootfs for a valid superblock and mounts the SquashFS partition (containing /etc) then runs /etc/preinit. (More info at technical.details)

2. /etc/preinit runs /sbin/mount_root

3. mount_root mounts the JFFS2 partition (/overlay) and combines it with the SquashFS partition (/rom) to create a new virtual root filesystem (/)

4. bootup continues with /sbin/init

/overlay was previously named /jffs2

The kernel boot process involves discovering of partitions within the NOR flash and it can be done by various target-dependent means:

• some bootloaders store a partition table at a known location

• some pass the partition layout via kernel command line

• some targets require specifying the kernel command line at the compile time (thus overriding the one provided by the bootloader).

Either way, if there is a partition named rootfs and MTD_ROOTFS_ROOT_DEV kernel config option is set to yes, this partition is automatically used for the root filesystem.

After that, if MTD_ROOTFS_SPLIT is enabled, the kernel adjusts the rootfs partition size to the minimum required by the particular SquashFS image and automatically adds rootfs_data to the list of the available mtd partitions setting its beginning to the first appropriate address after the SquashFS end and size to the remainder of the original rootfs partition. The resulting list is stored in RAM only, so no partition table of any kind gets actually modified.

For more details please refer to the actual patch at:https://dev.openwrt.org/browser/trunk/target/linux/generic/patches-2.6.37/065-rootfs_split.patch

For overlaying a special mini_fo filesystem is used, the README is available from the sources athttps://dev.openwrt.org/browser/trunk/target/linux/generic/patches-2.6.37/209-mini_fo.patch

Note:: It is possible to contain the entire root filesystem on a JFFS2-Partition only, instead of a combination of both.The advantage is that changes to included files no longer leaves behind an old copy on the read only filesystem. So you could end up saving space.The disadvantage of this would be, that you have no failsafe any longer and also, JFFS2 takes significantly more space then SquashFS.

Yes, it's technically possible, but a bit of a mess to actually pull off. The firmware has to be loaded as a trx file, which means that you have to put the JFFS2 data inside of the trx. But, as I said above, the trx has a checksum, meaning that if you ever change that data, you invalidate the checksum. The solution is that you install with the JFFS2 data contained within the trx, and then change the trx-boundaries at runtime. The end result is a single JFFS2 partition for the root filesystem. Why someone would want to do it is beyond me; it takes more space, and while it would allow you to upgrade the contents of the filesystem you would still be unable to replace the kernel (outside of the filesystem), meaning that a seamless upgrade between releases is still not possible! Having SquashFS gives you a failsafe mechanism where you can always ignore the JFFS2 partition and boot directly off SquashFS, or restore files to their original SquashFS versions.

I used to have a trick where I could convert a SquashFS install to a JFFS2 install at runtime by copying all the data onto the SquashFS partition and changing the partition boundaries. I never really had much use for the util -- not to mention it required a rather large flash to store both SquashFS and JFFS2 copies of the root during transition -- so support for it was dropped.

Example pictures: on formatted partition / how data is stored (and addressed on ext3)

• how data is stored and addressed by ext2:

• how data is stored and addressed by ext3:

• how data is stored and addressed by SquashFS:

• how data is stored and addressed by JFFS2: