Logging to an SD card without corrupting it on power loss

A data logger runs fine on the bench for weeks. Then a customer unplugs it, a battery sags, or a breaker trips, and the card comes back with a truncated file, a missing directory, or no filesystem at all. The firmware wasn’t obviously wrong. The problem is that three separate layers can each lose data, and fixing only one leaves the others.

The three layers that can fail

Your buffers. Data in a RAM buffer, or in the filesystem library’s cache, has not been written yet. If power drops, it is gone. This is the layer firmware controls completely.

The filesystem structures. FAT keeps a file’s length and cluster chain in directory entries and allocation tables, updated separately from the data. Cut power between those updates and the file can show the wrong size or the volume can need repair. littlefs is built differently, which we get to below.

The card itself. An SD card is not a bare memory chip. It has a controller that remaps blocks and does its own wear leveling. A write that is interrupted can leave that controller’s internal tables inconsistent, and no filesystem above it can see or repair that. One industrial card vendor describes this failure as “write abort” and says translation tables, firmware and metadata can all be damaged, not only the data being written. That is a vendor’s account of its own product category, but it matches what field returns look like: a card that is unreadable, not merely missing the last record.

Layer one: decide how much you can afford to lose

Every logger has a loss window, whether or not anyone chose it. The useful question is how many seconds of data the product can lose on a hard power cut, and whether that number is written down.

With FatFs, the answer is controlled by f_sync. The FatFs documentation describes it as working like f_close while leaving the file open, and says it suits long-running writes such as data loggers. Called at intervals, it limits what a sudden blackout, wrong media removal or disk error can take: only data written after the last sync is at risk. Without it, data written since the file was opened is exposed.

Sync too rarely and you lose minutes. Sync after every record and you pay in throughput, because each sync forces directory and table updates to the card. The practical pattern:

1. Buffer records in RAM and write them in card-friendly chunks.
2. Sync on a timer or record count chosen from the loss window you can accept.
3. Close and reopen, or start a new file, on a longer schedule so no single file stays open for days.
4. Keep the file open as little as the application allows.

ESP-IDF exposes this as a build option. Its FATFS documentation describes CONFIG_FATFS_IMMEDIATE_FSYNC, which calls f_sync() automatically after writes. It improves consistency and size reporting, and it costs performance, so Espressif recommends leaving it off unless your consistency needs require it. The same page notes that removing a card without closing files and unmounting can corrupt the filesystem, and that the SD card status check option can help when cards are pulled without unmounting.

Layer two: FAT or littlefs

This is the question we hear most. The short version: they solve different problems, and one is not simply the safer version of the other.

	FAT (FatFs)	littlefs
Designed for	Interchange with PCs	Raw flash on small systems
Power-loss behavior	Depends on when you sync; may need repair	Designed to fall back to last good state
Readable on a laptop	Yes, by plugging in the card	No, needs a tool or driver
Typical home	Removable SD card	Onboard SPI NOR flash or internal flash

The littlefs README says it is designed to handle random power failures and to fall back to the last known good state. It also provides wear leveling and bad-block detection, and says it is designed with flash in mind. The same README says that because of FAT’s limitations, FAT can’t provide power-loss resilience. Note also what it requires: file updates are not committed until sync or close is called, and if the storage layer caches writes, your sync callback must flush them or integrity can’t be guaranteed.

Here is where people get caught. littlefs’s guarantees assume it talks to the flash through read, program and erase callbacks. Put it on an SD card and the card’s own controller sits underneath those callbacks. The README does not target SD cards. Our reading is that the filesystem’s guarantee does not extend through a card controller you can’t inspect, though we haven’t seen a primary source that tests this either way, so treat it as a reason to test rather than a rule.

So the choice follows from the product:

• Customer pulls the card and reads it on a PC. Use FAT. Sync deliberately, unmount before removal where you can, and accept that a hard cut may need a repair pass.
• Data stays inside the product and is uploaded or read over a link. Put it on onboard flash with littlefs, sized for the write volume. There is no card to fail or be removed. If you are choosing a flash part, second-sourcing the SPI flash is worth reading first, since the log partition lives on that chip.
• You need both. Log to littlefs on onboard flash, then copy completed files to the card as a separate, retryable step. A half-copied file on the card is then a nuisance, not a lost record.

Layer three: give the card time to finish

Software can only do so much if power vanishes in the middle of a card write. The hardware answer is hold-up: enough stored energy after the supply drops that the card finishes the block it is writing, with firmware told early enough to stop starting new ones.

That is a design item, not a firmware patch:

• Detect the loss early. Watch the input rail upstream of the regulator with a comparator or an ADC pin, not the regulated output, which stays up and hides the event until the last moment.
• Stop new writes on that signal. The interrupt should set a flag that closes the file and stops the logger, not start new work.
• Size the reservoir for the card, not the MCU. Card write current is a peak, often higher than the processor draws. Check the card’s datasheet figure for write current and busy time, and use the worst card you will accept, not the one on your bench.
• Power the card from the held-up rail so it does not brown out while the filesystem is still finishing.
• Mind the voltage dip. A card that browns out partway through a write is the exact condition the vendor note above describes.

If you can’t add a reservoir, say so in the design and shrink the loss window instead. Don’t pretend the card is protected.

Test it the way it fails

A logger that survives a clean shutdown tells you little. Build a rig that cuts power at random points while the device is writing, and run it unattended for many cycles. After each cut, mount the card on a PC and run a filesystem check, then confirm the log still parses end to end and the last record is recent enough to meet your loss window.

Repeat it with the card brands you plan to allow, since the card is the part you don’t control. Record the make and model of the card you validated, and say in your documentation that others weren’t tested. This is the same discipline as a production test fixture: the rig is cheap, and finding the failure in the field is not.

How we approach it

We start from the loss window and who reads the data, then choose the filesystem, sync policy and hold-up circuit together, and prove the result with a power-cut rig. If you send the firmware’s write path, the power input schematic and the card or flash part you intend to use, we can tell you which layer is most likely to lose your data.