Installable File System

Created for Sming Framework Project August 2018 by mikee47

I struggled to find anything like this for embedded systems. Probably didn’t look hard enough, but it seemed like a fun thing to do so here we are.

The term ‘IFS’ came about because of the ‘I’ naming convention for virtual ‘Interface’ classes, hence IFileSystem. The term ‘installable’ is entirely appropriate because IFS allows file systems to be loaded and unloaded dynamically. Or maybe I just nicked the term from Microsoft :-)

Overview

IFS is written in C++ and has these core components:

FileSystem API

File systems are implemented using the IFS::IFileSystem virtual class. This is, in essence, a single large ‘function table’ you will see in regular filesystem implementations.

Class methods are similar to SPIFFS (which is POSIX-like).

Note

A single Stat structure is used both for reading directory entries and for regular fileStat() operations.

This differs from regular file APIs but is intended to simplify operation.

Applications will typically use IFS::FileSystem instead, which adds additional methods and overloads such as String parameter support. This used to implement the standard ‘flat’ Sming filesystem API, with a few minor changes and a number of additions.

Two wrapper clases (IFS::File and IFS::Directory) are provided for applications to manage access to files and folders.

Firmware FileSystem (FWFS)

Files, directories and metadata are all stored as objects in read-only image. FWFS images are compact, fast to access and use very little RAM (approx. 240 bytes for file descriptors, etc.)

To support read/write data a writeable filesystem can be mounted in a sub-directory.

A python tool fsbuild is used to build an FWFS image from user files. See Filesystem builder.

This is integrated into the build system using the fwfs-build target for the partition. Example Hardware configuration fragment:

"partitions": {
   "fwfs1": {
      "address": "0x280000",
      "size": "0x60000",
      "type": "data",
      "subtype": "fwfs",
      "filename": "out/fwfs1.bin",
      "build": {
          "target": "fwfs-build",   // To build a FWFS image
          "config": "fsimage.fwfs"  // Configuration for the image
      }
   }
}

Sming provides the Basic IFS sample application which gives a worked example of this.

Simple filesystem definitions can be defined in-situ, rather than using external file:

"build": {
      "target": "fwfs-build",   // To build a FWFS image
      "config": {
         "name": "Simple filesystem",
         "source": {
            "/": "files"
         }
      }
}

The following basic IFS implementations are provided in this library:

IFS::FWFS::FileSystem

Firmware Filesystem. It is designed to support all features of IFS, whereas other filesystems may only use a subset.

IFS::HYFS::FileSystem

Hybrid filesystem. Uses FWFS as the read-only root filesystem, with a writeable filesystem ‘layered’ on top.

When a file is opened for writing it is transparently copied to the SPIFFS partition so it can be updated. Wiping the SPIFFS partition reverts the filesystem to its original state.

Note that files marked as ‘read-only’ on the FWFS system are blocked from this behaviour.

IFS::Host::FileSystem

For Host architecture this allows access to the Linux/Windows host filesystem.

IFS::Gdb::FileSystem

When running under a debugger this allows access to the Host filesystem. (Currently only works for ESP8266.)

IFS (and FWFS) has the following features:

Attributes: Files have a standard set of attribute flags plus modification time and simple role-based access control list (ACL).
Directories: Fully supported, and can be enumerated with associated file information using a standard opendir/readdir/closedir function set.
User metadata: Supported for application use. The API for this is loosely based on Linux extended attributes (non-POSIX). Attributes are small chunks of data attached to files and directories, each identified by a numeric IFS::AttributeTag.
Filesystem API: The Sming FileSystem functions are now wrappers around a single IFileSystem instance, which is provided by the application.
Streaming classes: Sming provides IFS implementations for these so they can be constructed on any filesystem, not just the main (global) one.
Dynamic loading: File systems may be loaded/created and unloaded/destroyed at runtime
Multiple filesystems: Applications may use any supported filesystem, or write their own, or use any combination of existing filesystems to meet requirements. The API is the same.
Mount points: FWFS is designed for use as a read-only root filing system, and supports mounting other filesystems in special directories.

FWFS

Many applications require a default, often fixed set of files. The easiest way is just to use SPIFFS. The problem is that power outages can corrupt a filesystem. For an embedded device that’s bad news. SPIFFS is also a bit overkill if you’re just storing configuration data, or it’s just for read-only use.

So what do you do if your filesystem gets wiped? Resetting a system back to a functional, default state can be tricky if the core user interface web files are gone. You could reformat and pull a standard set of files off a server somewhere. If your storage requirements are minimal, you could link the file data into your firmware as constant data blocks.

That’s kind of what FWFS does, but in a more structured and user-friendly way.

FWFS offers a more convenient solution by providing all your default files in a compact, fast, read-only format. Images can be mounted in separate partitions, linked into the program image itself or stored as files within another filesystem.

Note

This behaviour is supported by partitions (see Storage Management) using custom Storage::Device objects.

Redirection

FWFS incorporates a redirector. This works by creating a mount point (a named object), which looks like an empty directory. When accessed, this get redirected to the root of another filesystem. The maximum number of mount points is fixed at compile time, but file systems can be mounted and dismounted at any time.

Mount points are identified explicitly in the build configuration file:

"mountpoints": {
   "path/to/use/spiffs": 0,
   "path/to/use/littlefs": 1
}

The filesystem builder creates the MountPoint objects and tags them with the given volume indices. For example, the directory “path/to/use/littlefs” is attached to volume index #0.

Note

Unlike other filesystems you cannot use a regular directory as a mountpoint. To change the name of a mountpoint requires the filesystem image to be re-built and re-flashed.

Applications use the IFileSystem::setVolume() method to install the actual filesystem.

Streaming backup/archive support

The IFS::FWFS::ArchiveStream class can be used to generate streaming filesystem backups from any supported filesystem. The archive files are in FWFS format.

Here are some examples of how it can be used:

Stream filesystem (or directory) images directly to remote servers
Make local filesystem backups
Compact log files which don’t change much (think of ZIP files - just needs a compression plugin)
Backup entire filesystem a local file, an empty partition, etc.
Defragment/compact or repair a damaged filesystem by re-formatting then restoring from backup

The archiver has some additional features:

Specify whether to archive an entire filesystem or start from a specific directory
Specify whether to follow links (e.g. other filesystems in mountpoints) or not
Exclude any file or directory via custom callback (or by overriding methods)
Perform custom file data encoding such as compression or encryption via callbacks
Add additional metadata to files (comments, encryption codes, etc.)

See the Basic IFS sample for

Access Control

This came about because I wanted to secure down my ESP8266 web server applications so that only the basic index.html, stylesheets and accompanying javascript would be publicly accessibly. Everything else would require user authentication.

I also wanted to prevent certain users from accessing restricted files. Other users would also be able to edit files. So a simple role-based access control mechanism seemed appropriate.

Access control typically encapsulates two areas:

Authentication: Is the user who they say they are? Usually performed by validating a username/password combination.
Authorisation: What is the user permitted to do?

I’ll step aside for a brief word on security. Authentication is the weakest link because it’s exposed to public scrutiny. To avoid compromise authentication must only be done over a secured link. That means SSL.

If you have the option it’s usually best to put all your smart devices behind a secure proxy. The raspberry Pi is great for stuff like this. The Pi deals with keeping the public connection secure, and translates it into a regular HTTP connection for the ESP8266.

If you don’t have this option, but you need to connect your ESP8266 to the internet, use the SSL build for Sming.

Having done this, we don’t need to worry about encrypting passwords as the SSL layer will do that. We just need to make sure they’re good passwords.

In my applications authentication is done by matching username/password against the user database, stored in a JSON file. If successful, the session gets a token which appears in every subsequent request. The user database indicates a User Role, one of public, guest, user, manager or admin. IFS keeps an ‘Access Control List’ (ACL) for each file containing two entries (ACE), one for read access and another for write access. The ACE specifies the minimum assigned IFS::UserRole required for access.

This is probably as much as the filesystem needs to do. I can’t see that file ownership, inherited permissions or more finely-grained access permissions would be required, but having said that extending this system would probably be fairly straightforward.

Configuration filesystem

If an application only requires write access for configuration files, SPIFFS is overkill. These files would be updated very infrequently, so wear-levelling would be un-necessary. The names and number of files would probably also be known at build time, and an individual file could be limited to a fixed size, for example one or two flash sectors. A ConfigFileSystem implementation would not need to support file creation or deletion.

Such a system would require almost no static RAM allocation and code size would be tiny.

However, the LittleFS has excellent metadata support and is ideal for storing configuration information. This can be done using :IFS::FileSystem::setUserAttribute and read using :IFS::FileSystem::getUserAttribute or :IFS::FileSystem::enumAttributes.

Note

The ESP-IDF has a mechanism for flash-based configuration space via the NVS component. It is robust and flexible but uses a significant amount of RAM for buffering which may preclude its use with the ESP8266.

FWFS Objects

All files, directories and associated information elements are stored as ‘objects’. Files and directories are ‘named’ objects, which may contain other objects either directly or as references. Small objects (255 bytes or less) are stored directly, larger ones get their own file. Maximum object size is 16Mbytes.

File content is stored in un-named data objects. A named object can have any number of these and will be treated as a single entity for read/write operations. File ‘fragments’ do not need to be contiguous, and are reassembled during read operations.

Named objects can be enumerated using IFS::IFileSystem::readdir(). Internally, FWFS uses handles to access any named object. Handles are allocated from a static pool to avoid excessive dynamic (heap) allocation. Users can attach their own data to any named object using custom object types.

The filesystem layout is displayed during initial mount if this library is built with DEBUG_VERBOSE_LEVEL = 3.

Why FWFS?

There are many existing candidates for a read-only system, so why do we need another one? Here are some reasons:

SPIFFS and LittleFS could be used in read-only mode but they are not designed for space-efficiency. Images are therefore larger than necessary, sometimes considerably larger. This is also true of other such filesystems designed for Linux, etc.

FWFS is designed to produce the smallest possible images to conserve limited flash storage. It therefore has a high effective capacity, i.e. you can put a lot more in there than with other filesystems.
With ROMFS, for example, information is laid out with headers first, followed by data. The root directory and volume information are at the front.

FWFS works in reverse by writing out file contents first, then file headers and then directory records. The root directory comes at the end, followed by the volume information record. This allows images to be created as a stream because directory records can be efficiently constructed in RAM as each file or subdirectory record is written out. This keeps memory usage low.

In addition, checksums and additional metadata can be created while file data is written out. This could be required for compressing or encrypting the contents, or for error tolerance. For example, if corruption is encountered whilst reading file contents this can be noted in the metadata which is written out afterwards.

Filesystem images are therefore generated in a single pass, with each file or directory only read once.
Standard attribute support not well-suited to embedded microsystems.

The small set of standard metadata defined by IFS is designed to solve specific problems with typical IOT applications.

Code dependencies

Written initially for Sming, the library should be fairly portable to other systems.

No definitions from SPIFFS or other modules should be used in the public interface; such dependencies should be managed internally.

Applications should avoid using filesystem-dependent calls, structures or error codes. Such code, if necessary, should be placed into a separate module.

Implementation details

The traditional way to implement installable filing systems is using function tables, such as you’ll see in Linux. One reason is because the Linux kernel is written in C, not C++. For Sming, a virtual class seems the obvious choice, however there are some pros and cons.

VMT

Advantages

Compiler ensures correct ordering of methods, parameter type checking
Simpler coding
Extending and overriding is natural

Function table

Advantages

Portable to C applications (although with some fudging so are VMTs).

Disadvantages

Care required to keep function order and parameters correct. Very likely we’d use a bunch of macros to deal with this.

Macros

We could #define the active filing system name which the FileSystem functions would map to the appropriate call. For example, fileOpen would get mapped to SPIFlashFileSystem_open(). We need to provide macros for defining file system functions.

Advantages

Fast

Disadvantages

Complicated

Prone to bugs

Not C++

Configuration variables

FWFS_DEBUG

default: 0

Set to 1 to enable more detailed debugging information.

ENABLE_FILE_SIZE64

default: disabled

Set to 1 to enable support for 64-bit files. This requires ENABLE_STORAGE_SIZE64 to be set.

API

namespace IFS

Return compression corresponding to given string

param str:

retval Compression::Type:

Compression::Type getCompressionType(const char *str, Compression::Type defaultValue = Compression::Type::None)

inline Compression::Type getCompressionType(const String &str, Compression::Type defaultValue = Compression::Type::None)

Return the access type value for the given string.

param str:
param defaultRole:: Returned if string isn’t recognsed

retval UserRole:

UserRole getUserRole(const char *str, UserRole defaultRole)

inline UserRole getUserRole(const String &str, UserRole defaultRole)

Typedefs

using AttributeEnumCallback = Delegate<bool(AttributeEnum &e)>: Return true to continue enumeration, false to stop.

using ErrorCode = int

using FileAttributes = BitSet<uint8_t, FileAttribute, size_t(FileAttribute::MAX)>: File attributes are stored as a bitmask.

using DirHandle = struct ImplFileDir*

using OpenFlags = BitSet<uint8_t, OpenFlag, size_t(OpenFlag::MAX)>

using FileHandle = int16_t

File handle.

References an open file

using FileID = uint32_t

File identifier.

Contained within Stat, uniquely identifies any file on the file system.

Enums

enum class AttributeTag : uint16_t

Identifies a specific attribute.

Values:

enumerator XX

enumerator User: First user attribute.

enum ControlCode

See IFS::IFileSystem::fcontrol

These are weakly typed as values may be defined elsewhere.

Values:

enumerator FCNTL_GET_MD5_HASH

Get stored MD5 hash for file.

FWFS calculates this when the filesystem image is built and can be used by applications to verify file integrity.

On success, returns size of the hash itself (16) If the file is zero-length then Error::NotFound will be returned as the hash is not stored for empty files.

enumerator FCNTL_SET_VOLUME_LABEL: Set volume label.

enumerator FCNTL_USER_BASE

Start of user-defined codes.

Codes before this are reserved for system use

enum class FileAttribute

Values:

enumerator XX

enumerator MAX

enum class OpenFlag

Values:

enumerator XX

enumerator MAX

enum class UserRole : uint8_t

Values:

enumerator XX

enumerator MAX: Actually maxmimum value + 1…

Functions

String getAclString(const IFS::ACL &acl)

Return a brief textual representation for an ACL Suitable for inclusion in a file listing.

Parameters:: acl –
Return values:: String –

inline AttributeTag getUserAttributeTag(uint8_t value)

size_t getAttributeSize(AttributeTag tag)

String getFileAttributeString(FileAttributes attr)

Get the string representation for the given set of file attributes suitable for inclusion in a file listing.

Parameters:: attr –
Return values:: String –

FileSystem *getDefaultFileSystem(): Framework should implement this method.

FileSystem *createFirmwareFilesystem(Storage::Partition partition)

Create a firmware filesystem.

Parameters:: partition –
Return values:: FileSystem* – constructed filesystem object

FileSystem *createHybridFilesystem(Storage::Partition fwfsPartition, IFileSystem *flashFileSystem)

Create a hybrid filesystem.

Parameters:

fwfsPartition – Base read-only filesystem partition
flashFileSystem – The filesystem to use for writing

Return values:

FileSystem* – constructed filesystem object

FileSystem *mountArchive(FileSystem &fs, const String &filename)

Mount an FWFS archive.

Parameters:

fs – Filesystem where file is located
filename – Name of archive file

Return values:

FileSystem* – constructed filesystem object

inline constexpr OpenFlags operator|(OpenFlag a, OpenFlag b)

time_t fsGetTimeUTC()

Get current timestamp in UTC.

Note

Filing systems must store timestamps in UTC Use this function; makes porting easier.

Return values:: time_t –

char getChar(UserRole role)

UserRole getUserRole(char code, UserRole defaultRole)

bool isRootPath(const char *&path)

Check if path is root directory.

Paths equal to “/” or “” are empty and considered equivalent to nullptr. Methods or functions can use this macro to resolve these for simpler parsing.

Parameters:: Path – to check, set to nullptr if it’s the root directory
Return values:: bool – true if path is root directory

void checkStat(Stat &stat)

FileSystem *createFatFilesystem(Storage::Partition partition)

Create a FAT filesystem.

Parameters:: partition –
Return values:: FileSystem* – constructed filesystem object

Variables

constexpr ErrorCode FS_OK = Error::Success

class DirectoryTemplate : public SectionTemplate 

#include <DirectoryTemplate.h>

Directory stream class.

Subclassed by IFS::HtmlDirectoryTemplate, IFS::JsonDirectoryTemplate

Public Functions

inline virtual bool nextRecord() override

Move to next record.

Return values:: bool – true to emit section, false to skip

class FileStream : public IFS::FsBase, public ReadWriteStream 

#include <FileStream.h>

File stream class.

Subclassed by FileStream, GdbFileStream, HostFileStream

Public Functions

void attach(FileHandle file, size_t size)

Attach this stream object to an open file handle.

Parameters:

file –
size –

bool open(const String &fileName, IFS::OpenFlags openFlags = OpenFlag::Read)

Open a file by path, and attach this stream object to it.

Note

call getLastError() to determine cause of failure

Parameters:

fileName – Full path to file
openFlags –

Return values:

bool – true on success, false on error

void close(): Close file.

inline virtual StreamType getStreamType() const override

Get the stream type.

Return values:: StreamType – The stream type.

virtual size_t write(const uint8_t *buffer, size_t size) override

Write chars to stream.

Note

Although this is defined in the Print class, ReadWriteStream uses this as the core output method so descendants are required to implement it

Parameters:

buffer – Pointer to buffer to write to the stream
size – Quantity of chars to write

Return values:

size_t – Quantity of chars written to stream

inline virtual int read() override

Read one character and moves the stream pointer.

Return values:: The – character that was read or -1 if none is available

virtual size_t readBytes(char *buffer, size_t length) override

Read chars from stream into buffer.

Terminates if length characters have been read or timeout (see setTimeout). Returns the number of characters placed in the buffer (0 means no valid data found).

Note

Inherited classes may provide more efficient implementations without timeout.

virtual uint16_t readMemoryBlock(char *data, int bufSize) override

Read a block of memory.

Todo:: Should IDataSourceStream::readMemoryBlock return same data type as its bufSize param?

Parameters:

data – Pointer to the data to be read
bufSize – Quantity of chars to read

Return values:

uint16_t – Quantity of chars read

virtual int seekFrom(int offset, SeekOrigin origin) override

Change position in stream.

Note

This method is implemented by streams which support random seeking, such as files and memory streams.

Parameters:

offset –
origin –

Return values:

New – position, < 0 on error

inline virtual bool isFinished() override

Check if all data has been read.

Return values:: bool – True on success.

String fileName() const

Filename of file stream is attached to.

Return values:: String – invalid if stream isn’t open

inline bool fileExist() const

Determine if file exists.

Return values:: bool – true if stream contains valid file

inline virtual String getName() const override

Returns name of the resource.

Note

Commonly used to obtain name of file

Return values:: String –

virtual MimeType getMimeType() const override

Get MIME type for stream content.

Return values:: MimeType –

inline virtual bool isValid() const override

Determine if the stream object contains valid data.

Note

Where inherited classes are initialised by constructor this method indicates whether that was successful or not (e.g. FileStream)

Return values:: bool – true if valid, false if invalid

inline size_t getPos() const

Get the offset of cursor from beginning of data.

Return values:: size_t – Cursor offset

inline size_t getSize() const

Get the total file size.

Return values:: size_t – File size

inline virtual int available() override

Return the maximum bytes available to read, from current position.

Return values:: int – -1 is returned when the size cannot be determined

virtual String id() const override

Returns unique id of the resource.

Return values:: String – the unique id of the stream.

bool truncate(size_t newSize)

Reduce the file size.

Parameters:: newSize –
Return values:: bool – true on success

inline bool truncate()

Truncate file at current position.

Return values:: bool – true on success

class HtmlDirectoryTemplate : public IFS::DirectoryTemplate : #include <HtmlDirectoryTemplate.h>

Read-only stream access to directory listing with HTML output.

class JsonDirectoryTemplate : public IFS::DirectoryTemplate : #include <JsonDirectoryTemplate.h>

Read-only stream providing directory listing in JSON format.

struct ACL: #include <Access.h>

struct AttributeEnum

#include <Attribute.h>

Attribute information passed to enumeration callback.

Public Members

AttributeTag tag = {}: The attribute tag.

size_t size = {0}: Size of returned data, may be less than attrsize if buffer too small.

size_t attrsize = {0}: Actual attribute size.

void *buffer: User-provided buffer with tag value.

size_t bufsize: User-provided buffer size.

struct Compression: #include <Compression.h>

A compression descriptor.

class Directory : public IFS::FsBase 

#include <Directory.h>

Wrapper class for enumerating a directory.

Public Functions

bool open(const String &dirName = nullptr)

Open a directory and attach this stream object to it.

Note

call getLastError() to determine cause of failure

Parameters:: dirName – Default is root directory
Return values:: bool – true on success, false on error

void close(): Close directory.

bool rewind()

Rewind directory stream to start so it can be re-enumerated.

Note

call getLastError() to determine cause of failure

Return values:: bool – true on success, false on error

inline const String &getDirName() const

Name of directory stream is attached to.

Return values:: String – invalid if stream isn’t open

inline bool dirExist() const

Determine if directory exists.

Return values:: bool – true if stream is attached to a directory

String getPath() const: Get path with leading separator /path/to/dir.

String getParent() const

Get parent directory.

Return values:: String – invalid if there is no parent directory

struct Extent

#include <Extent.h>

Defines the location of a contiguous run of file data.

e.g. offset: 0 length: 251 skip: 5 repeat: 30 Describes 30 blocks of data each of 251 bytes with a 5-byte gap between them. Thus run contains 7530 bytes of file data, consuming 7680 bytes of storage.

skip/repeat is necessary for SPIFFS to keep RAM usage sensible. Thus, a sample 267kByte application image has 1091 extents, but requires perhaps only 60 entries.

Public Members

storage_size_t offset: From start of partition.

uint32_t length: In bytes.

uint16_t skip: Skip bytes to next repeat.

uint16_t repeat: Number of repeats.

class File : public IFS::FsBase 

#include <File.h>

Wraps up all file access methods.

Common flag combinations

static constexpr OpenFlags ReadOnly = {OpenFlag::Read}: Read-only.

static constexpr OpenFlags WriteOnly = {OpenFlag::Write}: Write-only.

static constexpr OpenFlags ReadWrite = {OpenFlag::Read | OpenFlag::Write}: Read + Write.

static constexpr OpenFlags Create = {OpenFlag::Create}: Create file if it doesn’t exist.

static constexpr OpenFlags Append = {OpenFlag::Append}: Append to existing file.

static constexpr OpenFlags Truncate = {OpenFlag::Truncate}: Truncate existing file to zero length.

static constexpr OpenFlags CreateNewAlways{OpenFlag::Create | OpenFlag::Truncate}: Create new file or overwrite file if it exists.

Public Functions

inline bool stat(Stat &stat)

get file information

Parameters:: stat – structure to return information in, may be null
Return values:: bool – true on success

inline int control(ControlCode code, void *buffer, size_t bufSize)

Low-level and non-standard file control operations.

To simplify usage the same buffer is used for both input and output. Only the size of the buffer is provided. If a specific FCNTL code requires more information then it will be contained within the provided data.

Parameters:

code – FCNTL_XXX code
buffer – Input/Output buffer
bufSize – Size of buffer

Return values:

int – error code or, on success, data size

template<typename T> inline bool open(const T &path, OpenFlags flags = OpenFlag::Read)

open a file by name/path

Parameters:

path – full path to file
flags – opens for opening file

Return values:

bool – true on success

inline bool close()

close an open file

Return values:: bool – true on success

inline int read(void *data, size_t size)

read content from a file and advance cursor

Parameters:

data – buffer to write into
size – size of file buffer, maximum number of bytes to read

Return values:

int – number of bytes read or error code

inline int write(const void *data, size_t size)

write content to a file at current position and advance cursor

Parameters:

data – buffer to read from
size – number of bytes to write

Return values:

int – number of bytes written or error code

inline file_offset_t seek(file_offset_t offset, SeekOrigin origin)

change file read/write position

Parameters:

offset – position relative to origin
origin – where to seek from (start/end or current position)

Return values:

int – current position or error code

inline bool eof()

determine if current file position is at end of file

Return values:: bool – true if at EOF or file is invalid

inline file_offset_t tell()

get current file position

Return values:: int32_t – current position relative to start of file, or error code

inline bool truncate(file_size_t new_size)

Truncate (reduce) the size of an open file.

Parameters:: newSize –
Return values:: bool – true on success

inline bool truncate()

Truncate an open file at the current cursor position.

Return values:: bool – true on success

inline bool flush()

flush any buffered data to physical media

Return values:: bool – true on success

inline bool setacl(const ACL &acl)

Set access control information for file.

Parameters:: acl –
Return values:: bool – true on success

inline bool settime(time_t mtime)

Set modification time for file.

Note

any subsequent writes to file will reset this to current time

Return values:: bool – true on success

inline bool setcompression(const Compression &compression)

Set file compression information.

Parameters:: compression –
Return values:: bool – true on success

inline bool remove()

remove (delete) an open file (and close it)

Return values:: bool – true on success

inline file_size_t getSize()

Get size of file.

Return values:: uint32_t – Size of file in bytes, 0 on error

inline file_offset_t readContent(size_t size, ReadContentCallback callback)

Read from current file position and invoke callback for each block read.

Parameters:

size – Maximum number of bytes to read
callback –

Return values:

int – Number of bytes processed, or error code

inline file_offset_t readContent(ReadContentCallback callback)

Read from current file position to end of file and invoke callback for each block read.

Parameters:: callback –
Return values:: file_offset_t – Number of bytes processed, or error code

inline String getContent()

Read content of the file, from current position.

Note

After calling this function the content of the file is placed in to a string. The result will be an invalid String (equates to false) if the file could not be read. If the file exists, but is empty, the result will be an empty string “”.

Return values:: String – String variable in to which to read the file content

inline FileHandle release(): Return current file handle and release ownership.

class FileCopier

#include <FileCopier.h>

Class to manage copying of files and directories including attributes.

Public Types

using ErrorHandler = Delegate<bool(const ErrorInfo &info)>: Return true to ignore error and continue copying, false to stop.

struct ErrorInfo: #include <FileCopier.h>

Error information passed to callback.

class FileSystem : public IFS::IFileSystem 

#include <FileSystem.h>

Installable File System base class.

Adds additional methods to ease use over base IFileSystem.

rename a file

param oldpath:
param newpath:

retval int:: error code

using ReadContentCallback = Delegate<int(const char *buffer, size_t size)>

Callback for readContent method.

Param buffer:
Param size:
Retval int:: Return number of bytes consumed, < size to stop If < 0 then this is returned as error code to readContent call.

inline int remove(const String &path)

remove (delete) a file by path

Parameters:: path –
Return values:: int – error code

template<typename T> inline int setacl(const T &file, const ACL &acl)

Set access control information for file.

Parameters:

file – handle or path to file
acl –

Return values:

int – error code

template<typename T> inline int setattr(const T &file, FileAttributes attr)

Set file attributes.

Parameters:

file – handle or path to file
attr –

Return values:

int – error code

template<typename T> inline int settime(const T &file, time_t mtime)

Set modification time for file.

Note

any subsequent writes to file will reset this to current time

Parameters:: file – handle or path to file
Return values:: int – error code

template<typename T> inline int setcompression(const T &file, const Compression &compression)

Set file compression information.

Parameters:

file –
compression –

Return values:

int – error code

file_size_t getSize(FileHandle file)

Get size of file.

Parameters:: file – File handle
Return values:: file_size_t – Size of file in bytes, 0 on error

file_size_t getSize(const char *fileName)

Get size of file.

Parameters:: fileName – Name of file
Return values:: file_size_t – Size of file in bytes, 0 on error

file_offset_t readContent(FileHandle file, size_t size, ReadContentCallback callback)

Read from current file position and invoke callback for each block read.

Parameters:

file –
size – Maximum number of bytes to read
callback –

Return values:

file_offset_t – Number of bytes processed, or error code

file_offset_t readContent(FileHandle file, ReadContentCallback callback)

Read from current file position to end of file and invoke callback for each block read.

Parameters:

file –
callback –

Return values:

file_offset_t – Number of bytes processed, or error code

file_offset_t readContent(const String &filename, ReadContentCallback callback)

Read entire file content in blocks, invoking callback after every read.

Parameters:

filename –
callback –

Return values:

file_offset_t – Number of bytes processed, or error code

int remove(const char *path) = 0

remove (delete) a file by path

Parameters:: path –
Return values:: int – error code

Read content of a file

After calling this function the content of the file is placed in to a c-string Ensure there is sufficient space in the buffer for file content plus extra trailing null, i.e. at least bufSize + 1 Always check the return value!

param fileName:: Name of file to read from
param buffer:: Pointer to a character buffer in to which to read the file content
param bufSize:: Quantity of bytes to read from file

retval size_t:: Quantity of bytes read from file or zero on failure

Returns 0 if the file could not be read

Create or replace file with defined content

This function creates a new file or replaces an existing file and populates the file with the content of a c-string buffer.

param fileName:: Name of file to create or replace
param content:: Pointer to c-string containing content to populate file with

retval int:: Number of bytes transferred or error code

param length:: (optional) number of characters to write

Public Functions

inline int opendir(const String &path, DirHandle &dir): open a directory for reading

int makedirs(const char *path)

Create a directory and any intermediate directories if they do not already exist.

Parameters:: path – Path to directory. If no trailing ‘/’ is present the final element is considered a filename.
Return values:: int – error code

int truncate(const char *fileName, file_size_t newSize)

Truncate a file to a specific size.

Parameters:: fileName – File to truncate
Return values:: int – new file size, or error code

String getContent(const String &fileName)

Read content of a file.

Note

After calling this function the content of the file is placed in to a string. The result will be an invalid String (equates to false) if the file could not be read. If the file exists, but is empty, the result will be an empty string “”.

Parameters:: fileName – Name of file to read from
Return values:: String – String variable in to which to read the file content

int opendir(const char *path, DirHandle &dir) = 0

open a directory for reading

Parameters:

path – path to directory. nullptr is interpreted as root directory
dir – returns a pointer to the directory object

Return values:

int – error code

int mkdir(const char *path) = 0

Create a directory.

Only the final directory in the path is guaranteed to be created. Usually, this call will fail if intermediate directories are not present. Use IFS::FileSystem::makedirs() for this purpose.

Parameters:: path – Path to directory
Return values:: int – error code

int stat(const char *path, Stat *stat) = 0

get file information

Returned stat will indicate whether the path is a mountpoint or directory. For a mount point, stats for the root directory of the mounted filesystem must be obtained by opening a handle then using fstat:

int handle = fs.open("path-to-mountpoint");
Stat stat;
fs.fstat(handle, &stat);
fs.close(handle);

Parameters:

path – name or path of file/directory/mountpoint
s – structure to return information in, may be null to do a simple file existence check

Return values:

int – error code

int fstat(FileHandle file, Stat *stat) = 0

get file information

Parameters:

file – handle to open file
stat – structure to return information in, may be null

Return values:

int – error code

FileHandle open(const char *path, OpenFlags flags) = 0

open a file (or directory) by path

This function may also be used to obtain a directory handle to perform various operations such as enumerating attributes. Calls to read or write on such handles will typically fail.

Parameters:

path – full path to file
flags – Desired access and other options

Return values:

FileHandle – file handle or error code

int ftruncate(FileHandle file, file_size_t new_size) = 0

Truncate (reduce) the size of an open file.

Note

In POSIX ftruncate() can also make the file bigger, however SPIFFS can only reduce the file size and will return an error if newSize > fileSize

Parameters:

file – Open file handle, must have Write access
newSize –

Return values:

int – Error code

int rename(const char *oldpath, const char *newpath) = 0

rename a file

Parameters:

oldpath –
newpath –

Return values:

int – error code

class FsBase

#include <FsBase.h>

Subclassed by IFS::Directory, IFS::FWFS::ArchiveStream, IFS::File, IFS::FileStream

Public Functions

inline int getLastError()

determine if an error occurred during operation

Return values:: int – filesystem error code

class IFileSystem

#include <IFileSystem.h>

Installable File System base class.

Construction and initialisation of a filing system is implementation-dependent so there are no methods here for that.

Methods are defined as virtual abstract unless we actually have a default base implementation. Whilst some methods could just return Error::NotImplemented by default, keeping them abstract forces all file system implementations to consider them so provides an extra check for completeness.

Note

The ‘I’ implies Installable but could be for Interface :-)

Subclassed by IFS::FAT::FileSystem, IFS::FWFS::FileSystem, IFS::FileSystem, IFS::Gdb::FileSystem, IFS::HYFS::FileSystem, IFS::Host::FileSystem

Public Functions

inline virtual ~IFileSystem(): Filing system implementations should dismount and cleanup here.

virtual int mount() = 0

Mount file system, performing any required initialisation.

Return values:: error – code

virtual int getinfo(Info &info) = 0

get filing system information

Parameters:: info – structure to read information into
Return values:: int – error code

inline virtual int setProfiler(IProfiler *profiler)

Set profiler instance to enable debugging and performance assessment.

Parameters:: profiler –
Return values:: int – error code - profiling may not be supported on all filesystems

inline virtual String getErrorString(int err)

get the text for a returned error code

Return values:: String –

inline virtual int setVolume(uint8_t index, IFileSystem *fileSystem)

Set volume for mountpoint.

Parameters:

index – Volume index
fileSystem – The filesystem to root at this mountpoint

Return values:

int – error code

virtual int opendir(const char *path, DirHandle &dir) = 0

open a directory for reading

Parameters:

path – path to directory. nullptr is interpreted as root directory
dir – returns a pointer to the directory object

Return values:

int – error code

virtual int readdir(DirHandle dir, Stat &stat) = 0

read a directory entry

Note

File system allocates entries structure as it usually needs to track other information. It releases memory when closedir() is called.

Parameters:

dir –
stat –

Return values:

int – error code

virtual int rewinddir(DirHandle dir) = 0

Reset directory read position to start.

Parameters:: dir –
Return values:: int – error code

virtual int closedir(DirHandle dir) = 0

close a directory object

Parameters:: dir – directory to close
Return values:: int – error code

virtual int mkdir(const char *path) = 0

Create a directory.

Only the final directory in the path is guaranteed to be created. Usually, this call will fail if intermediate directories are not present. Use IFS::FileSystem::makedirs() for this purpose.

Parameters:: path – Path to directory
Return values:: int – error code

virtual int stat(const char *path, Stat *stat) = 0

get file information

Returned stat will indicate whether the path is a mountpoint or directory. For a mount point, stats for the root directory of the mounted filesystem must be obtained by opening a handle then using fstat:

int handle = fs.open("path-to-mountpoint");
Stat stat;
fs.fstat(handle, &stat);
fs.close(handle);

Parameters:

path – name or path of file/directory/mountpoint
s – structure to return information in, may be null to do a simple file existence check

Return values:

int – error code

virtual int fstat(FileHandle file, Stat *stat) = 0

get file information

Parameters:

file – handle to open file
stat – structure to return information in, may be null

Return values:

int – error code

inline virtual int fcontrol(FileHandle file, ControlCode code, void *buffer, size_t bufSize)

Low-level and non-standard file control operations.

To simplify usage the same buffer is used for both input and output. Only the size of the buffer is provided. If a specific FCNTL code requires more information then it will be contained within the provided data.

Parameters:

file –
code – FCNTL_XXX code
buffer – Input/Output buffer
bufSize – Size of buffer

Return values:

int – error code or, on success, data size

virtual FileHandle open(const char *path, OpenFlags flags) = 0

open a file (or directory) by path

This function may also be used to obtain a directory handle to perform various operations such as enumerating attributes. Calls to read or write on such handles will typically fail.

Parameters:

path – full path to file
flags – Desired access and other options

Return values:

FileHandle – file handle or error code

virtual int close(FileHandle file) = 0

close an open file

Parameters:: file – handle to open file
Return values:: int – error code

virtual int read(FileHandle file, void *data, size_t size) = 0

read content from a file and advance cursor

Parameters:

file – handle to open file
data – buffer to write into
size – size of file buffer, maximum number of bytes to read

Return values:

int – number of bytes read or error code

virtual int write(FileHandle file, const void *data, size_t size) = 0

write content to a file at current position and advance cursor

Parameters:

file – handle to open file
data – buffer to read from
size – number of bytes to write

Return values:

int – number of bytes written or error code

virtual file_offset_t lseek(FileHandle file, file_offset_t offset, SeekOrigin origin) = 0

change file read/write position

Parameters:

file – handle to open file
offset – position relative to origin
origin – where to seek from (start/end or current position)

Return values:

file_offset_t – current position or error code

virtual int eof(FileHandle file) = 0

determine if current file position is at end of file

Parameters:: file – handle to open file
Return values:: int – 0 - not EOF, > 0 - at EOF, < 0 - error

virtual file_offset_t tell(FileHandle file) = 0

get current file position

Parameters:: file – handle to open file
Return values:: file_offset_t – current position relative to start of file, or error code

virtual int ftruncate(FileHandle file, file_size_t new_size) = 0

Truncate (reduce) the size of an open file.

Note

In POSIX ftruncate() can also make the file bigger, however SPIFFS can only reduce the file size and will return an error if newSize > fileSize

Parameters:

file – Open file handle, must have Write access
newSize –

Return values:

int – Error code

virtual int flush(FileHandle file) = 0

flush any buffered data to physical media

Parameters:: file – handle to open file
Return values:: int – error code

virtual int fsetxattr(FileHandle file, AttributeTag tag, const void *data, size_t size) = 0

Set an extended attribute on an open file.

Note

Attributes may not be written to disk until flush() or close() are called

Parameters:

file – handle to open file
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int fgetxattr(FileHandle file, AttributeTag tag, void *buffer, size_t size) = 0

Get an extended attribute from an open file.

Parameters:

file – handle to open file
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual int fenumxattr(FileHandle file, AttributeEnumCallback callback, void *buffer, size_t bufsize) = 0

Enumerate attributes.

Parameters:

file – handle to open file
callback – Callback function to invoke for each attribute found
buffer – Buffer to use for reading attribute data. Use nullptr if only tags are required
bufsize – Size of buffer

Return values:

int – error code, on success returns number of attributes read

virtual int setxattr(const char *path, AttributeTag tag, const void *data, size_t size) = 0

Set an extended attribute for a file given its path.

Parameters:

path – Full path to file (or directory)
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int getxattr(const char *path, AttributeTag tag, void *buffer, size_t size) = 0

Get an attribute from a file given its path.

Parameters:

file – Full path to file (or directory)
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

inline virtual int fgetextents(FileHandle file, Storage::Partition *part, Extent *list, uint16_t extcount)

Get extents for a file.

Parameters:

file – Handle to open file
part – Partition where the file lives (OUT, OPTIONAL)
list – Buffer for extents (OPTIONAL)
extcount – Maximum number of extents to return in list

Return values:

int – Total number of extents for file (may be larger than ‘extcount’), or error code

virtual int rename(const char *oldpath, const char *newpath) = 0

rename a file

Parameters:

oldpath –
newpath –

Return values:

int – error code

virtual int remove(const char *path) = 0

remove (delete) a file by path

Parameters:: path –
Return values:: int – error code

virtual int fremove(FileHandle file) = 0

remove (delete) a file by handle

Parameters:: file – handle to open file
Return values:: int – error code

virtual int format() = 0

format the filing system

Note

this does a default format, returning file system to a fresh state The filing system implementation may define more specialised methods which can be called directly.

Return values:: int – error code

inline virtual int check()

Perform a file system consistency check.

Note

if possible, issues should be resolved. Returns 0 if file system checked out OK. Otherwise there were issues: < 0 for unrecoverable errors,

0 for recoverable errors.

Return values:: int – error code

struct Info

#include <IFileSystem.h>

Basic information about filing system.

Subclassed by IFS::IFileSystem::NameInfo

Public Members

Type type = {}: The filing system type identifier.

Attributes attr = {}: Attribute flags.

size_t maxNameLength = {255}: Maximum length of a single file name.

size_t maxPathLength = {255}: Maximum length of a full file path.

uint32_t volumeID = {0}: Unique identifier for volume.

NameBuffer name: Buffer for name.

volume_size_t volumeSize = {0}: Size of volume, in bytes.

volume_size_t freeSpace = {0}: Available space, in bytes.

struct NameInfo : public IFS::IFileSystem::Info : #include <IFileSystem.h>

Filing system information with buffer for name.

struct NameBuffer

#include <NameBuffer.h>

defines a ‘safe’ name buffer

Note

Instead of including a fixed name array in Stat (and IFileSystem::Info) structures, we use a NameBuffer to identify a separate buffer. This has several advantages:

- Maximum size is not fixed
- Finding and copying the name is optional
- Actual name length is returned in the 'length' field, regardless of size
- A NameBuffer structure (or one containing it) only requires initialising once before
a loop operation as buffer/size are preserved.

There are fancier ways to do this but a structure is transparent and requires no heap allocation.

Note

length always reflects the required name/path length, and may be longer than size.

Subclassed by IFS::FileNameBuffer

Public Functions

inline NameBuffer(String &s): Make a NameBuffer point to contents of a String.

inline int copy(const char *src, uint16_t srclen)

copies text from a source buffer into a name buffer

Note

length field is always set to srclen, regardless of number of characters copied.

Parameters:

src – source name
srclen – number of characters in name

inline int addSep()

When building file paths this method simplified appending separators.

Note

if the path is not empty, a separator character is appended

Return values:: int – error code

inline char *endptr()

get a pointer to the next write position

Note

use space() to ensure buffer doesn’t overrun When writing text be sure to call terminate() when complete

Return values:: char* –

inline uint16_t space()

get the number of free characters available

Note

returns 0 if buffer has overrun

Return values:: uint16_t –

inline void terminate(): ensure the buffer has a nul terminator, even if it means overwriting content

inline bool overflow() const: determine if name buffer overflowed

Note

Compares returned length with buffer size; A nul terminator is always appended, so size should be >= (length + 1)

Public Members

char *buffer = {nullptr}: Buffer to store name.

uint16_t size = {0}: IN: Size of buffer.

uint16_t length = {0}: OUT: length of name.

struct FileNameBuffer : public IFS::NameBuffer : #include <NameBuffer.h>

a quick’n’dirty name buffer with maximum path allocation

class IProfiler

#include <Profiler.h>

Filesystems may optionally provide performance statistics.

Subclassed by IFS::Profiler

Public Functions

virtual void erase(storage_size_t address, size_t size) = 0: Called BEFORE an erase operation.

class Profiler : public IFS::IProfiler 

#include <Profiler.h>

Public Functions

inline virtual void erase(storage_size_t address, size_t size) override: Called BEFORE an erase operation.

struct Stat: #include <Profiler.h>

struct Stat

#include <Stat.h>

File Status structure.

Subclassed by IFS::NameStat

Public Functions

inline Stat &operator=(const Stat &rhs): assign content from another Stat structure

Note

All fields are copied as for a normal assignment, except for ‘name’, where rhs.name contents are copied into our name buffer.

inline bool isDir() const: Is this a directory (or mountpoint) ?

Public Members

IFileSystem *fs = {nullptr}: The filing system owning this file.

NameBuffer name: Name of file.

file_size_t size = {0}: Size of file in bytes.

FileID id = {0}: Internal file identifier.

TimeStamp mtime = {}: File modification time.

ACL acl = {UserRole::None, UserRole::None}: Access Control.

struct NameStat : public IFS::Stat : #include <Stat.h>

version of Stat with integrated name buffer

Note

provide for convenience

struct TimeStamp

#include <TimeStamp.h>

Manage IFS timestamps stored as an unsigned 32-bit value.

An unsigned 32-bit value containing seconds will overflow in about 136 years. time_t starts 1 Jan 1970 (Unix epoch).

Times are stored in UTC (GMT).

Public Functions

String toString(const char *dtsep = " ") const: Convert time to string for display.

namespace Debug

Typedefs

using Options = BitSet<uint8_t, Option, 2>

Enums

enum class Option

Values:

enumerator recurse

enumerator attributes

Functions

void printFsInfo(Print &out, FileSystem &fs)

void printAttrInfo(Print &out, FileSystem &fs, const String &filename)

int listDirectory(Print &out, FileSystem &fs, const String &path, Options options = 0)

namespace Error

Enums

enum class Value

Values:

enumerator XX

enumerator MAX

Functions

String toString(int err)

get text for an error code

Parameters:: err –
Return values:: String –

inline bool isSystem(int err): Determine if the given IFS error code is system-specific.

inline int fromSystem(int syscode): Translate system error code into IFS error code.

inline int toSystem(int err): Translate IFS error code into SYSTEM code.

Variables

constexpr ErrorCode USER = {-100}

constexpr ErrorCode SYSTEM = {-1000}

namespace FAT

Functions

int translateFatfsResult(uint8_t result, bool diskio_write)

String fatfsErrorToStr(uint8_t err)

ErrorCode calculateFatParam(Partition partition, const FormatOptions &opt, FatParam &param)

Deduce FAT volume parameters for given space.

Parameters:

partition – The partition to format
opt – Formatting options
param – On success, contains calculated parameters for FAT volume

Return values:

ErrorCode – When partitioning using MBR format, this method can be used to determine the Sys indicator value setting.

ErrorCode formatVolume(Partition partition, const FatParam &param)

Format partition using pre-calculated FAT parameters.

Parameters:

partition – The partition to format
param – Detailed FAT parameters (returned from calculateFatParam)

Return values:

ErrorCode – This function allows fine control over exactly how a FAT partition is constructed. Generally the calculateFatParam function should be used to populate the param structure, then any modifications can be made as required before actually formatting the volume.

inline ErrorCode formatVolume(Partition partition, const FormatOptions &opt = {})

Format partition with a blank FAT volume.

Parameters:

partition – The partition to format
opt – Formatting options

Return values:

ErrorCode –

class FileSystem : public IFS::IFileSystem 

#include <FileSystem.h>

Wraps fatfs

Public Functions

virtual int mount() override

Mount file system, performing any required initialisation.

Return values:: error – code

virtual int getinfo(Info &info) override

get filing system information

Parameters:: info – structure to read information into
Return values:: int – error code

virtual int setProfiler(IProfiler *profiler) override

Set profiler instance to enable debugging and performance assessment.

Parameters:: profiler –
Return values:: int – error code - profiling may not be supported on all filesystems

virtual String getErrorString(int err) override

get the text for a returned error code

Return values:: String –

virtual int opendir(const char *path, DirHandle &dir) override

open a directory for reading

Parameters:

path – path to directory. nullptr is interpreted as root directory
dir – returns a pointer to the directory object

Return values:

int – error code

virtual int readdir(DirHandle dir, Stat &stat) override

read a directory entry

Note

File system allocates entries structure as it usually needs to track other information. It releases memory when closedir() is called.

Parameters:

dir –
stat –

Return values:

int – error code

virtual int rewinddir(DirHandle dir) override

Reset directory read position to start.

Parameters:: dir –
Return values:: int – error code

virtual int closedir(DirHandle dir) override

close a directory object

Parameters:: dir – directory to close
Return values:: int – error code

virtual int mkdir(const char *path) override

Create a directory.

Only the final directory in the path is guaranteed to be created. Usually, this call will fail if intermediate directories are not present. Use IFS::FileSystem::makedirs() for this purpose.

Parameters:: path – Path to directory
Return values:: int – error code

virtual int stat(const char *path, Stat *stat) override

get file information

Returned stat will indicate whether the path is a mountpoint or directory. For a mount point, stats for the root directory of the mounted filesystem must be obtained by opening a handle then using fstat:

int handle = fs.open("path-to-mountpoint");
Stat stat;
fs.fstat(handle, &stat);
fs.close(handle);

Parameters:

path – name or path of file/directory/mountpoint
s – structure to return information in, may be null to do a simple file existence check

Return values:

int – error code

virtual int fstat(FileHandle file, Stat *stat) override

get file information

Parameters:

file – handle to open file
stat – structure to return information in, may be null

Return values:

int – error code

virtual int fcontrol(FileHandle file, ControlCode code, void *buffer, size_t bufSize) override

Low-level and non-standard file control operations.

To simplify usage the same buffer is used for both input and output. Only the size of the buffer is provided. If a specific FCNTL code requires more information then it will be contained within the provided data.

Parameters:

file –
code – FCNTL_XXX code
buffer – Input/Output buffer
bufSize – Size of buffer

Return values:

int – error code or, on success, data size

virtual int fsetxattr(FileHandle file, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute on an open file.

Note

Attributes may not be written to disk until flush() or close() are called

Parameters:

file – handle to open file
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int fgetxattr(FileHandle file, AttributeTag tag, void *buffer, size_t size) override

Get an extended attribute from an open file.

Parameters:

file – handle to open file
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual int fenumxattr(FileHandle file, AttributeEnumCallback callback, void *buffer, size_t bufsize) override

Enumerate attributes.

Parameters:

file – handle to open file
callback – Callback function to invoke for each attribute found
buffer – Buffer to use for reading attribute data. Use nullptr if only tags are required
bufsize – Size of buffer

Return values:

int – error code, on success returns number of attributes read

virtual int setxattr(const char *path, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute for a file given its path.

Parameters:

path – Full path to file (or directory)
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int getxattr(const char *path, AttributeTag tag, void *buffer, size_t size) override

Get an attribute from a file given its path.

Parameters:

file – Full path to file (or directory)
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual FileHandle open(const char *path, OpenFlags flags) override

open a file (or directory) by path

This function may also be used to obtain a directory handle to perform various operations such as enumerating attributes. Calls to read or write on such handles will typically fail.

Parameters:

path – full path to file
flags – Desired access and other options

Return values:

FileHandle – file handle or error code

virtual int close(FileHandle file) override

close an open file

Parameters:: file – handle to open file
Return values:: int – error code

virtual int read(FileHandle file, void *data, size_t size) override

read content from a file and advance cursor

Parameters:

file – handle to open file
data – buffer to write into
size – size of file buffer, maximum number of bytes to read

Return values:

int – number of bytes read or error code

virtual int write(FileHandle file, const void *data, size_t size) override

write content to a file at current position and advance cursor

Parameters:

file – handle to open file
data – buffer to read from
size – number of bytes to write

Return values:

int – number of bytes written or error code

virtual file_offset_t lseek(FileHandle file, file_offset_t offset, SeekOrigin origin) override

change file read/write position

Parameters:

file – handle to open file
offset – position relative to origin
origin – where to seek from (start/end or current position)

Return values:

file_offset_t – current position or error code

virtual int eof(FileHandle file) override

determine if current file position is at end of file

Parameters:: file – handle to open file
Return values:: int – 0 - not EOF, > 0 - at EOF, < 0 - error

virtual file_offset_t tell(FileHandle file) override

get current file position

Parameters:: file – handle to open file
Return values:: file_offset_t – current position relative to start of file, or error code

virtual int ftruncate(FileHandle file, file_size_t new_size) override

Truncate (reduce) the size of an open file.

Note

In POSIX ftruncate() can also make the file bigger, however SPIFFS can only reduce the file size and will return an error if newSize > fileSize

Parameters:

file – Open file handle, must have Write access
newSize –

Return values:

int – Error code

virtual int flush(FileHandle file) override

flush any buffered data to physical media

Parameters:: file – handle to open file
Return values:: int – error code

virtual int rename(const char *oldpath, const char *newpath) override

rename a file

Parameters:

oldpath –
newpath –

Return values:

int – error code

virtual int remove(const char *path) override

remove (delete) a file by path

Parameters:: path –
Return values:: int – error code

virtual int fremove(FileHandle file) override

remove (delete) a file by handle

Parameters:: file – handle to open file
Return values:: int – error code

virtual int format() override

format the filing system

Note

this does a default format, returning file system to a fresh state The filing system implementation may define more specialised methods which can be called directly.

Return values:: int – error code

virtual int check() override

Perform a file system consistency check.

Note

if possible, issues should be resolved. Returns 0 if file system checked out OK. Otherwise there were issues: < 0 for unrecoverable errors,

0 for recoverable errors.

Return values:: int – error code

struct FormatOptions

#include <Format.h>

Formatting options.

Public Members

SysTypes types = {0}: Valid partition format types.

uint8_t numFats = {1}: Number of FATs (1 or 2)

unsigned align = {0}: Data area alignment (sector)

unsigned numRootEntries = {512}: Number of root directory entries.

uint32_t clusterSize = {4096}: Cluster size (byte)

struct FatParam

#include <Format.h>

Public Members

uint32_t sectorsPerBlock: Flash erase block size.

uint16_t sectorsPerCluster: Set to 0 for auto-calculation.

namespace FWFS

Functions

template<typename T> static T at_offset(const void *current, int offset)

template<typename T> static T at_offset(void *current, int offset)

FileAttributes getFileAttributes(Object::Attributes objattr)

Object::Attributes getObjectAttributes(FileAttributes fileAttr)

Variables

constexpr size_t FWFS_BASE_OFFSET = {sizeof(uint32_t)}

constexpr uint32_t FWFILESYS_START_MARKER = {0x53465746}

constexpr uint32_t FWFILESYS_END_MARKER = {0x46574653}

class ArchiveStream : public IFS::FsBase, public IDataSourceStream 

#include <ArchiveStream.h>

Supports direct streaming into FWFS archive format.

Data needs to be enumerated so that all child files and directories are written first. As this happens, the parent directory is built as a list of object references.

The size of all child objects must be known before the containing object is written out. It should be sufficient to buffer all of these into an internal stream. This would only be a problem if large child objects are used. This is possible, but the builder avoids doing it and we should too.

Top-level object is a Volume, below that is the root Directory. That means objects are written in the order: child files/directories Root directory Volume End marker

Subclassed by ArchiveStream

Public Functions

inline ArchiveStream(FileSystem *fileSystem, VolumeInfo volumeInfo, String rootPath = nullptr, Flags flags = 0)

Construct an archive stream.

Parameters:

fileSystem – The filesystem to read
rootPath – Where to root the generated filesystem
flags –

inline virtual bool filterStat(const Stat &stat)

Override this method to filter items.

Use to omit temporary files or directories.

Parameters:: stat – The current item
Return values:: bool – Return true to process the item, false to skip it

inline virtual IBlockEncoder *createEncoder(FileInfo &file)

Override this method to implement custom encoding such as compression or encryption.

To support compression or encryption, this method can create the appropriate stream type and set the appropriate attributes using methods of FileInfo.

Parameters:: file – Details of the file being archived
Return values:: IBlockEncoder* – Stream to use for file content. Return nullptr for default behaviour.

inline const String &getCurrentPath() const: Get the current path being processed.

virtual uint16_t readMemoryBlock(char *data, int bufSize) override

Read a block of memory.

Todo:: Should IDataSourceStream::readMemoryBlock return same data type as its bufSize param?

Parameters:

data – Pointer to the data to be read
bufSize – Quantity of chars to read

Return values:

uint16_t – Quantity of chars read

virtual int seekFrom(int offset, SeekOrigin origin) override

Change position in stream.

Note

This method is implemented by streams which support random seeking, such as files and memory streams.

Parameters:

offset –
origin –

Return values:

New – position, < 0 on error

inline virtual bool isFinished() override

Check if all data has been read.

Return values:: bool – True on success.

inline virtual MimeType getMimeType() const override

Get MIME type for stream content.

Return values:: MimeType –

void reset(): Reset stream to beginning.

class FileInfo

#include <ArchiveStream.h>

Passed to callbacks to allow modification of output data.

Public Functions

int setAttribute(AttributeTag tag, const void *data, size_t size)

Set an additional attribute on the file.

These are written out before existing file metadata is copied, so will take priority. For example, if we set the compression attribute here then that is the one which the filesystem will use when mounted. However, the original compression attribute will still be present which may be helpful.

template<typename ...ParamTypes> inline int setUserAttribute(uint8_t tagValue, ParamTypes... params): Set an additional user attribute.

struct VolumeInfo

#include <ArchiveStream.h>

Public Members

String name: Volume Name.

uint32_t id = {0}: File system ID to store.

TimeStamp creationTime = {0}: Volume creation time, default is current system time (UTC)

class IBlockEncoder

#include <BlockEncoder.h>

Virtual base class to support (file) data encryption and compression.

Encryption and compression are typically done in blocks of a fixed size. To support these operations an instance of this class is created which encodes data one block at a time. Each block is stored separately and the resulting file consists of a chain of these blocks. This is natively supported by FWFS.

If the final data size is known in advance then the implementation will return just a single data stream.

Subclassed by IFS::FWFS::BasicEncoder

Public Functions

virtual IDataSourceStream *getNextStream() = 0

@Implement this method and return nullptr when all blocks have been encoded.

The stream returned must know it’s size (i.e. available() must not return -1). The encoder owns any stream objects created so is responsible for destroying them when finished. This allows them to be re-used if appropriate.

class BasicEncoder : public IFS::FWFS::IBlockEncoder 

#include <BlockEncoder.h>

Public Functions

inline virtual IDataSourceStream *getNextStream() override

@Implement this method and return nullptr when all blocks have been encoded.

The stream returned must know it’s size (i.e. available() must not return -1). The encoder owns any stream objects created so is responsible for destroying them when finished. This allows them to be re-used if appropriate.

struct FWFileDesc

#include <FileSystem.h>

FWFS File Descriptor.

Public Members

FWObjDesc odFile: File object.

uint32_t dataSize: Total size of data.

uint32_t cursor: Current read/write offset within file data.

struct FWVolume: #include <FileSystem.h>

FWFS Volume definition for mount points.

class FileSystem : public IFS::IFileSystem 

#include <FileSystem.h>

Implementation of firmware filing system using IFS.

Public Functions

virtual int mount() override

Mount file system, performing any required initialisation.

Return values:: error – code

virtual int getinfo(Info &info) override

get filing system information

Parameters:: info – structure to read information into
Return values:: int – error code

virtual String getErrorString(int err) override

get the text for a returned error code

Return values:: String –

virtual int setVolume(uint8_t index, IFileSystem *fileSystem) override

Set volume for mountpoint.

Parameters:

index – Volume index
fileSystem – The filesystem to root at this mountpoint

Return values:

int – error code

virtual int opendir(const char *path, DirHandle &dir) override

open a directory for reading

Parameters:

path – path to directory. nullptr is interpreted as root directory
dir – returns a pointer to the directory object

Return values:

int – error code

virtual int readdir(DirHandle dir, Stat &stat) override

read a directory entry

Note

File system allocates entries structure as it usually needs to track other information. It releases memory when closedir() is called.

Parameters:

dir –
stat –

Return values:

int – error code

virtual int rewinddir(DirHandle dir) override

Reset directory read position to start.

Parameters:: dir –
Return values:: int – error code

virtual int closedir(DirHandle dir) override

close a directory object

Parameters:: dir – directory to close
Return values:: int – error code

virtual int mkdir(const char *path) override

Create a directory.

Only the final directory in the path is guaranteed to be created. Usually, this call will fail if intermediate directories are not present. Use IFS::FileSystem::makedirs() for this purpose.

Parameters:: path – Path to directory
Return values:: int – error code

virtual int stat(const char *path, Stat *stat) override

get file information

Returned stat will indicate whether the path is a mountpoint or directory. For a mount point, stats for the root directory of the mounted filesystem must be obtained by opening a handle then using fstat:

int handle = fs.open("path-to-mountpoint");
Stat stat;
fs.fstat(handle, &stat);
fs.close(handle);

Parameters:

path – name or path of file/directory/mountpoint
s – structure to return information in, may be null to do a simple file existence check

Return values:

int – error code

virtual int fstat(FileHandle file, Stat *stat) override

get file information

Parameters:

file – handle to open file
stat – structure to return information in, may be null

Return values:

int – error code

virtual int fcontrol(FileHandle file, ControlCode code, void *buffer, size_t bufSize) override

Low-level and non-standard file control operations.

To simplify usage the same buffer is used for both input and output. Only the size of the buffer is provided. If a specific FCNTL code requires more information then it will be contained within the provided data.

Parameters:

file –
code – FCNTL_XXX code
buffer – Input/Output buffer
bufSize – Size of buffer

Return values:

int – error code or, on success, data size

virtual int fsetxattr(FileHandle file, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute on an open file.

Note

Attributes may not be written to disk until flush() or close() are called

Parameters:

file – handle to open file
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int fgetxattr(FileHandle file, AttributeTag tag, void *buffer, size_t size) override

Get an extended attribute from an open file.

Parameters:

file – handle to open file
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual int fenumxattr(FileHandle file, AttributeEnumCallback callback, void *buffer, size_t bufsize) override

Enumerate attributes.

Parameters:

file – handle to open file
callback – Callback function to invoke for each attribute found
buffer – Buffer to use for reading attribute data. Use nullptr if only tags are required
bufsize – Size of buffer

Return values:

int – error code, on success returns number of attributes read

virtual int setxattr(const char *path, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute for a file given its path.

Parameters:

path – Full path to file (or directory)
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int getxattr(const char *path, AttributeTag tag, void *buffer, size_t size) override

Get an attribute from a file given its path.

Parameters:

file – Full path to file (or directory)
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual FileHandle open(const char *path, OpenFlags flags) override

open a file (or directory) by path

This function may also be used to obtain a directory handle to perform various operations such as enumerating attributes. Calls to read or write on such handles will typically fail.

Parameters:

path – full path to file
flags – Desired access and other options

Return values:

FileHandle – file handle or error code

virtual int close(FileHandle file) override

close an open file

Parameters:: file – handle to open file
Return values:: int – error code

virtual int read(FileHandle file, void *data, size_t size) override

read content from a file and advance cursor

Parameters:

file – handle to open file
data – buffer to write into
size – size of file buffer, maximum number of bytes to read

Return values:

int – number of bytes read or error code

virtual int write(FileHandle file, const void *data, size_t size) override

write content to a file at current position and advance cursor

Parameters:

file – handle to open file
data – buffer to read from
size – number of bytes to write

Return values:

int – number of bytes written or error code

virtual file_offset_t lseek(FileHandle file, file_offset_t offset, SeekOrigin origin) override

change file read/write position

Parameters:

file – handle to open file
offset – position relative to origin
origin – where to seek from (start/end or current position)

Return values:

file_offset_t – current position or error code

virtual int eof(FileHandle file) override

determine if current file position is at end of file

Parameters:: file – handle to open file
Return values:: int – 0 - not EOF, > 0 - at EOF, < 0 - error

virtual file_offset_t tell(FileHandle file) override

get current file position

Parameters:: file – handle to open file
Return values:: file_offset_t – current position relative to start of file, or error code

virtual int ftruncate(FileHandle file, file_size_t new_size) override

Truncate (reduce) the size of an open file.

Note

In POSIX ftruncate() can also make the file bigger, however SPIFFS can only reduce the file size and will return an error if newSize > fileSize

Parameters:

file – Open file handle, must have Write access
newSize –

Return values:

int – Error code

virtual int flush(FileHandle file) override

flush any buffered data to physical media

Parameters:: file – handle to open file
Return values:: int – error code

virtual int fgetextents(FileHandle file, Storage::Partition *part, Extent *list, uint16_t extcount) override

Get extents for a file.

Parameters:

file – Handle to open file
part – Partition where the file lives (OUT, OPTIONAL)
list – Buffer for extents (OPTIONAL)
extcount – Maximum number of extents to return in list

Return values:

int – Total number of extents for file (may be larger than ‘extcount’), or error code

virtual int rename(const char *oldpath, const char *newpath) override

rename a file

Parameters:

oldpath –
newpath –

Return values:

int – error code

virtual int remove(const char *path) override

remove (delete) a file by path

Parameters:: path –
Return values:: int – error code

virtual int fremove(FileHandle file) override

remove (delete) a file by handle

Parameters:: file – handle to open file
Return values:: int – error code

inline virtual int format() override

format the filing system

Note

this does a default format, returning file system to a fresh state The filing system implementation may define more specialised methods which can be called directly.

Return values:: int – error code

inline virtual int check() override

Perform a file system consistency check.

Note

if possible, issues should be resolved. Returns 0 if file system checked out OK. Otherwise there were issues: < 0 for unrecoverable errors,

0 for recoverable errors.

Return values:: int – error code

struct Object

#include <Object.h>

Object structure.

Note

all objects conform to this structure. Only the first word (4 bytes) are required to nagivate the file system. All objects have an 8, 16 or 24-bit size field. Content is always immediately after this field. Reference objects are always 8-bit sized.

Public Types

enum class Attribute

Object attributes.

Note

these are bit values

Values:

enumerator ReadOnly: Object should not be modified or deleted.

enumerator Archive: Object modified flag.

Note

Object has been changed on disk. Typically used by backup applications

enumerator Encrypted

Object data is encrypted.

This is just a hint. Applications will typically provide additional user metadata to provide any additional information required for decryption.

enumerator MAX

using ID = uint32_t: Object identifier (offset from start of image)

Public Functions

inline size_t contentOffset() const: return offset to start of object content

inline uint32_t contentSize() const

return size of object content, excluding header and size fields

Note

must check return error code

Return values:: size – or error code

inline uint32_t size() const

total size this object occupies in the image

Return values:: size – or error code

Public Members

uint8_t typeData: Stored type plus flag.

uint32_t value: 32-bit identifier, e.g. volume ID

uint16_t _contentSize: Object size (excluding this header)

uint8_t namelen: Length of object name.

uint8_t _contentSizeHigh: Allows data up to 16MByte.

struct FWObjDesc

#include <Object.h>

FWFS Object Descriptor.

Public Members

Object::ID id: location

Object obj = {}: The object structure.

class ObjectBuffer: #include <ObjectBuffer.h>

Class to manage writing object data into a stream.

namespace Gdb

class FileSystem : public IFS::IFileSystem 

#include <FileSystem.h>

IFS implementation of Host filing system.

Public Functions

inline virtual int mount() override

Mount file system, performing any required initialisation.

Return values:: error – code

virtual int getinfo(Info &info) override

get filing system information

Parameters:: info – structure to read information into
Return values:: int – error code

virtual String getErrorString(int err) override

get the text for a returned error code

Return values:: String –

inline virtual int opendir(const char *path, DirHandle &dir) override

open a directory for reading

Parameters:

path – path to directory. nullptr is interpreted as root directory
dir – returns a pointer to the directory object

Return values:

int – error code

inline virtual int rewinddir(DirHandle dir) override

Reset directory read position to start.

Parameters:: dir –
Return values:: int – error code

inline virtual int readdir(DirHandle dir, Stat &stat) override

read a directory entry

Note

File system allocates entries structure as it usually needs to track other information. It releases memory when closedir() is called.

Parameters:

dir –
stat –

Return values:

int – error code

inline virtual int closedir(DirHandle dir) override

close a directory object

Parameters:: dir – directory to close
Return values:: int – error code

inline virtual int mkdir(const char *path) override

Create a directory.

Only the final directory in the path is guaranteed to be created. Usually, this call will fail if intermediate directories are not present. Use IFS::FileSystem::makedirs() for this purpose.

Parameters:: path – Path to directory
Return values:: int – error code

virtual int stat(const char *path, Stat *stat) override

get file information

Returned stat will indicate whether the path is a mountpoint or directory. For a mount point, stats for the root directory of the mounted filesystem must be obtained by opening a handle then using fstat:

int handle = fs.open("path-to-mountpoint");
Stat stat;
fs.fstat(handle, &stat);
fs.close(handle);

Parameters:

path – name or path of file/directory/mountpoint
s – structure to return information in, may be null to do a simple file existence check

Return values:

int – error code

virtual int fstat(FileHandle file, Stat *stat) override

get file information

Parameters:

file – handle to open file
stat – structure to return information in, may be null

Return values:

int – error code

inline virtual int fsetxattr(FileHandle file, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute on an open file.

Note

Attributes may not be written to disk until flush() or close() are called

Parameters:

file – handle to open file
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

inline virtual int fgetxattr(FileHandle file, AttributeTag tag, void *buffer, size_t size) override

Get an extended attribute from an open file.

Parameters:

file – handle to open file
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

inline virtual int fenumxattr(FileHandle file, AttributeEnumCallback callback, void *buffer, size_t bufsize) override

Enumerate attributes.

Parameters:

file – handle to open file
callback – Callback function to invoke for each attribute found
buffer – Buffer to use for reading attribute data. Use nullptr if only tags are required
bufsize – Size of buffer

Return values:

int – error code, on success returns number of attributes read

inline virtual int setxattr(const char *path, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute for a file given its path.

Parameters:

path – Full path to file (or directory)
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

inline virtual int getxattr(const char *path, AttributeTag tag, void *buffer, size_t size) override

Get an attribute from a file given its path.

Parameters:

file – Full path to file (or directory)
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual FileHandle open(const char *path, OpenFlags flags) override

open a file (or directory) by path

This function may also be used to obtain a directory handle to perform various operations such as enumerating attributes. Calls to read or write on such handles will typically fail.

Parameters:

path – full path to file
flags – Desired access and other options

Return values:

FileHandle – file handle or error code

virtual int close(FileHandle file) override

close an open file

Parameters:: file – handle to open file
Return values:: int – error code

virtual int read(FileHandle file, void *data, size_t size) override

read content from a file and advance cursor

Parameters:

file – handle to open file
data – buffer to write into
size – size of file buffer, maximum number of bytes to read

Return values:

int – number of bytes read or error code

virtual int write(FileHandle file, const void *data, size_t size) override

write content to a file at current position and advance cursor

Parameters:

file – handle to open file
data – buffer to read from
size – number of bytes to write

Return values:

int – number of bytes written or error code

virtual file_offset_t lseek(FileHandle file, file_offset_t offset, SeekOrigin origin) override

change file read/write position

Parameters:

file – handle to open file
offset – position relative to origin
origin – where to seek from (start/end or current position)

Return values:

file_offset_t – current position or error code

virtual int eof(FileHandle file) override

determine if current file position is at end of file

Parameters:: file – handle to open file
Return values:: int – 0 - not EOF, > 0 - at EOF, < 0 - error

virtual file_offset_t tell(FileHandle file) override

get current file position

Parameters:: file – handle to open file
Return values:: file_offset_t – current position relative to start of file, or error code

inline virtual int ftruncate(FileHandle file, file_size_t new_size) override

Truncate (reduce) the size of an open file.

Note

In POSIX ftruncate() can also make the file bigger, however SPIFFS can only reduce the file size and will return an error if newSize > fileSize

Parameters:

file – Open file handle, must have Write access
newSize –

Return values:

int – Error code

inline virtual int flush(FileHandle file) override

flush any buffered data to physical media

Parameters:: file – handle to open file
Return values:: int – error code

virtual int rename(const char *oldpath, const char *newpath) override

rename a file

Parameters:

oldpath –
newpath –

Return values:

int – error code

virtual int remove(const char *path) override

remove (delete) a file by path

Parameters:: path –
Return values:: int – error code

inline virtual int fremove(FileHandle file) override

remove (delete) a file by handle

Parameters:: file – handle to open file
Return values:: int – error code

inline virtual int format() override

format the filing system

Note

this does a default format, returning file system to a fresh state The filing system implementation may define more specialised methods which can be called directly.

Return values:: int – error code

inline virtual int check() override

Perform a file system consistency check.

Note

if possible, issues should be resolved. Returns 0 if file system checked out OK. Otherwise there were issues: < 0 for unrecoverable errors,

0 for recoverable errors.

Return values:: int – error code

namespace Host

Functions

FileSystem &getFileSystem()

inline int syserr(): Get IFS error code for the current system errno.

int mapFlags(OpenFlags flags): Get corresponding host flags for given IFS flags.

String getErrorString(int err)

class FileSystem : public IFS::IFileSystem 

#include <FileSystem.h>

IFS implementation of Host filing system.

Public Functions

virtual int mount() override

Mount file system, performing any required initialisation.

Return values:: error – code

virtual int getinfo(Info &info) override

get filing system information

Parameters:: info – structure to read information into
Return values:: int – error code

virtual String getErrorString(int err) override

get the text for a returned error code

Return values:: String –

virtual int opendir(const char *path, DirHandle &dir) override

open a directory for reading

Parameters:

path – path to directory. nullptr is interpreted as root directory
dir – returns a pointer to the directory object

Return values:

int – error code

virtual int rewinddir(DirHandle dir) override

Reset directory read position to start.

Parameters:: dir –
Return values:: int – error code

virtual int readdir(DirHandle dir, Stat &stat) override

read a directory entry

Note

File system allocates entries structure as it usually needs to track other information. It releases memory when closedir() is called.

Parameters:

dir –
stat –

Return values:

int – error code

virtual int closedir(DirHandle dir) override

close a directory object

Parameters:: dir – directory to close
Return values:: int – error code

virtual int mkdir(const char *path) override

Create a directory.

Only the final directory in the path is guaranteed to be created. Usually, this call will fail if intermediate directories are not present. Use IFS::FileSystem::makedirs() for this purpose.

Parameters:: path – Path to directory
Return values:: int – error code

virtual int stat(const char *path, Stat *stat) override

get file information

Returned stat will indicate whether the path is a mountpoint or directory. For a mount point, stats for the root directory of the mounted filesystem must be obtained by opening a handle then using fstat:

int handle = fs.open("path-to-mountpoint");
Stat stat;
fs.fstat(handle, &stat);
fs.close(handle);

Parameters:

path – name or path of file/directory/mountpoint
s – structure to return information in, may be null to do a simple file existence check

Return values:

int – error code

virtual int fstat(FileHandle file, Stat *stat) override

get file information

Parameters:

file – handle to open file
stat – structure to return information in, may be null

Return values:

int – error code

virtual int fsetxattr(FileHandle file, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute on an open file.

Note

Attributes may not be written to disk until flush() or close() are called

Parameters:

file – handle to open file
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int fgetxattr(FileHandle file, AttributeTag tag, void *buffer, size_t size) override

Get an extended attribute from an open file.

Parameters:

file – handle to open file
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual int fenumxattr(FileHandle file, AttributeEnumCallback callback, void *buffer, size_t bufsize) override

Enumerate attributes.

Parameters:

file – handle to open file
callback – Callback function to invoke for each attribute found
buffer – Buffer to use for reading attribute data. Use nullptr if only tags are required
bufsize – Size of buffer

Return values:

int – error code, on success returns number of attributes read

virtual int setxattr(const char *path, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute for a file given its path.

Parameters:

path – Full path to file (or directory)
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int getxattr(const char *path, AttributeTag tag, void *buffer, size_t size) override

Get an attribute from a file given its path.

Parameters:

file – Full path to file (or directory)
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual FileHandle open(const char *path, OpenFlags flags) override

open a file (or directory) by path

This function may also be used to obtain a directory handle to perform various operations such as enumerating attributes. Calls to read or write on such handles will typically fail.

Parameters:

path – full path to file
flags – Desired access and other options

Return values:

FileHandle – file handle or error code

virtual int close(FileHandle file) override

close an open file

Parameters:: file – handle to open file
Return values:: int – error code

virtual int read(FileHandle file, void *data, size_t size) override

read content from a file and advance cursor

Parameters:

file – handle to open file
data – buffer to write into
size – size of file buffer, maximum number of bytes to read

Return values:

int – number of bytes read or error code

virtual int write(FileHandle file, const void *data, size_t size) override

write content to a file at current position and advance cursor

Parameters:

file – handle to open file
data – buffer to read from
size – number of bytes to write

Return values:

int – number of bytes written or error code

virtual file_offset_t lseek(FileHandle file, file_offset_t offset, SeekOrigin origin) override

change file read/write position

Parameters:

file – handle to open file
offset – position relative to origin
origin – where to seek from (start/end or current position)

Return values:

file_offset_t – current position or error code

virtual int eof(FileHandle file) override

determine if current file position is at end of file

Parameters:: file – handle to open file
Return values:: int – 0 - not EOF, > 0 - at EOF, < 0 - error

virtual file_offset_t tell(FileHandle file) override

get current file position

Parameters:: file – handle to open file
Return values:: file_offset_t – current position relative to start of file, or error code

virtual int ftruncate(FileHandle file, file_size_t new_size) override

Truncate (reduce) the size of an open file.

Note

In POSIX ftruncate() can also make the file bigger, however SPIFFS can only reduce the file size and will return an error if newSize > fileSize

Parameters:

file – Open file handle, must have Write access
newSize –

Return values:

int – Error code

virtual int flush(FileHandle file) override

flush any buffered data to physical media

Parameters:: file – handle to open file
Return values:: int – error code

virtual int rename(const char *oldpath, const char *newpath) override

rename a file

Parameters:

oldpath –
newpath –

Return values:

int – error code

virtual int remove(const char *path) override

remove (delete) a file by path

Parameters:: path –
Return values:: int – error code

inline virtual int fremove(FileHandle file) override

remove (delete) a file by handle

Parameters:: file – handle to open file
Return values:: int – error code

inline virtual int format() override

format the filing system

Note

this does a default format, returning file system to a fresh state The filing system implementation may define more specialised methods which can be called directly.

Return values:: int – error code

inline virtual int check() override

Perform a file system consistency check.

Note

if possible, issues should be resolved. Returns 0 if file system checked out OK. Otherwise there were issues: < 0 for unrecoverable errors,

0 for recoverable errors.

Return values:: int – error code

namespace HYFS

class FileSystem : public IFS::IFileSystem 

#include <FileSystem.h>

Public Functions

virtual int mount() override

Mount file system, performing any required initialisation.

Return values:: error – code

virtual int getinfo(Info &info) override

get filing system information

Parameters:: info – structure to read information into
Return values:: int – error code

virtual String getErrorString(int err) override

get the text for a returned error code

Return values:: String –

virtual int setVolume(uint8_t index, IFileSystem *fileSystem) override

Set volume for mountpoint.

Parameters:

index – Volume index
fileSystem – The filesystem to root at this mountpoint

Return values:

int – error code

virtual int opendir(const char *path, DirHandle &dir) override

open a directory for reading

Parameters:

path – path to directory. nullptr is interpreted as root directory
dir – returns a pointer to the directory object

Return values:

int – error code

virtual int readdir(DirHandle dir, Stat &stat) override

read a directory entry

Note

File system allocates entries structure as it usually needs to track other information. It releases memory when closedir() is called.

Parameters:

dir –
stat –

Return values:

int – error code

virtual int rewinddir(DirHandle dir) override

Reset directory read position to start.

Parameters:: dir –
Return values:: int – error code

virtual int closedir(DirHandle dir) override

close a directory object

Parameters:: dir – directory to close
Return values:: int – error code

virtual int mkdir(const char *path) override

Create a directory.

Only the final directory in the path is guaranteed to be created. Usually, this call will fail if intermediate directories are not present. Use IFS::FileSystem::makedirs() for this purpose.

Parameters:: path – Path to directory
Return values:: int – error code

virtual int stat(const char *path, Stat *stat) override

get file information

Returned stat will indicate whether the path is a mountpoint or directory. For a mount point, stats for the root directory of the mounted filesystem must be obtained by opening a handle then using fstat:

int handle = fs.open("path-to-mountpoint");
Stat stat;
fs.fstat(handle, &stat);
fs.close(handle);

Parameters:

path – name or path of file/directory/mountpoint
s – structure to return information in, may be null to do a simple file existence check

Return values:

int – error code

virtual int fstat(FileHandle file, Stat *stat) override

get file information

Parameters:

file – handle to open file
stat – structure to return information in, may be null

Return values:

int – error code

virtual int fcontrol(FileHandle file, ControlCode code, void *buffer, size_t bufSize) override

Low-level and non-standard file control operations.

To simplify usage the same buffer is used for both input and output. Only the size of the buffer is provided. If a specific FCNTL code requires more information then it will be contained within the provided data.

Parameters:

file –
code – FCNTL_XXX code
buffer – Input/Output buffer
bufSize – Size of buffer

Return values:

int – error code or, on success, data size

virtual int fsetxattr(FileHandle file, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute on an open file.

Note

Attributes may not be written to disk until flush() or close() are called

Parameters:

file – handle to open file
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int fgetxattr(FileHandle file, AttributeTag tag, void *buffer, size_t size) override

Get an extended attribute from an open file.

Parameters:

file – handle to open file
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual int fenumxattr(FileHandle file, AttributeEnumCallback callback, void *buffer, size_t bufsize) override

Enumerate attributes.

Parameters:

file – handle to open file
callback – Callback function to invoke for each attribute found
buffer – Buffer to use for reading attribute data. Use nullptr if only tags are required
bufsize – Size of buffer

Return values:

int – error code, on success returns number of attributes read

virtual int setxattr(const char *path, AttributeTag tag, const void *data, size_t size) override

Set an extended attribute for a file given its path.

Parameters:

path – Full path to file (or directory)
tag – The attribute to write
data – Content of the attribute. Pass nullptr to remove the attribute (if possible).
size – Size of the attribute in bytes

Return values:

int – error code

virtual int getxattr(const char *path, AttributeTag tag, void *buffer, size_t size) override

Get an attribute from a file given its path.

Parameters:

file – Full path to file (or directory)
tag – The attribute to read
buffer – Buffer to receive attribute content
size – Size of the buffer

Return values:

int – error code, on success returns size of attribute (which may be larger than size)

virtual FileHandle open(const char *path, OpenFlags flags) override

open a file (or directory) by path

This function may also be used to obtain a directory handle to perform various operations such as enumerating attributes. Calls to read or write on such handles will typically fail.

Parameters:

path – full path to file
flags – Desired access and other options

Return values:

FileHandle – file handle or error code

virtual int close(FileHandle file) override

close an open file

Parameters:: file – handle to open file
Return values:: int – error code

virtual int read(FileHandle file, void *data, size_t size) override

read content from a file and advance cursor

Parameters:

file – handle to open file
data – buffer to write into
size – size of file buffer, maximum number of bytes to read

Return values:

int – number of bytes read or error code

virtual int write(FileHandle file, const void *data, size_t size) override

write content to a file at current position and advance cursor

Parameters:

file – handle to open file
data – buffer to read from
size – number of bytes to write

Return values:

int – number of bytes written or error code

virtual file_offset_t lseek(FileHandle file, file_offset_t offset, SeekOrigin origin) override

change file read/write position

Parameters:

file – handle to open file
offset – position relative to origin
origin – where to seek from (start/end or current position)

Return values:

file_offset_t – current position or error code

virtual int eof(FileHandle file) override

determine if current file position is at end of file

Parameters:: file – handle to open file
Return values:: int – 0 - not EOF, > 0 - at EOF, < 0 - error

virtual file_offset_t tell(FileHandle file) override

get current file position

Parameters:: file – handle to open file
Return values:: file_offset_t – current position relative to start of file, or error code

virtual int ftruncate(FileHandle file, file_size_t new_size) override

Truncate (reduce) the size of an open file.

Note

In POSIX ftruncate() can also make the file bigger, however SPIFFS can only reduce the file size and will return an error if newSize > fileSize

Parameters:

file – Open file handle, must have Write access
newSize –

Return values:

int – Error code

virtual int flush(FileHandle file) override

flush any buffered data to physical media

Parameters:: file – handle to open file
Return values:: int – error code

virtual int fgetextents(FileHandle file, Storage::Partition *part, Extent *list, uint16_t extcount) override

Get extents for a file.

Parameters:

file – Handle to open file
part – Partition where the file lives (OUT, OPTIONAL)
list – Buffer for extents (OPTIONAL)
extcount – Maximum number of extents to return in list

Return values:

int – Total number of extents for file (may be larger than ‘extcount’), or error code

virtual int rename(const char *oldpath, const char *newpath) override

rename a file

Parameters:

oldpath –
newpath –

Return values:

int – error code

virtual int remove(const char *path) override

remove (delete) a file by path

Parameters:: path –
Return values:: int – error code

virtual int fremove(FileHandle file) override

remove (delete) a file by handle

Parameters:: file – handle to open file
Return values:: int – error code

virtual int format() override

format the filing system

Note

this does a default format, returning file system to a fresh state The filing system implementation may define more specialised methods which can be called directly.

Return values:: int – error code

virtual int check() override

Perform a file system consistency check.

Note

if possible, issues should be resolved. Returns 0 if file system checked out OK. Otherwise there were issues: < 0 for unrecoverable errors,

0 for recoverable errors.

Return values:: int – error code

References

Used by

Sming (main) Component
SPI Flash Component
FatIFS Library
LittleFS Library
SPIFFS IFS Library Library

Environment Variables

ENABLE_FILE_SIZE64
FSBUILD
FSBUILD_OPTIONS
FWFS_DEBUG

SoC support

esp32
esp32c2
esp32c3
esp32s2
esp32s3
esp8266
host
rp2040