ffe

Display all input data using the specified data configuration

$ ffe [[-c|--configuration]] [path/to/config.ffe] [path/to/input]

$ ffe --output=[path/to/output] [[-c|--configuration]] [path/to/config.ffe] [path/to/input]

$ ffe [[-s|--structure]] [structure] [[-p|--print]] [format] [path/to/input]

$ ffe [[-f|--field-list]] "[FirstName,LastName,Age]" [[-c|--configuration]] [path/to/config.ffe] [path/to/input]

$ ffe [[-e|--expression]] "[LastName=Smith]" [[-c|--configuration]] [path/to/config.ffe] [path/to/input]

$ ffe [[-?|--help]]

ffe {-f format_file | -i input_format_spec} [-o output_format_spec] [OPTIONS] [FILE...]

-f format_file
    Specifies an external format file (often YAML-like) containing detailed record descriptions, positions, and field mappings. This is the primary method for complex layouts.

-i input_spec
    Defines the input format directly on the command line using field specifications (e.g., name:1-10,age:11-13). Useful for simpler, ad-hoc transformations.

-o output_spec
    Defines the output format directly on the command line, allowing reordering, renaming, and reformatting of fields from the input.

-F type
    Sets the desired output format type. Common types include fixed, delimited, csv, sqlinsert, xml, and json.

-d delimiter
    Specifies the field delimiter character to use when generating delimited output formats (e.g., , for CSV or \t for TSV).

-t tag_field
    Identifies a field whose value determines the record type. Essential for processing files that contain multiple distinct record layouts within a single file.

-h
    Treats the first record of the input file as a header row and skips it during data processing, optionally using its values for field names.

-s num
    Skips the first num records from the input file before beginning processing. Useful for bypassing introductory data.

-c num
    Processes only the next num records after any initial skipping. Useful for testing or processing partial files.

-v
    Enables verbose output, displaying detailed information about the processing steps, record parsing, and field transformations.

ffe is a powerful command-line utility meticulously crafted for processing fixed-format text files. These files, commonly encountered in legacy systems or mainframe data exports, typically organize information into columns of predefined widths. ffe excels at parsing, reordering, reformatting, and extracting specific fields from such datasets.

Its advanced capabilities include handling complex record layouts, even those containing multiple record types identified by distinct field values. This makes it an indispensable tool for non-interactive transformation of large data volumes, crucial for tasks like data migration, report generation, and system integration. ffe supports diverse output formats, ranging from delimited and fixed-width to SQL INSERT statements, XML, and JSON. It also offers features for basic data validation and on-the-fly field transformation.

ffe has a relatively steep learning curve due to its powerful, domain-specific nature, especially when dealing with complex format files. Precise definition of fixed-width layouts is crucial, and incorrect specifications can lead to cryptic error messages or unexpected output. It is primarily designed for batch processing and is not an interactive editor.

The true power of ffe often lies in its external format files. These configuration files, typically written in a YAML-like syntax, allow users to define complex record types, specific field names, precise byte positions, lengths, data types, and even transformation or validation rules. They enable ffe to handle highly structured, multi-record-type fixed-width data with high precision and flexibility.

ffe is an invaluable tool in scenarios involving:
- Data Migration: Seamlessly transforming fixed-width mainframe data into modern relational database formats.
- Reporting and Analytics: Extracting specific, structured fields from large fixed-width log files or reports for further analysis.
- Data Integration: Converting incoming fixed-width data feeds from external systems into a normalized or delimited format suitable for internal applications.

When defining fields via the -i or -o options, ffe supports various syntaxes. Fields can be defined by their exact start and end byte positions (e.g., field1:1-10), or by a field name followed by its length (e.g., field1:10), where positions are automatically calculated. For delimited data, regular expressions can also be used to define field boundaries.

ffe was created by J.R. "Bob" Muench to address the specific need for robust fixed-format data manipulation on Unix-like systems, often encountered when working with legacy or mainframe data. It has been actively developed and maintained, evolving to support modern data formats like JSON and XML, showcasing its adaptability and continued relevance in diverse data processing pipelines.

awk(1), cut(1), sed(1), grep(1), tr(1)