sc_warts2text

Output the information in warts files as text

$ sc_warts2text [path/to/file1.warts path/to/file2.warts ...]

sc_warts2text [options] warts_file [warts_file ...]

-p
    Pretty print the output in JSON format with human-readable indentation.

-r
    Produce raw (un-indented) JSON output, often preferred for scripting and large data processing.

-m
    Include metadata fields in the output, providing additional information about the measurement context.

-o
    Write the converted output to the specified file instead of standard output (stdout).

-J
    Force output in JSON format. This is often the default behavior in newer versions but can be explicitly specified for clarity.

-t
    Force output in plain text format, typically tab-separated values. This was the default in older versions of the utility.

-S
    Skip records in the input file until the specified byte offset is reached. Useful for resuming processing or partial reads of large files.

-P
    Process only a specified length of bytes from the input file, starting from the current position or after any -S offset.

-d
    Specify a directory containing custom schema files for interpreting extended or non-standard warts structures. Rarely used by typical users.

-F
    Print the version information of the utility and exit.

-h
    Print a help message with usage instructions and available options, then exit.

The sc_warts2text command is a crucial utility within the scamper suite, specifically designed to convert binary warts files into a human-readable text format. Warts files serve as the native, efficient binary output format for network measurement data collected by scamper during active probing campaigns, containing detailed results from operations like traceroutes, pings, and various other internet path measurements.

This command empowers network researchers, operators, and analysts to easily access and inspect the granular measurement data. It facilitates the conversion into widely compatible formats such as JSON or plain text (often tab-separated), making the complex binary data accessible for further processing, scripting, statistical analysis, and seamless integration with other data processing tools and platforms. sc_warts2text provides a set of flexible options to precisely control the output format, verbosity, and file handling, catering to diverse analysis needs.

When processing very large warts files, the resulting text output can be extremely verbose and may consume significant disk space or terminal buffer. Users should consider piping output to other tools (e.g., less, grep, jq) or redirecting to a file for efficient handling. The default output format (JSON vs. plain text) might vary slightly depending on the scamper version; explicit use of -J or -t is recommended for consistent results across different environments.

The warts file format is a proprietary binary format designed by CAIDA to store network measurement results efficiently. It supports various types of measurements (e.g., traceroute, ping, active probing) and is optimized for the compact storage and high-speed retrieval of large datasets generated by scamper. sc_warts2text is the primary command-line tool for converting this specialized binary format into a more widely accessible text representation.

When utilizing the JSON output mode (either by default or explicitly with the -J option), the output of sc_warts2text can be seamlessly piped into the jq command-line JSON processor. This powerful combination allows for advanced filtering, parsing, and manipulation of the measurement data. For example, to extract all source IP addresses from a warts file containing traceroute records, one could use: sc_warts2text file.warts -J | jq '.records[].srcip'.

sc_warts2text is an integral component of the scamper measurement tool suite, primarily developed and maintained by the CAIDA (Cooperative Association for Internet Data Analysis) project. Scamper was initially created to perform highly scalable and flexible active network measurements, producing its results in the efficient binary warts format. As such, sc_warts2text emerged as a crucial companion tool, empowering researchers and network operators to extract and interpret this complex binary data without requiring specialized programming. Its development has closely mirrored that of scamper, evolving to support new warts file versions and adapting to modern data interchange standards like JSON, reflecting the increasing demand for structured data suitable for automated analysis and visualization.

scamper(1), sc_remap(1), sc_filter(1), sc_tbit(1)