sc_warts2csv

Convert traceroute data in warts files to CSV and output it

$ sc_warts2csv [path/to/file1.warts path/to/file2.warts ...]

sc_warts2csv [options] [warts_file ...]

-o <file>
    Specifies the output file for the CSV data. If not provided, output is written to standard output (stdout).

-a
    Includes all possible fields in the CSV output, overriding any default field selections.

-s
    Suppresses the output of the header line in the CSV file, which normally contains the column names.

-d <delimiter>
    Sets the column delimiter for the CSV output. Default is typically a comma (,).

-f <fields>
    Specifies a comma-separated list of fields to include in the CSV output. E.g., 'hop_ip,rtt,ttl'.

-F <file>
    Reads a list of desired fields from the specified <file>, with one field per line.

-m <type>
    Filters the output to include only measurements of a specific type, such as 'ping', 'traceroute', or 'paris-traceroute'.

warts_file ...
    One or more input warts files to be processed. If no files are specified, input is read from standard input (stdin).

sc_warts2csv is a utility designed to transform binary network measurement data, typically collected by the scamper tool, from its native warts format into a comma-separated values (CSV) format. The warts format is a compact, binary representation optimized for storing large volumes of network probing results, such as those from traceroute or ping operations.

Converting to CSV makes the data easily readable by humans and importable into various analysis tools, spreadsheets (like Excel, Google Sheets, LibreOffice Calc), and databases. This conversion is crucial for post-measurement analysis, enabling researchers and network administrators to extract specific metrics, visualize trends, and perform statistical analysis on the collected network performance data without needing specialized warts parsers. It simplifies data sharing and interoperability with other data processing pipelines.

sc_warts2csv requires warts files as input, which are typically generated by the scamper network measurement tool. It cannot directly process raw network traffic captures.

The exact set of available fields for CSV output can vary depending on the warts file's content (i.e., what type of measurements it contains) and the version of scamper that generated it. Not all fields are universally present across all warts records. Converting very large warts files can be memory and CPU intensive, and the resulting CSV can also be very large.

By default, sc_warts2csv attempts to output a reasonable set of common fields based on the measurement type (e.g., traceroute will typically have hop_ip, rtt, ttl). Users can override this with the -a, -f, or -F options to customize the output fields.

If no input warts_file is specified, sc_warts2csv will read warts data from standard input (stdin). This allows it to be efficiently chained with other scamper utilities, for example: scamper -O warts | sc_warts2csv -s -m ping.

scamper and its associated utilities, including sc_warts2csv, were developed by the Center for Applied Internet Data Analysis (CAIDA) as part of their efforts to provide robust tools for large-scale internet measurement. The warts format itself was designed specifically for scamper to efficiently store diverse network measurement data.

sc_warts2csv was created to make this rich, binary data more accessible for analysis using common spreadsheet and database tools, reflecting the community's need for interoperable data formats in network research. Its development paralleled the growth and refinement of the scamper tool itself, ensuring that collected data could be easily consumed and analyzed by a broader audience of researchers and operators.

scamper(1), sc_warts2text(1), sc_warts2json(1)