demangle Demangle C++ function names in the CFG (requires that PolyFile was installed with the `demangle` option, or that the `cxxfilt` Python module is installed.) libmagic has an esoteric, poorly documented domain-specific language (DSL) for specifying its matching signatures. Full usage instructions follow: usage: polyfile [-h] [--format {file,mime,html,json,sbud}] [--output OUTPUT] definition, determine why a specific file fails to be classified as the expected MIME type, or step through a parser. TrID matching code is still shipped with PolyFile and can be invoked programmatically, but it is not used by default. Output Format

file format’s grammar. This is an area of active research intended to achieve fully automated grammar extraction from a identification rather than the libmagic file definitions. This proved to be very slow (since TrID has many duplicate debugger, -db drop into an interactive debugger for libmagic file definition matching and PolyFile parsing For an example that exercises all of these file formats, run: curl -v --silent https://www.sultanik.com/files/ESultanikResume.pdf | polyfile --html ESultanikResume.html -

Contributing to PyPI

As mentioned above, polymerge attempts to match each semantic type of the input file to a set of functions that are GDB. You can enter this debugger by passing the --debugger or -db argument to PolyFile. It is useful for both PolyFile has several options for outputting its results, specified by its --format option. For computer-readable output, PolyFile has an extension of the SBuD JSON format described in the documentation. Prior to version 0.5.0 this was the default output format of PolyFile. However, now the default output format is to mimic the behavior of the file command. To maintain the original behavior, use the --format sbud option. libMagic Implementation

entries) and prone to false positives (since TrID's file definitions are much simpler than libmagic's). The original PolyFile has an interactive debugger both for its file matching and parsing. It can be used to debug a libmagic pattern You can read the minimal and—as we have discovered in our cleanroom implementation— incomplete documentation by running For the CFG generation options, only render functions that participated in dataflow. `--dataflow 10` means that only functions in the dataflow related to byte 10 should be included. `--dataflow 10:30` means that only functions operating on bytes 10 through 29 should be included. The beginning or end of a range can be omitted and will default to the beginning and end of the file, respectively. Multiple `--dataflow` ranges can be specified. `--dataflow :` will render the CFG only with functions that operated on tainted bytes. Omitting `--dataflow` will produce a CFG containing all functions. For each type within the semantic hierarchy, a list of functions that are “most specialized” in processing that type.polymerge can also optionally emit a Graphviz .dot file or rendered PDF of the runtime control-flow graph recorded For each function, calculate the Shannon entropy of the different types on which that function operated; man 5 magic. PolyFile implements an interactive debugger for stepping through the DSL specifications, modeled after explicitly match against the given filetype or filetype wildcard (default is to match against all filetypes) To be used in conjunction with `--dataflow`. If enabled, only functions in the dataflow graph if they operated on the tainted bytes. This can result in a disjoint dataflow graph.

A separate utility called polymerge is installed with PolyFile specifically designed to merge the output of both For each semantic type in the input file, collect the functions that operated on bytes from that type; It can be invoked programmatically by running: from polyfile.magic import MagicMatcher with open ( "file_to_test" , "rb" ) as f : # the default instance automatically loads all file definitions for match in MagicMatcher . DEFAULT_INSTANCE . match ( f . read ()): for mimetype in match . mimetypes : print ( f "Matched MIME: { mimetype } " ) print ( f "Match string: { match !s} " )

License and Acknowledgements

most specialized” in operating on that type. This is an active area of academic research and is likely to change in no-debug-python by default, the `--debugger` option will break on custom matchers and prompt to debug using PDB. This option will suppress those prompts.

To load a specific or custom file definition: list_of_paths_to_definitions = [ "def1" , "def2" ] matcher = MagicMatcher . parse ( * list_of_paths_to_definitions ) with open ( "file_to_test" , "rb" ) as f : for match in matcher . match ( f . read ()): ... Debugging the libmagic DSL Optional path to output a Graphviz .dot file representing the type hierarchy extracted from PolyFile curl -v --silent https://www.sultanik.com/files/ESultanikResume.pdf | polyfile --html ESultanikResume.html - cfg CFG, -c CFG Optional path to output a Graphviz .dot file representing the control flow graph of the program trace