recon tool suite

Based on the services identified on a network host, we often need to run specific tools to assess the security/configuration of these services. The recon tool suite can help you automate that and analyze/summarize the results.

motivation

Instead of manually running various tools (e.g. testssl.sh, Nikto, feroxbuster, etc.) and having to remember all commands and the necessary options, we can configure them once (see config/scanner.toml) and have the scanner (i.e. scan.py) run the required/appropriate tools based on what the Nmap service scan (e.g. services.xml) has found. In addition to that, the suite also provides a tool to analyze and summarize the results of some scans (e.g. HTTP response headers, various protocol-specific configurations, etc.). This allows for an automated and consistent assessment of specific services (i.e. no longer are testers forced to analyze configurations by hand).

installation

The tools in this suite (i.e. analyze.py and scan.py) require Python 3.11+, and the defusedxml and Jinja modules.

Clone the git repository:

# this step is optional; change the directory name to your preference
mkdir --parents $HOME/tools && cd $_

git clone https://github.com/4elta/recon.git
cd recon

Install the required tools:

mkdir --parents $HOME/tools
./install-required-tools.sh $HOME/tools

The script will install the following tools:

• BIND 9
• curl
• enum4linux-ng
• IKE scanner
• NFS support
• Nmap
• Nikto
• Python libraries
  • defusedxml
  • dnspython
  • Impacket
  • Jinja
• RPC support
• Samba client
• SecLists
• SIPVicious
• testssl.sh
• WhatWeb

Based on the scan config (i.e. config/scanner.toml) you are using, you might have to install additional tools.

Add symbolic links to the scripts to /usr/local/bin. Please make sure, that the names for analyze and scan don't conflict with any binaries already installed.

sudo ln --symbolic "$(realpath analyze.py)" /usr/local/bin/analyze
sudo ln --symbolic "$(realpath scan.py)" /usr/local/bin/scan

usage

1. run your "standard" Nmap port and service scan(s); make sure to use -sV and -oX services.xml for the service scan
2. run the scanner on the results of the Nmap service scan; be aware that this (like the Nmap scan itself) will send requests to the target system(s)
3. run the analyzer on the results of the scanner; no network traffic will be generated during this step

You can customize the configuration for the scanner (i.e. what tools to run, etc.) by modifying the provided one (i.e. config/scanner.toml), or you can specify your own with the --config argument. Similarly, you can modify the recommendations based on what the analyzers will evaluate certain services. Make sure to have a look at the architecture documentation and/or study the provided configuration files.

scanner

% scan -h
usage: scan [-h] [-i path [path ...]] [-o path] [-c path [path ...]]
            [-t number] [-s number] [-m seconds] [-n]
            [-f <host> <protocol> <port> <service> 
               [<host> <protocol> <port> <service> ...]]
            [-y] [-d character] [--ignore-uid]

Schedule and execute various tools based on the findings of an Nmap service
scan.

options:
  -h, --help            show this help message and exit
  -i, --input path [path ...]
                        path to the result file(s) of the Nmap service scan
                        (default: 'services.xml')
  -o, --output path     path to where the results are stored (default:
                        './recon')
  -c, --config path [path ...]
                        path to the scanner configuration file(s); see
                        '/path/to/recon/config/scanner.toml'
  -t, --concurrent-targets number
                        number of targets that should be scanned concurrently
                        (default: 3)
  -s, --concurrent-scans number
                        number of scans that should be running concurrently on
                        a single target (default: 2)
  -m, --max-time seconds
                        maximum time in seconds each scan is allowed to take
                        (default: 3600)
  -n, --dry-run         do not run any command; just create/update the
                        'commands.csv' file
  -f, --filter <host> <protocol> <port> <service> 
              [<host> <protocol> <port> <service> ...]
                        specify hosts/protocols/ports/services you want to
                        (re)scan and overwrite their result files if they
                        exist; use '*' if you cannot or don't want to specify
                        a host/protocol/port/service part
  -y, --overwrite-results
                        overwrite existing result files
  -d, --delimiter character
                        character used to delimit columns in the
                        'commands.csv' and 'services.csv' files (default: ',')
  --ignore-uid          ignore the warning about potentially lacking
                        permissions

After running the scanner, the results directory (e.g. recon/) will contain the following files/directories:

• commands.csv: contains information about the executed commands (incl. start time, end time and return code)
• config_<YYYY-mm-dd_HH-MM-SS>.json: the scan configuration used for the scan
• scanner_<YYYY-mm-dd_HH-MM-SS>.log: the debug/error log of the scanner
• services.csv: contains information about the identified services (incl. whether they have been scanned or not)
• <IP address>/: each host has its own directory where the result files of the various tools are stored
  • the result files follow a specific naming scheme: <service>[,<transport protocol>,<port>,...],<tool>.<ext>
• <IP address>.md: this file contains a list of services identified on this particular host

analyzer

% analyze -h
usage: analyze [-h] [-c path] [-s code] [-S name] [-r path] [-i path]
               [-l code] [-f code] [--template path] [-o path]

Analyze and summarize the results of specific tools previously run by the
scanner of the recon tool suite (i.e. 'scan').

options:
  -h, --help            show this help message and exit
  -c, --config path     path to the analyzer configuration file (default:
                        '/path/to/recon/config/analyzer.toml')
  -s, --service code    service that should be analyzed (choices: ['dns',
                        'ftp', 'http', 'isakmp', 'ntp', 'rdp', 'smb', 'ssh',
                        'tls'])
  -S, --scan name       name of the tool/scan whose results should be parsed
  -r, --recommendations path
                        path to the recommendations document (default: '/path/
                        to/recon/config/recommendations/<service>/default.toml')
  -i, --input path      path to the root directory that holds the results to
                        be analysed (default: './recon')
  -l, --language code   language of the analysis (default: 'en')
  -f, --format code     format of the analysis (choices: ['csv', 'json',
                        'md']; default: 'md')
  --template path       path to the Jinja2 template for the analysis; this
                        option overrides '-f/--format'
  -o, --output path     path to the directory where the analysis result(s)
                        will be saved

The following analyzers (and parsers) are currently implemented:

• DNS configuration (nase, nmap)
• FTP configuration (nmap)
• HTTP response headers (curl, nmap)
• ISAKMP/IKE configuration (ike)
• NTP configuration (nmap, ntp)
• RDP configuration (nmap)
• SMB (and some AD) configuration (enum4linux-ng, nmap)
• SSH configuration (nmap)
• TLS configuration (nmap, sslscan, sslyze, testssl)

The following languages are currently available for the analysis:

• de: Deutsch
• en: English

The analyzer can print its results in Markdown, JSON or CSV. If you need the analysis in a markup format other than Markdown, just pipe the output of the analyzer to pandoc and you are good to go. Below is an example of a conversion to docx:

$ analyze [...] | pandoc --from=markdown --to=docx --output="/path/to/analysis.docx"