Merge pull request #4 from shogunlab/fuzzy_detection

Fuzzy detection of partial XSS reflections enabled, new detect_xss() method, docstrings

Author: shogunlab

.gitignore

@@ -7,3 +7,5 @@ ghostdriver.log
 /payloads/*
 /logs/*
 /screenshots/*
+
+.vscode

README.md

@@ -17,8 +17,9 @@ Shuriken works with [Python](http://www.python.org/download/) **2.7.x** on any p
 ## Features
 - Easily specify where in a URL the payload should be injected with the "{xss}" string.
 - Quickly change payload lists.
-- Take screenshots of successful XSS hits.
+- Take screenshots of successful XSS payloads.
 - Save logs of reflected XSS payloads.
+- Use fuzzy detection to log partial XSS reflections.
 
 ## Usage
 To get a list of options and switches, enter:
@@ -29,34 +30,60 @@ To test a list of payloads against a target URL, specify where the payloads will
 
 `python shuriken_xss.py -u "http://example.com/target.php?name={xss}" -p "xss-payload-list.txt"`
 
-If you would like to screenshot and save all reflected XSS payloads, use the *--screen* flag with a name for the screenshot images and enter:
+### Taking screenshots
 
-`python shuriken_xss.py -u "http://example.com/target.php?name={xss}" -p "xss-payload-list.txt" --screen ExampleTarget`
+If you would like to screenshot and save all reflected XSS payloads, use the *-s* or *--screen* flag with a name for the screenshot images and enter:
+
+`python shuriken_xss.py -s ExampleTarget -u "http://example.com/target.php?name={xss}" -p "xss-payload-list.txt"`
 
 To wait a specific amount of time in between requests, use the *-t* flag with the amount of time to wait in seconds and enter:
 
-`python shuriken_xss.py -u "http://example.com/target.php?name={xss}" -p "xss-payload-list.txt" -t 1.5`
+`python shuriken_xss.py -t 1.5 -u "http://example.com/target.php?name={xss}" -p "xss-payload-list.txt"`
+
+### Fuzzy XSS payload detection
+
+To enable partial or fuzzy detection of XSS payloads in HTML source code, use the *-f* or *--fuzzy* flag with the level of detection you want to log. For example, the following command will only log XSS payload reflections that have a 75% matching score or above in the HTML source code returned:
+
+`python shuriken_xss.py -f 75 -u "http://example.com/target.php?name={xss}" -p "xss-payload-list.txt"`
+
+The default matching score supplied is 50% and will be applied when a flag with no number is given (e.g. -f or --fuzzy): 
+
+`python shuriken_xss.py -f -u "http://example.com/target.php?name={xss}" -p "xss-payload-list.txt"`
+
+Partial detection is applied through the use of SeatGeek's [FuzzyWuzzy](https://github.com/seatgeek/fuzzywuzzy) Python library `token_set_ratio()` method and additional information regarding this library can be found [here](http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/). 
+
+Partial XSS reflections will be logged in a separate text file ending with "_partials.txt".
+
+### Misc. usage and performance notes
 
 **You must specify a payload and URL**, if you don't then you'll get an error. For an example payload to test with, check out this list of [common XSS payloads](https://github.com/foospidy/payloads/blob/master/owasp/fuzzing_code_database/xss/common.txt).
 
+You also must have PhantomJS installed and configured in order for the tool to run in its default mode. See the next section for more details on this.
+
+**There may be a noticeable slowdown of the tool when it is being used in a virtual machine such as VirtualBox.** For best performance, use Shuriken on a native machine. I am currently looking to address this virtual machine slowdown in a future update.
+
 ## Third party libraries and dependencies
 This tool depends on the proper configuration and installation of the following:
 - [Python 2.7.x](https://www.python.org/downloads/) - Python 2 is needed to run the tool.
 - [Splinter](https://splinter.readthedocs.io/en/latest/install.html) - Python library allowing use of a headless web browser for testing.
 - [PhantomJS](http://phantomjs.org/download.html) - Headless WebKit browser used by Splinter for testing.
 - [Selenium 2.0](http://www.seleniumhq.org/docs/03_webdriver.jsp) - WebDriver required by PhantomJS browser.
+- [FuzzyWuzzy](https://github.com/seatgeek/fuzzywuzzy) - Partial XSS logging using fuzzy detection methods.
+- [python-Levenshtein](https://pypi.python.org/pypi/python-Levenshtein/0.12.0) - Python extension for computing string edit distances and similarities. Allows faster fuzzy detection from the FuzzyWuzzy library.
 
 Python dependencies can be installed using pip: `pip install -r requirements.txt`. Use your platform-specific mechanism to install PhatomJS (e.g. `brew` on OSX, `apt-get` on Debian or Ubuntu, etc). 
 
 If you would prefer that this tool ***use a different browser for testing***, you can read the [Splinter docs](https://splinter.readthedocs.io/en/latest/#drivers) and insert your preferred browser in the "inject_payload" method where it says `browser = Browser("phantomjs")`. Leaving it blank as `browser = Browser()` will default to Firefox.
 
 ## Screenshots
 **Basic usage**
-![screen_1](https://i.imgur.com/91dGSfS.png "Shuriken Screenshot #1")
+![screen_1](https://i.imgur.com/LzPpTsh.png "Shuriken Screenshot #1")
 
+**With *-s* or *--screen* option to record screenshots and *-t* option to delay requests by specific amount**
+![screen_2](https://i.imgur.com/EjYdAGD.png "Shuriken Screenshot #2")
 
-**With *--screen* option to record screenshots and *-t* option to delay requests by specific amount**
-![screen_2](https://i.imgur.com/md9j82N.png "Shuriken Screenshot #2")
+**With *-f* or *--fuzzy* option to fuzzy detect and log partial XSS payload reflections**
+![screen_3](https://i.imgur.com/9lwcFxl.png "Shuriken Screenshot #3")
 
 ## Legal
 Shuriken was derived from the excellent XSS command line tool by Faizan Ahmad, called [XssPy](https://github.com/faizann24/XssPy). The Shuriken XSS tool is under an MIT license, you can read it [here](https://github.com/shogunlab/shuriken/blob/master/LICENSE.md).

requirements.txt

@@ -1,2 +1,4 @@
 splinter==0.7.5
 selenium==3.4.3
+fuzzywuzzy==0.15.1
+python-Levenshtein==0.12.0

shuriken_xss.py

@@ -1,4 +1,5 @@
-#!/usr/bin/env python
+# !/usr/bin/env python
+"""Set Python environment."""
 # -*- coding: utf-8 -*-
 
 import argparse
@@ -9,10 +10,12 @@
 import time
 
 from splinter import Browser
+from fuzzywuzzy import fuzz
 
 
 class Color:
-    # Use colors to make command line output prettier
+    """Use colors to make command line output prettier."""
+
     RED = '\033[91m'
     GREEN = '\033[92m'
     YELLOW = '\033[93m'
@@ -21,22 +24,31 @@ class Color:
 
 
 class Shuriken:
+    """Object for testing lists of XSS payloads."""
 
     def __init__(self):
+        """Initiate object with default encoding and other required data."""
         # Expect some weird characters from fuzz lists, make encoding UTF-8
         reload(sys)
         sys.setdefaultencoding('utf8')
 
         # All potential XSS findings
         self.xss_links = []
 
+        # Fuzzy string matching partials list
+        self.xss_partials = []
+
+        # Keep index of screens for log files
+        self.screen_index = ""
+
         # Get user args and store
         self.user_args = self.parse_args()
 
         # PhantomJS browser
         self.browser = Browser("phantomjs")
 
     def main(self):
+        """Call functions to inject XSS and test for vulnerabilities."""
         # Print out a welcome message
         print "\n"
         print "=" * 34 + Color.YELLOW + "\nWelcome to the" + Color.RED + \
@@ -49,11 +61,16 @@ def main(self):
                       self.user_args.SCREENSHOT_NAME)
         print Color.GREEN + "\n=== Testing complete! ===\n" + Color.END
 
-        # If the test found possible XSS vulnerabilities, ask if we should
-        # log
+        # If the test found possible XSS vulnerabilities, ask if we should log
         if self.xss_links:
             print Color.YELLOW + \
-                "Potential XSS vulnerabilities were detected!" + \
+                "XSS vulnerabilities were detected!" + \
+                Color.END
+            self.log_file(self.xss_links)
+        # There were partials detected by fuzzy detection
+        elif self.xss_partials:
+            print Color.YELLOW + \
+                "Partial XSS vulnerabilities were detected!" + \
                 Color.END
             self.log_file(self.xss_links)
         else:
@@ -64,14 +81,16 @@ def main(self):
             print "\n"
 
     def make_sure_path_exists(self, path):
+        """Ensure that file path exists before writing."""
         try:
             os.makedirs(path)
         except OSError as exception:
             if exception.errno != errno.EEXIST:
                 raise
 
-    def inject_payload(self, payload, link, request_delay, screenshot_target):
-        # Visit user supplied link with injected payload
+    def inject_payload(self, payload, link, request_delay,
+                       user_screenshot_name):
+        """Inject XSS payload string from user supplied payload list."""
         browser = self.browser
 
         # Let user specify where in the URL fuzz values should be injected
@@ -95,31 +114,69 @@ def inject_payload(self, payload, link, request_delay, screenshot_target):
         # linked to line nums in log
         self.screen_index = str(len(self.xss_links) + 1)
 
-        # Check to see if payload was reflected in HTML source
-        if payload in browser.html:
-            print Color.GREEN + "\n[+] Potential XSS vulnerability found:" + \
+        # Check to see if payload was reflected in HTML source,
+        # if so, take screenshot depending on user flag
+        self.detect_xss(payload, browser, user_screenshot_name, injected_link)
+
+    def detect_xss(self, payload, browser_object, user_screenshot_name,
+                   injected_link):
+        """Check the HTML source to determine if XSS payload was reflected."""
+        # If fuzzy detection chosen, evaluate partial reflection of XSS
+        # by tokenizing the HTML source and detecting parts of the payload
+        # and source common to both.
+        #
+        # Other methods of scoring include fuzz.ratio(), fuzz.partial_ratio()
+        # and fuzz.token_sort_ratio()
+        partial_score = fuzz.token_set_ratio(
+            payload.lower(), browser_object.html.lower())
+        # Set the level of detection asked for by the user, e.g. Only detect
+        # matches with score higher than 50% fuzzy detection
+        fuzzy_level = self.user_args.FUZZY_DETECTION
+
+        if payload.lower() in browser_object.html.lower():
+            print Color.GREEN + "\n[+] XSS vulnerability found:" + \
                 Color.END
-            # If user set the --screen flag to target, capture screen of
+
+            # If user set the --screen flag to target, capture screenshot of
             # payload
-            if screenshot_target is not None:
-                # Check if screenshots directory exists, if not then create it
-                self.make_sure_path_exists("screenshots")
-                screenshot_file_name = "screenshots/" + \
-                    screenshot_target + "_" + \
-                    datetime.datetime.now().strftime("%Y%m%d-%H%M%S") + \
-                    "_" + self.screen_index + ".png"
-                # Save screenshot to directory
-                browser.driver.save_screenshot(screenshot_file_name)
-                print Color.YELLOW + "Screenshot saved: " + \
-                    screenshot_file_name + Color.END
+            if user_screenshot_name is not None:
+                self.take_screenshot(user_screenshot_name,
+                                     browser_object, self.screen_index)
+
             # Add link to list of all positive XSS hits
             self.xss_links.append(injected_link)
-            return Color.BLUE + injected_link + Color.END
+            print Color.BLUE + injected_link + Color.END
+        # If user enabled fuzzy detection and partial score was larger than
+        # fuzz level, add it to partials list and print results
+        elif fuzzy_level and (partial_score >= fuzzy_level):
+            print Color.YELLOW + \
+                "\n[-] Partial XSS vulnerability found:" + Color.END
+            print Color.BLUE + injected_link + Color.END
+            self.xss_partials.append(injected_link)
+            print "Detection score: %s" % partial_score
         else:
-            return Color.YELLOW + "\n[+] Tested, but no XSS found at: \n" + \
-                Color.RED + injected_link + Color.END
+            print Color.RED + "\n[+] No XSS detected at: \n" + \
+                Color.BLUE + injected_link + Color.END
+            if (fuzzy_level):
+                print "Detection score: %s" % partial_score
+
+    def take_screenshot(self, user_screenshot_name, browser_object,
+                        screen_index):
+        """Take a screenshot of the page in the browser object."""
+        # Check if screenshots directory exists, if not then create it
+        self.make_sure_path_exists("screenshots")
+        screenshot_file_name = "screenshots/" + \
+            user_screenshot_name + "_" + \
+            datetime.datetime.now().strftime("%Y%m%d-%H%M%S") + \
+            "_" + screen_index + ".png"
+        # Save screenshot to directory
+        browser_object.driver.save_screenshot(screenshot_file_name)
+        print Color.YELLOW + "Screenshot saved: " + \
+            screenshot_file_name + Color.END
 
-    def test_xss(self, payloads_param, link, request_delay, screenshot_target):
+    def test_xss(self, payloads_param, link, request_delay,
+                 user_screenshot_name):
+        """Load string from payload list and call function to inject it."""
         # If the user added time delay, show them what it is set to.
         if request_delay is not None:
             print Color.YELLOW + "\n[!] Request delay is set to [" + \
@@ -128,34 +185,48 @@ def test_xss(self, payloads_param, link, request_delay, screenshot_target):
         # Load the payload file and inject all payloads
         # into user supplied URL to test for XSS
         payloads = []
-        with open(payloads_param) as file:
-            for line in file:
+        with open(payloads_param) as payload_file:
+            for line in payload_file:
                 line = line.strip()
                 payloads.append(line)
         for item in payloads:
-            print self.inject_payload(item, link, request_delay,
-                                      screenshot_target)
+            self.inject_payload(item, link, request_delay,
+                                user_screenshot_name)
 
     def log_file(self, link_list):
+        """Log successful XSS payload reflections to file."""
         # Prompt the user to confirm log file, if yes, log XSS hits
         log_confirm = raw_input(
             "\nWould you like to save these results? [y/n] > ")
         if log_confirm == "y":
             target_name = raw_input("Please enter the target name > ")
             # Check if logs directory exists, if not then create it
             self.make_sure_path_exists("logs")
-            # Save log file to directory
+            # Set file name
             file_name = "logs/" + target_name + "_" + \
-                datetime.datetime.now().strftime("%Y%m%d-%H%M%S") + \
-                ".txt"
-            with open(file_name, 'w') as file:
-                for link in link_list:
-                    file.write(link)
-                    file.write("\n")
-                # Add metadata about what payload file was used
-                file.write("\n*** Created from the payload file >>> " +
-                           self.user_args.PAYLOADS_LIST)
-                file.close()
+                datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
+            # Save log file to directory if XSS links is populated
+            if self.xss_links:
+                with open(file_name + ".txt", 'w') as link_file:
+                    for link in link_list:
+                        link_file.write(link)
+                        link_file.write("\n")
+                    # Add metadata about what payload file was used
+                    link_file.write(
+                        "\n*** Created from the payload file >>> " +
+                        self.user_args.PAYLOADS_LIST)
+                    link_file.close()
+            # If the user enabled fuzzy detection, log partial results
+            if (self.user_args.FUZZY_DETECTION):
+                with open(file_name + "_partials.txt", 'w') as partial_file:
+                    for link in self.xss_partials:
+                        partial_file.write(link)
+                        partial_file.write("\n")
+                    # Add metadata about what payload file was used
+                    partial_file.write(
+                        "\n*** Created from the payload file >>> " +
+                        self.user_args.PAYLOADS_LIST)
+                    partial_file.close()
             print "\nFile successfully saved as: " + \
                 Color.BLUE + file_name + Color.END
             print "\n"
@@ -164,20 +235,24 @@ def log_file(self, link_list):
             print "\n"
 
     def parse_args(self):
-        # Parse arguments from the user sent on command line
+        """Parse arguments from the user sent on command line."""
         parser = argparse.ArgumentParser()
         parser.add_argument(
             '-u', action='store', dest='URL',
-            help='The URL to analyze', required=True)
+            help='The URL to inject XSS payloads into.', required=True)
         parser.add_argument(
             '-p', action='store', dest='PAYLOADS_LIST',
-            help='The payload list to use', required=True)
+            help='The payload list to use for injection.', required=True)
         parser.add_argument(
             '-t', action='store', dest='REQUEST_DELAY',
-            help='Amount of time in seconds to delay between requests')
+            help='Amount of time (in seconds) to delay between requests.')
+        parser.add_argument(
+            '-s', '--screen', action='store', dest='SCREENSHOT_NAME',
+            help='Enable screenshots of XSS hits.')
         parser.add_argument(
-            '--screen', action='store', dest='SCREENSHOT_NAME',
-            help='Screens of target')
+            '-f', '--fuzzy', action='store', dest='FUZZY_DETECTION', type=int,
+            const=50, nargs="?",
+            help='Fuzzy detection rate of XSS [0 to 100 match] (default=50).')
 
         arguments = parser.parse_args()