Saturday, 7 May 2016

OWTF 2.0a "Tikka Masala" released, plz RT!

It has been a long wait, but finally, OWTF 2.0a "Tikka Masala" is here!
Although partly a tribute to delicious Indian food, this release is especially dedicated to all those hard working Indian contributors who have continously demonstrated their passion, professionalism, brainpower and incredible performance, without which OWTF would not be the awesome tool it is today. This release is named after all of you, thank you!
IMPORTANT: Migrating from 1.0.1 to 2.0a includes breaking changes and requires a complete DB clean and initialisation - use the installer and the script scripts/db_setup.sh to do that. If you are already on the develop branch , you can directly pull the latest changes.
Therefore, if you are coming from an old OWTF version, please run the following commands after downloading OWTF 2.0:
WARNING: This will delete everything in your OWTF database!
  1. bash scripts/db_setup.sh clean
  2. bash scripts/db_setup.sh init

New to OWTF? No problem!

Get it here :)

Release Notes

This release includes many new features and countless bug fixes. This release would not have been possible without the help of a number of pre-GSoC contributors, mentors, and everybody who sent us cool ideas, feedback or reported bugs. In particular, this release is dedicated to our Indian contributors without whom this release would not have been possible. As a wrapper tool that depends on many tools, migration from Kali 1.x to Kali 2.x was a little bumpy: this road saw more bugs fixes/reports from new contributors and users, occasional feature requests and countless fixes (that were long due) which made this release possible.
Important Features and fixes
  • Kali 2.x support
  • Functional tests suite included => build passing(!)
  • Progress bar added to the web interface
  • HTTPrint signatures updated
  • Updated CMS Explorer lists
  • Minimal auxiliary plugin support added back
  • SSL Labs API integration
  • Resolves SQLAlchemy deadlock and improved proxy handling
  • Fixes all Metasploit plugin functionality
  • General UI improvements
  • CWE and OWASP Top 10 mappings
  • Improved worker UI controls = adds Pause All, Resume All functionality
  • Supports Debian-based distributions
  • Target manager UI improvements = bulk delete/remove

Implemented enhancements:
  • xxx_testgroups.cfg should be moved to /profiles #670
  • OWTF takes few steps to start #638
  • Session Modal breaks for large session names #635
  • Check for tools before running commands #632
  • Adding Issue and Pull Request templates #599
  • Debian and Samurai install scripts are not executable. #573
  • Increase readability of manual installation output on terminal. #564
  • Installer Issues #534
  • Passive google searches should use @@@domain@@@ instead of @@@host_path@@@ #529
  • Increase proxy CA security #526
  • Add https://censys.io/ to the passive search #523
  • install/install.py skip sudo password #519
  • Using a remote server #510
  • potential command to add to the install scripts (develop branch) #473
  • Timestamps not present in transaction log #472
  • Evaluate the possible implementation of JS templating for all client-side OWTF interactions #467
  • External XSS plugin resource: XSS Payloads #466
  • What is the hurdle in doing passive scan's #464
  • Rank should collapse the plugin, at least in some cases #459
  • Suggested improvements for the transaction log #458
  • Integration with punk spider for passive tests #457
  • Clean up colours from various tools prior to saving it in a file #456
  • Export targets feature (UI) #454
  • Lack of filters on target page (UI) #453
  • Improve curl commands #446
  • CPU spikes: Lack of Indexing on OWTF db? #444
  • Add “Pause All / Resume All” to the worker monitoring #440
  • Review OWTF CPU usage post-DirBuster #437
  • Smarter Runner #430
  • Unable to “delete all” from worklist on UI #427
  • OWTF should check if postgresql client is installed as well #413
  • External Command Injection plugin link #412
  • Mobile responsive #406
  • [develop] OWTF should start NET plugins when target is an IP #375
  • ImportError: No module named backports.ssl_match_hostname #374
  • Settings > HTTP AUTH #369
  • Setup gemnasium #358
  • Worklist search boxes should not be case sensitive #355
  • Automated Bug reporter improvement #352
  • Possible improvement for the UI worker buttons #350
  • Minor intuitiveness improvements #349
  • Arachni changed from --user-agent to --http-user-agent #347
  • Ensure running postgres before running install script #337
  • Issues on Ubuntu #334
  • OWTF should check if postgres is running #311
  • [zest] Updating the zest jars #293
  • [wapiti] HTML report is not available anymore #287
  • Moving external plugin reports away from targets subreports #111
  • Check if the service that is going to be scanned speaks HTTP before launching ANY web test #108
  • filter by severity feature added #576 (saganshul)
Fixed bugs
  • PostgreSQL Fix in db_setup.sh should use SHOW config_file; #669
  • PostgreSQL Fix in db_setup.sh restarts postgresql daemon in any case #668
  • ConfigDB silently fails when default.cfg not found #666
  • Bash 'which' error in db_setup.sh script #662
  • Improper Set-Cookie header handling in proxy #582
  • Same rank cannot be given to a plugin twice #570
  • Listing plugins option (-l) not working #556
  • Plugin Filter Display not working properly #547
  • Proxy errors (silent) in logs #528
  • Workers do not pick items from worklist #527
  • Unable to open directory from browser #525
  • Error calling make_dirs when a long URL is passed #521
  • [develop] plugin getting stuck stops the whole scan... #515
  • Getting error while running plugins. Error "Oops! Server replied: Bad Request" #481
  • The grep stats for header matche percent are incorrect #470
  • UI doesn't cope with multiple simultaneous tabs / actions? :P #455
  • CPU spikes: Lack of Indexing on OWTF db? #444
  • Bug - "Ops unable to add some targets” #443
  • BUG in “Testing For Ssl-Tls” plugin in latest Kali #442
  • Directory Brute-forcing should be towards the end of the scan #441
  • postgres “idle in transaction” processes occasionally spike CPU usage #438
  • Ocassional Crash after running skipfish #435
  • Occassional failure to close children processes #434
  • Target shuffling #433
  • Bug in MiTM proxy Cookie parser #428
  • Unreasonable use of CPU/memory by postgres / owtf processes #426
  • Nikto plugin not realising when nikto has finished #422
  • bootstrap.sh Fails while Installing in Kali #416
  • ValueError when OWTF is run without postgresql properly configured #414
  • OWTF should check if postgresql client is installed as well #413
  • Add target UI issue #405
  • OWTF-DV-004 semi passive no output #404
  • Transaction Logger Bug #403
  • Adding a Target Issue #402
  • [develop] User overriding the 2nd plugin of a test case to Passing won't update the test case #400
  • Create Zest Script Error #383

      Tuesday, 4 November 2014

      OWTF 1.0 "Lionheart": OWTF's WAF bypasser!

      REMINDER: We just released OWTF 1.0 "Lionheart", Please try it and give us feedback!

      NOTE: This blog post is a guestpost by Marios Kourtesis, who authored one of the sexiest GSoC 2014 projects this year: WAF Bypasser. An epic joint venture between two OWASP projects: OWASP ByWaf and OWASP OWTF.

      NOTE: WAF Bypasser is a tool that can be used both from inside of OWTF as well as a standalone tool, you will keep seeing this theme in upcoming cool projects ;)

      And with that, a big welcome and THANK YOU to Marios!


      OWTF WAF-Bypasser


      Abstract:
      Web Application Security is moving to another level. Hardware and software implementations of application firewalls are taking place to secure the Web infrastructure. These technologies are putting effort on delivering more secure Web Applications, unfortunately what they are really doing is too far away from what they are promising. There are many cases that WAF are increasing the attacking surface by introducing new vulnerabilities. WAFs can be easily bypassed and expose systems under the secure feeling of the defenders.
      WAF-Bypasser is a tool that assists the penetration testers to test the quality of a WAF(Web Application Firewall) or a poor written WAF rule and potentially bypass it. It is developed as an auxiliary OWASP OWTF plugin and standalone project. It is important to pinpoint that during the research and development some zero day exploits were found.[1]


      Purpose:
      In this article I am demonstrating how to analyze and bypass mod_security WAF protected with OWASP CRS Version 2.2.8. The exploit has been tested and working at version 2.2.9 as well.
      For the demonstration I will use DigitalOcean’s configuration and examples[2](see the article in the references, is important to check it in order to follow this post). My target is a fully updated Ubuntu system with the latest package of modsecurity installed.
      The problem:
      Ubuntu mod_security package contains OWASP CRS Version 2.2.8.


      Step 1)
      Installation:
      WAF bypasser have tornado framework as a dependency. Install it into your system as you usually do.
      Step 2)
      Detecting the blocked characters:
      We need to specifying the following:
      • The target url.
      • The way that I will detect when a request is blocked. Mod_security responds to the blocked HTTP requests with a 403 HTTP Error Code. So in my case specify I will detect the blocked requests with the response code.
      • The post data with the variable that I want to fuzz.
      • The scanning mode for detecting the allowed characters. Additionally this mode attempts to bypass the WAF by encoding the characters and etc.

      Putting all together:


      python wafbypasser.py --target http://192.168.0.103/login.php  --data "login=1&username=@@@fuzzhere@@@"
      --response_code 403 --mode detect_chars


      In the following screenshot we can see that the “ and ‘ characters among others, are being detected by the WAF.
      (Finding detected characters)


      Scrolling down the output we can see in the next screenshot that the we can pass the ‘ (single quote) character by having in between two undetected characters.
      (undetected quotes, small bypass)
      Example:
      ‘ is detected
      a‘a is not detected
      a’ is detected
      ‘a is detected
      Note: The [a] character has been found by WAF bypasser to be an undetected character.


      Step 3)
      Exploiting the target:
      As we see from the MySQL query and the logic of this super vulnerable authentication system, the only thing we need to do is to make the query return true.
      $result = mysqli_query($con, "SELECT * FROM `users` WHERE username='$username' AND password='$password'");
             if(mysqli_num_rows($result) == 0)
                 echo 'Invalid username or password';
             else
                 echo '<h1>Logged in</h1><p>A Secret for you....</p>';


      Having in mind that the WAF will not block a detected character surrounded by undetect characters and that space character is not in list of detected characters, lets write manually some payloads will make the SQL query to return true.
      vim payloads.txt

      a' or ‘1’=’1’ or 'a
      a' or ‘ 1 ’ = ’ 1 ’ or 'a
      a' or ‘2’=’2’ or 'a
      a' or true or 'a
      a' or not false or 'a
      a' or not  ‘1’=’2’ or 'a


      Now lets fuzz the target with our payloads.
      In the fuzzing mode we need to specify the file with me payloads. In my case the payloads.txt.


      python wafbypasser.py --target http://192.168.0.103/login.php --data "login=1&username=@@@fuzzhere@@@" --response_code 403 --mode fuzz --payloads payloads.txt



      In the screenshot we see that we found 2 bypasses. In the lower part of the image, we are testing one of the undetected payloads using curl where we can see from the results, that we have successfully bypassed the WAF and do an SQL injection that allowed us to log in to our vulnerable Web Application as we was going to do without the presence of the WAF.
      (fuzzing the target)


      References:

      [2]”How to set up mod_security with apache on debian-ubuntu”, URL: [https://www.digitalocean.com/community/tutorials/how-to-set-up-mod_security-with-apache-on-debian-ubuntu]

      Demos / talks:








      Monday, 27 October 2014

      OWTF 1.0 "Lionheart": Zest support and ZAP integration

      REMINDER: We just released OWTF 1.0 "Lionheart", Please try it and give us feedback!

      NOTE: This blog post is a guestpost by Deep 'dscarson' Shah, who authored one of the most amazing GSoC 2014 projects this year: Zest support and ZAP integration.

      And with that, a big welcome and THANK YOU to Deep!


      OWASP OWTF - Zest support and ZAP integration
      As part of my GSoC project , I had to integrate Mozilla Zest and OWASP ZAP into OWTF.


      This image summarizes my work :


       
      Lets dive into details :


          → Zest is an experimental specialized scripting language developed by the Mozilla security team and is intended to be used in web oriented security tools.
      Generating Zest scripts from OWTF provides an automated mechanism to replicate exploitation of security vulnerabilities in a format that facilitates information exchange between tools such as ZAP and others which can reproduce the same vulnerabilities in their own development environment.

      ZAP is an easy to use integrated penetration-testing tool for finding vulnerabilities in web applications, which has in-built functionality to run Zest Scripts.
      The ZAP supports allows OWTF to export its HTTP transactions to OWASP ZAP for further analysis and fuzzing.

      The features implemented are :


      • Zest script creation from single HTTP transaction
      • Zest script creation from multiple HTTP transaction (macro of requests)
      • HTTP request editing window (from which you can replay the request)
      • Zest script Console
      • “ Record a Zest script ” functionality
      • Zest script Runner
      • Forward HTTP request to ZAP


      1) Zest script creation from single HTTP transaction:


          


      Clicking on the button “Create a Zest Script” creates a Zest script for the given transaction in owtf_review/targets/given_target/zest directory.


      2) Zest script creation from multiple HTTP transactions:


      A single zest script can be created from multiple transactions of a specific target.




      Clicking on the button “Create Zest Script” lets you select the transactions you want to include in zest script which resides at owtf_review/targets/given_target/zest directory.

      3) HTTP Request Editing Window (Replay Function)


      An editing window similar to ZAP to change/modify/edit the particular request and get response according to that.


      1)Select View from the transaction you wish to edit and replay.
      2)Click replay.
      3)Edit the request.
      4)Click Send.


      4) Zest scripting console


      A new window where all the target scripts and record scripts are listed and viewed.




      You can go to zest console from transaction_log window by clicking ‘Zest Script Console’.

      5) Record Functionality


      A functionality to record the transactions while browsing the web using OWTF proxy, and creating a Zest script from the recorded transactions. Similar to record a zest script functionality in ZAP.


      1. Go to zest console and click button ‘Record a Zest Script’
      2. Now browse the web
      3. Hit ‘Stop recording’ when done



      A zest script will be created in owtf_review/misc/recorded_scripts/ directory.


      6) Zest Running Functionality


      Runs the zest script from the zest console and displays its output there.


      Click ‘Run the Zest Script’ in zest console.


      7)Forward HTTP request to ZAP


      A functionality to forward HTTP requests to ZAP to analyse and testing purpose.




      Just click ‘Forward to ZAP’ in particular transaction window. (Make sure ZAP is running).

      Resources :


      Project wiki pages (with diagrams)


      Documentation of other project (zest-owtf):


      A video showing all the features of this GSoC project :

      Friday, 17 October 2014

      OWTF 1.0 "Lionheart": UI and Database

      REMINDER: We just released OWTF 1.0 "Lionheart", Please try it and give us feedback!

      NOTE: This blog post is a guestpost by Bharadwaj 'tunnelshade' Machiraju, who devised and implemented the UI and Database idea from conception to implementation, full props to you, my friend :)

      And with that, a big welcome and THANK YOU to Bharadwaj!


      OWASP OWTF - User Interface and Database support


      How OWTF used to be?



      OWTF initially was a CLI program, which produced an interactive html report. Though OWTF was highly configurable, its usability was limited to huge configuration files.


      console_view.png


      What was done during this project?



      The project had one main goal, i.e build an interface so as to control all aspects of OWTF. This involved more than a few challenges:


      • Refactoring the codebase to make use of databases.
      • Building a RESTful api to make the interface interactive.
      • Creating a web interface using the REST api.
      • Extending control on plugin execution (or worker processes as we cal it).


      The technology stack which was finalized for this project:


      • Tornado (for interface and api servers)
      • Postgresql (for database)
      • SQLAlchemy (for ORM)


      How does OWTF look now?



      After the completion, OWTF must be launched from the command line and then everything can be done from the web interface. The following screenshots will take you through a tour, but for detailed explainations, a visit to our user docs is required (http://docs.owtf.org/en/latest/usage.html).


      target_manager.png
      Target Manager


      plugin_launcher.png
      Plugin Launcher


      rated_target_report.png
      Target Report
      plugin_report.png
      Plugin Report


      Screen Shot 2014-09-16 at 4.38.27 am.png
      Transaction Log


      worker_manager_1.png
      Worker Manager


      worklist_manager_2.png

      Worklist Manager


      Resources!