Monday, 27 October 2014

OWTF 1.0 "Lionheart": Zest support and ZAP integration

REMINDER: We just released OWTF 1.0 "Lionheart", Please try it and give us feedback!

NOTE: This blog post is a guestpost by Deep 'dscarson' Shah, who authored one of the most amazing GSoC 2014 projects this year: Zest support and ZAP integration.

And with that, a big welcome and THANK YOU to Deep!


OWASP OWTF - Zest support and ZAP integration
As part of my GSoC project , I had to integrate Mozilla Zest and OWASP ZAP into OWTF.


This image summarizes my work :


 
Lets dive into details :


    → Zest is an experimental specialized scripting language developed by the Mozilla security team and is intended to be used in web oriented security tools.
Generating Zest scripts from OWTF provides an automated mechanism to replicate exploitation of security vulnerabilities in a format that facilitates information exchange between tools such as ZAP and others which can reproduce the same vulnerabilities in their own development environment.

ZAP is an easy to use integrated penetration-testing tool for finding vulnerabilities in web applications, which has in-built functionality to run Zest Scripts.
The ZAP supports allows OWTF to export its HTTP transactions to OWASP ZAP for further analysis and fuzzing.

The features implemented are :


  • Zest script creation from single HTTP transaction
  • Zest script creation from multiple HTTP transaction (macro of requests)
  • HTTP request editing window (from which you can replay the request)
  • Zest script Console
  • “ Record a Zest script ” functionality
  • Zest script Runner
  • Forward HTTP request to ZAP


1) Zest script creation from single HTTP transaction:


    


Clicking on the button “Create a Zest Script” creates a Zest script for the given transaction in owtf_review/targets/given_target/zest directory.


2) Zest script creation from multiple HTTP transactions:


A single zest script can be created from multiple transactions of a specific target.




Clicking on the button “Create Zest Script” lets you select the transactions you want to include in zest script which resides at owtf_review/targets/given_target/zest directory.

3) HTTP Request Editing Window (Replay Function)


An editing window similar to ZAP to change/modify/edit the particular request and get response according to that.


1)Select View from the transaction you wish to edit and replay.
2)Click replay.
3)Edit the request.
4)Click Send.


4) Zest scripting console


A new window where all the target scripts and record scripts are listed and viewed.




You can go to zest console from transaction_log window by clicking ‘Zest Script Console’.

5) Record Functionality


A functionality to record the transactions while browsing the web using OWTF proxy, and creating a Zest script from the recorded transactions. Similar to record a zest script functionality in ZAP.


  1. Go to zest console and click button ‘Record a Zest Script’
  2. Now browse the web
  3. Hit ‘Stop recording’ when done



A zest script will be created in owtf_review/misc/recorded_scripts/ directory.


6) Zest Running Functionality


Runs the zest script from the zest console and displays its output there.


Click ‘Run the Zest Script’ in zest console.


7)Forward HTTP request to ZAP


A functionality to forward HTTP requests to ZAP to analyse and testing purpose.




Just click ‘Forward to ZAP’ in particular transaction window. (Make sure ZAP is running).

Resources :


Project wiki pages (with diagrams)


Documentation of other project (zest-owtf):


A video showing all the features of this GSoC project :

Friday, 17 October 2014

OWTF 1.0 "Lionheart": UI and Database

REMINDER: We just released OWTF 1.0 "Lionheart", Please try it and give us feedback!

NOTE: This blog post is a guestpost by Bharadwaj 'tunnelshade' Machiraju, who devised and implemented the UI and Database idea from conception to implementation, full props to you, my friend :)

And with that, a big welcome and THANK YOU to Bharadwaj!


OWASP OWTF - User Interface and Database support


How OWTF used to be?



OWTF initially was a CLI program, which produced an interactive html report. Though OWTF was highly configurable, its usability was limited to huge configuration files.


console_view.png


What was done during this project?



The project had one main goal, i.e build an interface so as to control all aspects of OWTF. This involved more than a few challenges:


  • Refactoring the codebase to make use of databases.
  • Building a RESTful api to make the interface interactive.
  • Creating a web interface using the REST api.
  • Extending control on plugin execution (or worker processes as we cal it).


The technology stack which was finalized for this project:


  • Tornado (for interface and api servers)
  • Postgresql (for database)
  • SQLAlchemy (for ORM)


How does OWTF look now?



After the completion, OWTF must be launched from the command line and then everything can be done from the web interface. The following screenshots will take you through a tour, but for detailed explainations, a visit to our user docs is required (http://docs.owtf.org/en/latest/usage.html).


target_manager.png
Target Manager


plugin_launcher.png
Plugin Launcher


rated_target_report.png
Target Report
plugin_report.png
Plugin Report


Screen Shot 2014-09-16 at 4.38.27 am.png
Transaction Log


worker_manager_1.png
Worker Manager


worklist_manager_2.png

Worklist Manager


Resources!





Wednesday, 15 October 2014

OWTF 1.0 "Lionheart": Brucon 5x5 video, slides and more

REMINDER: We just released OWTF 1.0 "Lionheart", Please try it and give us feedback!

Just a quick note to say that the materials used by the OWTF Crew during the Brucon 5x5 presentations are now online:

Slides here:
https://speakerdeck.com/tunnelshade/brucon-2014-5by5-owasp-owtf

OWTF 1.0 "Lionheart" - Brucon 5x5 Video:



Talk structure and higher resolution demos

    1. (From minute: 0) Introduction to OWTF and discussion of the Web UI, REST API and DB by Bharadwaj Machiraju (higher resolution: OWTF UI Demo)
    2. (From minute: ~15:50) Discussion of WAF Bypasser AND OWTF Botnet mode by Marios Kourtesis (higher resolution: WAF bypasser fuzzing DemoWAF Bypasser detect allowed characters)
    3. (From minute: ~24:50) OWTF re-architecture by Alessandro Fanio Gonz├ílez

    On top of this, Bharadwaj became the Brucon 2014 Lightening talk winner with his awesome talk about Flashbang, which he developed for Cure53. The talk is now public and you can watch it here:


    Congratulations Bharadwaj, you rock! ;)




    Friday, 10 October 2014

    OWTF 1.0 "Lionheart": Automated Rankings

    REMINDER: We just released OWTF 1.0 "Lionheart", Please try it and give us feedback!

    NOTE: This blog post is a guestpost by Tao 'depierre' Sauvage, who authored one of the most successful GSoC 2014 projects for OWASP OWTF this year: OWASP OWTF: Automated Rankings

    Helicopter view:
    Ever had to test 30 URLs in 5 days and wondered where to start? OWTF will now take the MAX severity from ALL tools run against EACH target and tell you where :)


    And with that, a big THANK YOU and welcome to Tao! :)

    Introduction


    Thanks to GSoC, I had the opportunity to work on the OWASP - OWTF project. My task consisted in implementing an automated ranking system but first of all, let us have a quick overview of OWTF.

    As you surely know, because you are reading this blog, OWTF is a framework that helps user -- might be a security expert as well as an unsavvy but curious person -- in security assessments. It will take care of the unpleasant part of the job and automatically generates an interactive report containing all the information for the selected plugins.

    The powerful feature here is the interactive report. In a few words, instead of having a report that cannot be modified (like Skipfish or W3AF for instance), OWTF will take into account the user's actions. For instance, it might be interesting to add a screenshot for the plugin XY to clearly show the SQL injection that was found, and, you can!

    no_ranking.png


    But before GSoC began, the user had to manually evaluate the security risks for each plugin. If you had an assessment concerning 30 different websites it would take a lot of times. Therefore, the need showed up by itself: OWTF needed to pre-evaluate the security risks of its plugins.

    The ranking system


    By the end of GSoC, the automated ranking system has been completed and integrated to OWTF, a good news for its users.

    It has been developed following the simple rules below:

    1. OWTF's ranking scale would be Unranked/No risk, Informational, Low , Medium, High and Critical risk (6 different values).
    2. OWTF cannot automatically rank the outputs higher than high.
    3. The automated rankings will be highlighted as such.
    4. The user will be able to confirm/override the ranking.
    5. If the ranking has been confirmed/overridden, the highlight is removed.



    The first rule is kind of obvious, it is based on the most common scales that are used by any security tools.

    rankings.png

    6 available rankings


    The second rule is more interesting. We, OWASP contributors, decided that only the user is able to correctly estimate a critical risk because they are the only one aware of all parameters, such as the application context.



    mixed_risks2.png
    Highlight of the Critical ranking
    Let's say that tool XY found an SQL injection on target AB. According to most of vulnerability scores, such discovery is estimated really hazardous. On the other hand, OWTF cannot know what the database contains and if its information is critical or not. Therefore, instead of yelling the critical big red flag, it ranks the discovery as high and let the user decide whether it deserve a higher or lower ranking.

    zoom_medium.png
    Difference between an automated ranking and a confirmed one

    The three last rules have been chosen according to OWTF's philosophy, which is to have interactive reports. Therefore the automated ranking system would have its rankings more transparent than the ones of the user. That way, a quick view of the report will allow them to see what has been found and who found it.mixed_risks3.png
    Example of OWTF with its new automated ranking system

    As a visual result, OWTF’s automated ranking system will save a lot of time for the users. On the report above, a fraction of second is needed to understand what security aspect should be reviewed first.

    Current development state


    At the time of writing, OWTF's automated ranking system supports a couple of plugins and can rank at most 103 different ones. The following table describes which plugins are ranked:
    Supported tools
    Number of possible corresponding plugins
    Arachni
    1 plugin (OWTF-WVS-001)
    DirBuster
    12 plugins
    Metasploit
    85 plugins
    OWASP
    1 plugin (OWTF-CM-008)
    robots.txt
    1 plugin (OWTF-IG-001)
    Skipfish
    1 plugin (OWTF-WVS-006)
    W3AF
    1 plugin (OWTF-WVS-004)
    Wapiti
    1 plugin (OWTF-WVS-003)
    Supported plugins that can be ranked by OWTF

    Even though 103 seems a big number, a lot of plugins still has to be supported. The automated ranking system is in its early-stage development and will keep growing time after time in order to support each and every OWTF's plugins.

    Reusable


    During the development phase, I decided to export the automated ranking system in a standalone library that I baptized ptp (Pentester's Tools Parser). It means that OWTF's ranking system is reusable by anyone and to be honest, it is quite easy to embed ptp in your own project.
    If you are curious about developing a similar tool as OWTF but you don't want to bother ranking the discoveries, then have a look at ptp's documentation (linked in the resources section).

    Resources


    Before I let you go back to your activities, here are some useful links that would give you more information on the topic:


    If you made it this far, don't miss out Tao 'depierre' Sauvage's personal blog here!

    Tuesday, 7 October 2014

    Deadline=Oct 15th: Get credits, help OWASP, meet OWASP Winter Code Sprint, plz RT!

    NOTE: OWTF 1.0 "Lionheart" has been released!, please help us get it straight reporting bugs here.

    There is an awesome OWASP initiative this winter called OWASP Winter Code Sprint (OWCS).

    In essence, if you are a university student this lets you to contribute to a participating OWASP project in exchange for university credits and other perks.

    The full list of participating OWASP projects and ideas can be found here.

    Important deadlines:
    Please have a look at the OWCS page for more information and apply for your favourite project today! :).


    If any of the following seem interesting to you, please contact me ASAP on name.surname@owasp.org -minor thinking required ;)-
    NOTE: Yes, these are the ideas that are FREE for the 2nd round! :)

    Sunday, 5 October 2014

    OWTF 1.0 "Lionheart" released!

    UPDATE: 10/10/2014 - Added link to OWTF 1.0 "Lionheart": Automated Rankings

    OWTF 1.0 "Lionheart" (beta) is dedicated to everybody that helped make this challenging release happen, in particular to the courage of all this people, who overcame their sweat, blood and tears to make OWTF the amazing tool it is now, to all of you, thank you!

    We would like to take this opportunity to thank all contributors, mentors, everybody who sent us cool ideas, feedback or reported bugs, you all played a role in making OWTF be what is is now, many thanks to you all!.

    Some links:

    OWTF 1.0 “Lionheart” (beta) is our biggest release ever, this contains many cool projects implemented by many , so, in no particular order, here is a quick overview of the new major features! :)


    NOTE: For a more detailed explanation please see OWTF 1.0 "Lionheart": Automated Rankings

    When testing a bunch of web applications, OWTF will append the ranking associated with each target for an easy quick overview.



    More information can be retrieved when accessing to one specific target report. Here, OWTF uses labels and color codes to help the user reviewing the results.mixed_risks3.png

    Sexy Web UI + REST API + Database by Bharadwaj Machiraju






  • OWTF now features a Web UI from where you can run scans, plugins, etc.
  • RESTful API to send commands to OWTF!
  • Highly performant postgres database






  • Interactively send HTTP requests to a target
  • Record and play Zest scripts
  • Pass interesting HTTP requests to ZAP using the ZAP API



  • WAF bypasser by Marios Kourtesis

    Detecting blocked characters and searching for bypasses:
    Fuzzing with XSS payloads:



    OWASP OWTF - State Improvements by Viyat Bhalodia








    Improved Plug-n-hack support by Punga Cornel

    Plug-n-Hack is a proposed Mozilla standard.
    Now OWTF supports Plug-n-Hackv2, which allows you to intercept, change and fuzz client side messages!


    configuration page (nightly version).png


    configuration commands to injected probe.png



    monitor events message.png


    Online passive scanner, Boilerplate templates and Flexible Mappings by Anirudh Anand



    • Online Boilerplate templates: Easy to copy-paste stuff into your pentest reports (i.e. mitigation, etc.)
    • Passive Online scanner: Try some of the OWTF passive tests without installing anything! :)
    • Flexible mappings: OWTF will now let you view plugins by OWASP Top 10, OWASP Testing Guide v4, OWASP Testing Guide v3 and even NIST! :)

    demo.jpg