Integrating E2E and Application Security Testing: HOWTo with NightwatchJS and OWASP ZAP

  • Powerful front-end JavaScript frameworks like Angular/React/Vue/Mojo2, etc that are often Single Page Apps, and generate links and other content on the fly, based on events and user actions. These frameworks have made it very easy for a user to interact with a web application, making them highly responsive and user-friendly, but have made it notoriously difficult for spiders to identify content, forms and parameters, thereby reducing the spider’s result to a “nothingburger”.
  • A lot of the apps we test are web services or micro-services today. With the rise of mobile apps and multiple client requirements, developers seldom create “monolithic” web applications any more, but rather prefer a modular approach to creating a web service that has the ability to work with multiple clients, and other applications as API. Clearly, spiders are completely useless here as there’s no content to discover in the first place.
  1. The Test runs against the application, with the DAST scanner as the proxy.
  2. Once the test completes the scan process, the DAST Scan starts scanning for security vulnerabilities
  3. Once the scan is done, the DAST scanner generates/publishes reports/artifacts that can be used by engineering teams to triage and fix the identified vulnerabilities.
  • The entire test has valid parameters, urls and modules that are being used by the e2e test. Therefore, this can be used against even complex front-end heavy apps, web services, etc. This makes the “coverage” aspect of a security test more specific, with all the right parameters and inputs. This makes the test more effective, and the possibility of finding more security vulnerabilities, more probable.
  • The entire test can be run in an Automated manner, thereby making it a great candidate for DevOps integration. Once this E2E test is available, and ready to use, it becomes a matter of running and re-running the security test against the application
  • Security Testing becomes more modular — You can choose to test only specific modules of the app that are captured by the E2E test. Let’s say you want to test the Patient Management Functionality of the application, you can do so, without having to configure the scanner’s crawler from identifying all kinds of URLs that may even be out of scope.
  • NightwatchJS (an E2E Testing Framework for NodeJS that is meant to facilitate “Browser Automation Testing”. It uses Selenium under the hood =>
  • Selenium Server
  • ChromeDriver => I felt that Chrome was far faster and more reliable than Firefox for Selenium Tests recently.
  • Export Report => Its a ZAP Add-on that I use to generate my JSON report
  • An intentionally vulnerable web application called “we care” that was developed by us at we45. Its available as a Docker container =>
  • ZAP JSON-RPC Service. A Small JSON-RPC service that I created that you can use to interact with ZAP’s API. =>
  • OWASP ZAP is probably one of the best tools that you can use for integration into an automated pipeline. Its API is extremely powerful and allows the user to control even the smallest operational aspect of ZAP. Highly recommended for this reason. ZAP also has a host of other benefits including some really powerful Add-ons etc
  • Writing the End-to-End test in NightwatchJS was a breeze. But there were several issues with the other things (see “The Bad”). Nightwatch also provides Test Hooks. These are events that can be fired before the test, after the test and before and after each test. I have used test hooks (before and after) extensively in this example
  • (Shameless Plug) I am glad I wrote my minimalistic ZAP JSON RPC service. While ZAP does have a REST API, its painful to use REST endpoints over more intuitive “functions” that a JSON-RPC service allows you to use.
  • Nightwatch is NodeJS, which is JavaScript, which is Asynchronous. There were several times that this led me to nearly tear my hair out in frustration. Async events tend to be a little alien to Python devs like myself. Also, with Testing, you like things to run in sequence. Node and JS make you work for it. You’d see evidence of my frustration in the before and after hooks, where I have used “hacky” solutions to stitch things together.
  • OWASP ZAP Javascript API is not very usable. I had to make up for it with the JSON-RPC service. It all worked out in the end 🙂
  • Firefox and Geckodriver. I had initially envisioned using Firefox and Geckodriver for the test. But the proxy experience was so terrible and Geckodriver was so slow, I decided to switch the Chrome. It was far far more consistent, and wayyy faster.
Chrome Proxy Settings in nightwatch.conf.js
Other Settings that capture ZAP Settings
‘before’ hook with ZAP Start
NightwatchJS Tests — Authenticate and Browse another page
‘after’ test hook — Start ZAP Scan, Print status, Export Report and Shutdown ZAP
  • I make this recommendation to all my clients. And I maintain it. Never write massive tests. They result in scans running for hours. Smaller tests have smaller scan runtimes, hence more focused results.
  • Optimize ZAP’s Scan Policy to suit your application. I highly recommend that you don’t run the default, especially with plugins like Active Scan ++, etc enabled. I have used a scan policy called “Light” (in this example) which runs a low intensity set of checks for only a few vulnerabilities. You should tailor your scan policy to your application. For instance an Active Scan check for “Trace axd Information Disclosure” would probably be useless against a NodeJS app as its only meant to capture flaws with a ASP.NET application. Or similarly, checking for SQL Injection with SQLite doesnt make any sense when your DB is MySQL.
  • If you are testing HTTPS sites, you should consider installing ZAP’s certificate into the browser that you are planning on using.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Abhay Bhargav

Abhay Bhargav

CTO of we45 (An AppSec Company), DevSecOps Greasemonkey, Passionate Security Technologist and Creator