Tes$ng Web Applica$ons. Willem Visser RW334

Transcription

1 Tes$ng Web Applica$ons Willem Visser RW334

2 Overview Tes$ng Theory Tes$ng Web Apps Tools

3 A test consists of The Basics Input Expected output (also called the Oracle) White Box Tes$ng Considering the code when coming up with tests Black Box Tes$ng Only consider the input- output interface Automated Tes$ng is the Holy Grail

4 Unit Tes$ng Integra$on Tes$ng Systems Tes$ng Regression Tes$ng Acceptance Tes$ng Alpha/Beta Tes$ng Usability Tes$ng Performance Tes$ng Robustness Tes$ng Security Tes$ng Types of Tes$ng

5 Lower Level Code Tes$ng Unit Tes$ng Test individual components of the code Can be as liple as one func$on White box Integra$on Tes$ng Interface between components White Box Can be as much as the whole system (which case it becomes more black box) Code Coverage as Test Adequacy Measure

6 Func$onal Tes$ng Tes$ng that the systems meets the requirements Systems Tes$ng In the deployed environment Unlike Integra$on tes$ng Always black box

7 Cross CuTng Regression Tes$ng When code changes (some) old tests must s$ll pass Could be at unit, integra$on or systems level Very high cost involved in regression tes$ng Regression failure has high cost to fix

8 Acceptance Tes$ng Customer Facing Is the customer happy with the product? Alpha/Beta Tes$ng Let the system out to a few of your customers and see how they feel about it Usability Tes$ng Typically for GUI/Web/Mobile to not just check that the system is correct but also easy to use Harder to create an Oracle

9 Non- Func$onal Tes$ng Performance Load See how the system behaves at peak load Stress Push the system beyond its limits to see how far it will survive Security Tes$ng Check whether there are vulnerabili$es that might lead to loss of privacy or other security issues

10 Web App Tes$ng Full Gambit of Features Front End GUI Usability issues Hard to test Browser Compa$bility issues Server Side (including storage) Performance issues Security issues

11 Unit Tes$ng UPerly important! Finding bugs early saves money Makes regression tes$ng much more effec$ve Write once, run o\en What do you test? Business logic! No need to test simple code or interac$ons with 3 rd party libraries Un$l they fail of course! Use Coverage tools to help you decide if you have tested enough

12 Stubs or Mocks In unit tes$ng you are interested in local behavior and assume other things you might be using behave correctly Most unit tes$ng frameworks provides stubs for these 3 rd party components Good example is datastore and memcache stubs provided by GAE Only problem is that some$mes these stubs don t respect the behavior of the real thing!

13 Unit Tes$ng Frameworks Java JUnit The most famous of them all PHP PHPUnit Python PyUnit Actually just import unipest And many more XUnit tools for language X For example GAEUnit, but it looks dead now Use coverage tools along side, examples Coverage.py for Python (Ecl)Emma for (Eclipse) Java

14 Unit Tes$ng and GUIs When doing Unit Tes$ng you try and stay well clear of the GUI Clicking and entering text through a GUI is not automated, although we will see later that with Replay Technology it can also be done Isola$ng the GUI to allow more efficient tes$ng is the reason to use Model- View- Presenter rather than Model- View- Controller design papern

15 Webapp2 example import webapp2 Code to Test class HelloHandler(webapp2.RequestHandler): def get(self): self.response.write('hello, world!') app = webapp2.wsgiapplication([('/', HelloHandler)]) import unittest import webapp2 # from the app main.py import main Test class TestHandlers(unittest.TestCase): def test_hello(self): #input request = webapp2.request.blank('/ ) #output response = request.get_response(main.app) #oracle self.assertequal(response.status_int, 200) self.assertequal(response.body, 'Hello, world! )

16 Integra$on Tes$ng Thin line between unit and integra$on Strictly speaking when more than one component is used you are doing integra$on tes$ng For example if your web app uses a datastore then the test on the previous slide that came in via a GET request could be an integra$on test not a unit test This is not worth worrying about Unit/Integra$on Tes$ng is fine

17 System Tes$ng Now the full round- trip is being tested, including the Browser component Unit/Integra$on Tes$ng can be done on a local environment, but System Tes$ng need to be in the deployed environment State- of- the- Prac$ce Record and replay tests

18 Selenium hpp://docs.seleniumhq.org/ The #1 tool in Web System Tes$ng automates browsers. That s it! Much like the webapp2 example earlier, just many more op$ons, including browser specific drivers IDE for record and replay With scripts that can be edited WebDriver that allows one to run without a browser at all

19 Selenium Example from selenium import webdriver from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait # available since from selenium.webdriver.support import expected_conditions as EC # available since # Create a new instance of the Firefox driver driver = webdriver.firefox() # go to the google home page driver.get(" # find the element that's name attribute is q (the google search box) inputelement = driver.find_element_by_name("q") # type in the search inputelement.send_keys("cheese!") inputelement.submit() try: # we have to wait for the page to refresh, the last thing that seems to be updated is the title WebDriverWait(driver, 10).until(EC.title_contains("cheese!")) # You should see "cheese! - Google Search" print driver.title finally: driver.quit()

20 Java Example import com.thoughtworks.selenium.*; // This is the driver's import. You'll use this for instantiating a // browser and making it do what you need. public class NewTest extends SeleneseTestCase { public void setup() throws Exception { setup(" "*firefox"); } } public void testnew() throws Exception { selenium.open("/"); selenium.type("q", "selenium rc"); selenium.click("btng"); selenium.waitforpagetoload("30000"); asserttrue(selenium.istextpresent("results * for selenium rc")); }

21 Usability Tes$ng Based on user opinions Manual according to a list of tasks Observer records behavior Could use things like eye- tracking for more precise results Automated Usability Tes$ng is s$ll a research topic

22 Robustness Tes$ng Try anything and see if something breaks Most famous of these is Android Monkey Sends random keystrokes to your Android app gremlins.js does the same thing for web apps <script src="path/to/gremlins.min.js"></script> <script> gremlins.createhorde().unleash(); </script>

23 Performance Tes$ng System performance under specific loads Concurrent users doing a certain number of transac$ons for a certain dura$on This is a very hard kind of tes$ng to do Needs lots of infrastructure O\en the performance bopleneck is your tes$ng framework and not the system under test In the end we know where the problem is THE DATABASE! More generally where something has to wait for something else to finish

24 Web Performance Tes$ng Measure throughput or transac$on rate Server response $me Rendering, but that might need addi$onal scripts on the client side You might not know what is the expected performance, so o\en you profile the performance You will quickly no$ce bad performance

25 Web front- end Performance Tools Google Pagespeed tools Google Chrome Developer Tools Nice summary of tools related to performance can be found at hpp://samsaffron.com/archive/2012/03/23/ sam- s- ul$mate- web- performance- tools- and- resources

26 JMeter Load and Performance Tes$ng Server side Not a browser, but can simulate some ac$ons typically done by a browser HTTP, etc. No javascript execu$on

27 Security Tes$ng This is HUGE! A whole course or even degree can be devoted to it Nice checklist of things to do at hpps:// Web_Applica$on_Security_Tes$ng_Cheat_Sheet Large part of security tes$ng is manual Penetra$on Tes$ng Find security vulnerabili$es that can be exploited Tools to find buffer overflows is one of the most effec$ve automa$c approaches Fuzz Tes$ng Semi- structured random tes$ng

28 Zed APack Proxy (ZAP) Automated Penetra$on Tes$ng Point it to URL and it does the rest Includes Fuzzing and many more Find it here hpps:// OWASP_Zed_APack_Proxy_Project