May 12, 2015

Depicted - Make your deployment QA safer

Depicted is an API server and toolkit which allows you to view screenshots called 'perceptual diffs' of the current production pages and the ready-to-release pages. It's open source so anyone can continue to improve it and is on Github. The perceptual diffs were rendered by Google when they were making map tiles and wanted to match them up before and after release.

Depicted's creator Brett Slatkin, an engineer at Google, created it when he wanted to find improvements and redice regression in their release cycle and follow continuous deployment methods. He had been meddling with CSS adding an animated pony to a Google product and neither him nor the QA team had picked it up until it had been live for 24 hours! Such visual things can easily be missed by tests, automation, code reviews and QA teams; visual representation would fair much better here.

I think this is a brilliant tool to reduce regression, reduce the amount of human resource and increase confidence in the team where some people may be worried about pushing live. You can pick up subtle changes and spacing issues.

If you push your new code to a staging environment, or have it running on a local server Depicted can 'diff' against the pixels of the previous and live versions with screenshots of bothin parallel. You can run these diffs at the same time as your automation tests so it doesn't have to be an extra stage.

There are extra scripts you can run to tell the tool to hide or blur certain CSS in the test and you can also run Javascript events into the tests. This helps with AJAX/dynamic data driven pages.

So how do we do this?

The Readme gives excellent instructions but i'll also explain how to do this locally in Terminal on Mac (obviously this works on live environments too.)

  1. Make sure you have Python 2.7 installed. Yosemite comes with it out of the box.
  2. You may also need to make sure PhantomJS and ImageMagick are installed (very easy via MacPorts)
  3. pip is a tool for easy install of Python packages that is recommended over easy_install. Install it - help is here by running 'sudo easy_install pip'
  4. run 'pip install dpxdt'
  5. In the 'dpxdt' folder that has downloaded, create a dummy HTML file and save it in the 'dpxdt' folder. The HTML can contain anything, try adding a bunch of text. I shall refer to it as demo.html. i.e.
    <h2>dpxdt local demo</h2>
    <p>dpxdt can be used to spot changes on local web servers.</p>
  6. Within the 'dpxdt/tests' folder create and save a .yaml file containing:
    # (tests/test.yaml)
    setup: |
        python -m SimpleHTTPServer
        url: http://localhost:8000/demo.html
        timeout_secs: 5
      - name: demo
        url: http://localhost:8000/demo.html
                width: 800
                height: 600
            injectCss: |
                body {
                  background-color: white;
  7. Local dpxdt will generate a screenshot via Terminal by running 'dpxdt update tests' which looks for the .yaml file, runs a Python server and captures and saves a screenshot in the /tests folder.
  8. The screenshot demo i get can be viewed here (until i sort out my image include on my CMS!)
  9. To view 'perceptual diffs' of a before and after and any failure we can use a different mode 'test' instead of 'update'. Instead of just saving a screenshot we can instead compare them. To do this, make a change to your demo.html file and press save. (For my example i changed 'wow' to 'yaaaar' in my text.)
  10. Run 'dpdxdt test tests'
  11. You will notice in Terminal that the test failed and it has output three screenshots: reference, run and diff} (you need to check your Terminal window for the correct path to the image folder its saved them into.)
  12. The three images I see looks like You can see that the bottom left image shows in red the area that has shown a change or regression.

I've touched the surface here but I aim to use this in my work and will update with notes on integration tips.