JuliaPlots/VisualRegressionTests.jl

Name: VisualRegressionTests.jl

Owner: JuliaPlots

Description: Automated integrated regression tests for graphics libraries

Created: 2015-11-24 18:10:29.0

Updated: 2018-05-23 06:31:08.0

Pushed: 2017-05-17 01:30:45.0

Homepage: null

Size: 122

Language: Julia

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

VisualRegressionTests

Easy regression testing for visual packages. Automated tests compare similarity between a newly generated image and a reference image using the Images package. While in interactive mode, the tests can optionally pop up a Gtk GUI window showing a side-by-side comparison of the test and reference image, and then optionally overwrite the reference image with the test image. This allows for straightforward regression testing of image data, even when the “correct” images change over time.

Author: Thomas Breloff (@tbreloff)

Setup:

a> using VisualRegressionTests
a> using Plots; gr();

Our test function. Make a scatter plot using the GR backend, and save a png to the location given.

a> func = fn -> begin
       scatter(rand(100), size=(300,400))
       png(fn)
   end
nymous function)

Create a reference image:

a> reffn = tempname() * ".png"
p/juliaSVc8Po.png"

a> func(reffn)

Now do a test. It should fail since we're plotting different random values.

a> result = test_images(VisualTest(func, reffn))
ING: Got error: ErrorException("Arrays differ.  Difference: 0.10559923034291491  eps: 0.02")
ING: Image did not match reference image /tmp/juliaSVc8Po.png. err: ErrorException("Arrays differ.  Difference: 0.10559923034291491  eps: 0.02")
ING: Should we make this the new reference image?
alRegressionTests.VisualTestResult("/tmp/juliaB4oxB0.png",RGB Images.Image with:
ta: 300x400 Array{ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}},2}
operties:
colorspace: sRGB
spatialorder:  x y,"/tmp/juliaSVc8Po.png",RGB Images.Image with:
ta: 300x400 Array{ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}},2}
operties:
colorspace: sRGB
spatialorder:  x y,DOES_NOT_MATCH::VisualRegressionTests.VisualTestStatus,0.10559923034291491,ErrorException("Arrays differ.  Difference: 0.10559923034291491  eps: 0.02"))

The result contains some useful fields, including status, diff, and err, as well as the filenames and Images which were tested.

For automated tests, you can use Base.success:

a> using Base.Test

a> @test test_images(VisualTest(func, reffn), popup=false) |> success

ING: Got error: ErrorException("Arrays differ.  Difference: 0.10323637729974526  eps: 0.02")
ING: Image did not match reference image /tmp/juliaSVc8Po.png. err: ErrorException("Arrays differ.  Difference: 0.10323637729974526  eps: 0.02")
R: test failed: test_images(VisualTest(func,reffn),popup=false) |> success
expression: test_images(VisualTest(func,reffn),popup=false) |> success
error at ./error.jl:21
default_handler at test.jl:30
do_test at test.jl:53

Example GUI popup: