Avoiding The Needless Multiplication Of Forms

Automated Testing In Docker

Dr Andrew Moss


Let's speed up a bit and wrap up a testing sequence inside Docker. This is the Dockerfile that we use to build the container. This is only a first step to check that we can get the results that we need - these six tests are not dependent on one another and so failure in one should not prevent the rest from executing. This build script does replicate much of the testing. Something that is apparent immediately is that this way of working is akin to single-user mode. The file-system can be stomped over (making various top-level directories and ignoring the rest), the system runs as root so we can ignore file-permissions and users completely.
FROM ubuntu_localdev ADD student.tgz /testroot/ COPY 1.lua /testcases/ COPY 2.lua /testcases/ COPY 3.lua /testcases/ COPY 4.lua /testcases/ COPY 5.lua /testcases/ WORKDIR /testroot/ass1-int RUN ls RUN make RUN mkdir /testresults/ RUN ./int >/testresults/noargs RUN ./int /testcases/1.lua >/testresults/1 RUN ./int /testcases/2.lua >/testresults/2 RUN ./int /testcases/3.lua >/testresults/3 RUN ./int /testcases/4.lua >/testresults/4 RUN ./int /testcases/5.lua >/testresults/5
This runs and we can jump in to get results directly (picking a student that did well in the assignment helps - test the positive test cases first and then go back to test the failures) :
$ docker build -t=current_test . Sending build context to Docker daemon 326.7 kB Step 0 : FROM ubuntu_localdev ---> f899773591db Step 1 : ADD student.tgz /testroot/ ---> Using cache ---> 7efb17601604 Step 2 : COPY 1.lua /testcases/ ---> Using cache ---> e1981c1bd075 Step 3 : COPY 2.lua /testcases/ ---> Using cache ---> a1097f1868e5 Step 4 : COPY 3.lua /testcases/ ---> Using cache ---> 29438a35f659 Step 5 : COPY 4.lua /testcases/ ---> Using cache ---> 185197776daf Step 6 : COPY 5.lua /testcases/ ---> Using cache ---> 5b10271f9ed8 Step 7 : WORKDIR /testroot/ass1-int ---> Using cache ---> d5dc06b80cd7 Step 8 : RUN ls ---> Using cache ---> 39d93f0e0d73 Step 9 : RUN make ---> Using cache ---> dcfa1cb02748 Step 10 : RUN mkdir /testresults/ ---> Using cache ---> d1f6ece4509a Step 11 : RUN ./int >/testresults/noargs ---> Using cache ---> ef45f84616c6 Step 12 : RUN ./int /testcases/1.lua >/testresults/1 ---> Using cache ---> 349c50894154 Step 13 : RUN ./int /testcases/2.lua >/testresults/2 ---> Using cache ---> c5586940a4fb Step 14 : RUN ./int /testcases/3.lua >/testresults/3 ---> Using cache ---> 888c850b9164 Step 15 : RUN ./int /testcases/4.lua >/testresults/4 ---> Using cache ---> f75184fa8fa8 Step 16 : RUN ./int /testcases/5.lua >/testresults/5 ---> Using cache ---> 61ec01ac3dba Successfully built 61ec01ac3dba
Have an interactive poke around to check the results make sense...
docker run -it current_test /bin/bash root@a02b5139bb7f:/testroot/ass1-int# ls /testresults 1 2 3 4 5 noargs root@a02b5139bb7f:/testroot/ass1-int# cat /testresults/noargs digraph graphname { n1 [label="startProg"];n1 -> n3; ... <snip big-ass file>
Oh yes, now I remember that they output the Graphviz dot-format for an intermediate representation in that assignment. So that means that our testing container isn't equipped to deal with the test. Have we struck disaster, or is Docker so amazingly choice that we can overcome this? Stay tuned to find out..
$ docker restart moon_base1 moon_base1 $ docker attach moon_base1 root@b8e173f8481f:/# apt-get install graphviz
Lots of snipped apt-get output snipped, the gist of which was that installing the missig packages will probably not screw up the development image so lets go ahead and do that. Then we simply rebuild the test... If we think of the container as a delta from an image then we've edited the image so when we add the two together we get a new container:
$ docker commit moon_base1 ubuntu_localdev 74b59407d66fb5fd67fbb3255f9f435f079476bcac9ea259978ad85cf0005260 $ docker build -t=current_test . Sending build context to Docker daemon 326.7 kB Step 0 : FROM ubuntu_localdev ... *snip* ... Removing intermediate container 341861334c56 Successfully built eebb3556c88f $ docker run -it current_test /bin/bash root@c993bd4fc1d0:/testroot/ass1-int# ls /testresults 1 2 3 4 5 noargs root@c993bd4fc1d0:/testroot/ass1-int# dot -Tpdf /testresults/1 -o /testresults/1.pdf root@c993bd4fc1d0:/testroot/ass1-int# ls -l /testresults total 44 -rw-r--r-- 1 root root 614 Sep 7 13:32 1 -rw-r--r-- 1 root root 13852 Sep 7 13:35 1.pdf -rw-r--r-- 1 root root 1891 Sep 7 13:32 2 -rw-r--r-- 1 root root 3552 Sep 7 13:32 3 -rw-r--r-- 1 root root 2748 Sep 7 13:32 4 -rw-r--r-- 1 root root 6772 Sep 7 13:32 5 -rw-r--r-- 1 root root 76 Sep 7 13:32 noargs
Very nice Rodney. We just rewrote the past a little because the testing can be applied automatically to the new environment. Cool, so how do we get the results out to look at on the host machine?
$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES c993bd4fc1d0 current_test "/bin/bash" 3 minutes ago Exited (0) 3 seconds ago sad_turing $ docker cp sad_turing:/testresults/1.pdf . $ ls -l 1.pdf -rw-r--r-- 1 andrewmoss staff 13852 Sep 7 15:35 1.pdf
Excellent Smithers... A full testing cycle. Next time we will sort out the independence of the individual tests and maybe even do some actual grading...