How We Evolved our QA TestScripts into a Distributed TestService with Elixir!
I have recently joined the elixir bandwagon toward the end of last year, and now I have a personal mission to use this technology to advance QA testing into its most needed next phase. As I write this I do believe both Clojure and Elixir are the only two development ecosystems in position to advance QA automation effectively. In this post I'm going to explain how I've set up our system architecture at Spongecell using Elixir, Robot Framework, and a little imagination to build a robust Test Service known as DUEY.
There are 5 main parts that make up the full test service. Each part has a specific responsibility, and the cool part is users of the test service only need to know the commands to send to the test service in order to use it. That is it. This way there is no confusion, no extra training, and each part can be updated independently as our product changes, and as the ad tech industry evolves. Updates are able to happen without the user even being aware that they are going on. A visual representation of this architecture is presented below
(A quick rant about how much of a god send Elixir is thanks to Jose' Valim) Our Test Service needed to be used by anyone or anything in the company. This was a huge requirement, and it could not have been done without the use of Elixir/Erlang's BEAM environment. With the use of these technologies we have been able to distribute our test automation tools to anyone regardless of their office location, or time zone. In addition to that, because this service is exposed to so many people and offices, its bound to get a bad request from someone or something, but thanks to the fault tolerance features, our service restarts and lets the user know that they probably need to choose from a list of appropriate requests.
Here is how the architecture breaks down!
- The User
As I said before the user can be anyone or anything. It can be a Jenkins Server, a QA tester, developer, or any random company employee. They decide internally what they want to test and they send that request to the the test service that is always up and waiting for commands from interested parties.
- The Test Service
The test service which is built entirely in elixir using the GenServer behavior, and GenEvent Behavior modules, is designed to listen for many types of requests from mobile automation, all the way to api and web integration test requests. once it gets a valid request that it knows how to handle from a user it begins to carryout that action for the user. In order to do that there are many things that need to happen after that request that involve the other pieces of the architecture. The user never even realizes that these things are happening, nor should they.
- The Robot Framework
The robot framework is a generic DSL that allows users to write test steps in a plain english like syntax. To find out more about it click here. This layer is used by the qa team. It allows all members to write their own tests without needing to worry about coding in a specific programming language. It also saves development from having to worry too much about implementing integration tests for their features because it is handled as a part of the QA cycle. The thing is for this layer to be effective the test logic that governs the english based commands on the robot level have to be implemented somewhere. This is where the Test libraries come in.
- The Test Libraries
The test libraries expose themselves as modules that can be imported by the robot framework. This level is where all the heavy lifting happens. This is where I spend the majority of my time coding up generic steps that it might take to complete a specific task. This layer was originally done in python, but because of the python 2/3 versioning hell that ensued for us, all the logic has been moved over to Elixir. The users of the Robot Framework don't need to know how the functions do what they do, they only need to know the function module, and function name, along with the inputs. These functions are then exposed to the robot framework to be used by the QA members while making their test scripts.
- The Data Manager
The elephant in the room for any tester is data setup. I side stepped this headache by building out a data manger that knows how to communicate with the test libraries and serve up any test data that is needed to carry out a specific test task. The data source can be anything from Kafka, to Cassandra, to Mysql, to a generic endpoint, or a flat out csv file!!! The user should NEVER have to worry about data setup for a test. If that burden is ever placed on a user then you are missing out on the full benefits of automation.
- The System Under Test
This can be anything, from a web app, mobile app, or Adtag server. After each layer has done its job it now takes all that information to the system under test and performs its action. When the action is completed a response is sent back to the user saying all is ok, or there is a specific thing wrong.
This whole loop from request to response takes on average 2 to 5 seconds depending on how complicated the request is. We have cases of thousands of adtags being checked in just under 2 seconds at times!!! Here is a quick use case in IEX of what its like to use our new system to check live AD Tags on the web.
Duey is simply a node that can take a whole bunch of commands. The checks above normally take less than 2 seconds when utilizing concurrency. It previously took 15 seconds when it was written in Python!!
Honestly, the sky is the limit with Elixir. As QA Automation Engineers we should really start investigating these next generation technologies, because automated test system requirement complexity will crush you without knowledge of these languages. I'm a huge fan of automation and automated systems, and to me Elixir gives you many tools straight out of the box that allows you to do this effectively while keeping up with the complexities of modern day system requirements. Your team will never know the complexity involved, all they will know is that integration testing is automated and way less of a pain than it use to be. Developers and stakeholders get rapid feedback, and the QA team gets satisfaction out of having a service that catches many failures in each product! As I stated before on Twitter, Elixir is now my go to tool for automation projects.
Stay tuned for my open source library called Robot Remote Server Elixir! This will allow you to execute Robot Framework commands written specifically in Elixir!!