So you've successfully automated your test cases and looking for the next step. Chances are, you will need to integrate or create a CI pipeline for it. If you are using GitLab as your code repository and looking for a way how to create a simple test execution pipeline, this blog is for you!

Before proceeding to the setup, make sure you are fairly familiar with pipelines, jobs, and artifacts.

The setup 💻

To start off, create a .gitlab-ci.yml file in the root folder of your GitLab repository. If there is an existing .gitlab-ci.yml file, just add the job and stage that we will be creating.

Define the stages 📊

Start by adding the keyword stages inside the file. This keyword would define the pipeline execution flow while each stage may contain single or multiple jobs. Add a single stage since we will only need one for this blog. It should look like this:

stages:
  - test-execution

This simply means that the pipeline would only have 1 stage called test-execution.

Create a job 🤖

The next step is to create a job by adding the job name with the same level of stages keyword. Jobs may contain multiple keywords for different reasons. But for test automation, we only need a few keywords to get things running.

stages:
  - test-execution

automated-tests-execution:
  stage: test-execution

In the example above, we have created a job named automated-tests-execution that has a job keyword called stage. This keyword indicates that the job will execute when the test-execution stage runs.

Image and variables 🖼

The image keyword is used to select a docker image that the job will be using. If you are not familiar with it, check out the Docker documentation!

Optional

We also used variables keyword to define a variable for the tests' root folder path. This can be useful in some scenarios, like if you are sharing a repository with your developer's source code. Feel free to not include this step seeing that this is situational.

stages:
  - test-execution

automated-tests-execution:
  stage: test-execution
  image: node:latest
  variables:
    TESTS_FOLDER : "./automated-tests"

Artifacts 🗿

Artfacts

Assuming the pipeline has successfully executed your tests and some test cases failed. How would you get the test execution report and investigate it? This is where the artifacts would come. Job artifacts are files produced during job execution like a test execution result (html/xml report).

Here are the keywords we used for artifacts:

• paths - list of files or folders to upload as artifacts after job execution.
• when - indicates when to upload the artifacts. Artifacts would be uploaded when the job succeeds by default. We set it to always so it would always upload the artifacts.
• expire_in - expiration time of artifacts. For test automation, it is recommended to set an expiration for artifacts since we often execute tests.

stages:
  - test-execution

automated-tests-execution:
  stage: test-execution
  image: node:latest
  variables:
    TESTS_FOLDER : "./automated-tests"
  artifacts:
      paths:
        - $TESTS_FOLDER/output/
      when: always
      expire_in: 1 week

The script 📜

Finally, we can run the tests by adding the script keyword. The command that will be supplied will depend on your test framework and library. Since I am using cucumber for my tests, I have added yarn cucumber-js -p test features/.

Check out my other blog on how to setup PactumJS with Cucumber for API testing!

stages:
  - test-execution

automated-tests-execution:
  stage: test-execution
  image: node:latest
  variables:
    TESTS_FOLDER : "./automated-tests"
  artifacts:
      paths:
        - $TESTS_FOLDER/output/
      when: always
      expire_in: 1 week
  script:
    - cd $TESTS_FOLDER
    - yarn cucumber-js -p test features/

Run the pipeline 🏃🏽‍♀️

So you've successfully configured the pipeline, let's run the GitLab pipeline and see the result. Navigate to CI/CD > Pipelines, click Run pipeline and select the branch where you pushed the .gitlab.yml file.

Congrats! If your configuration is correct, the pipeline should run without any problem!