"Acceptance Tests with Codeception and Gitlab-CI"

or

...how to solve a hen-egg problem

Ole Rößner | neusta GmbH | o.roessner@neusta.de |      @djbasster

This work is licensed under a Creative Commons Attribution 4.0 International License

whoami

  • Ole Rößner
  • Bremen
  • Father
  • neusta GmbH (Bremen)
  • DevCoachConsultantOps
  • Symfony Enthusiast
  • Clean Code Evangelist
  • Noisemaker off duty


        @djbasster

Acceptance Tests with Codeception and Gitlab-CI

Acceptance Tests with Cypress and Jenkins*

*not tested personally

Acceptance Tests with Cucumber and TeamCity*

*not tested personally

LOOK MA,

 

NO HANDS!

SERVERS!

It's about a Concept!

Acceptance Tests with

and

-CI

Disappointment first...

Acceptance Tests?

Test Pyramid

Unit Tests

Integration Tests

Acceptance Tests

End-2-End Tests

Acceptance Tests

Frontend

Backend

Database

Cache

Search Engine

Queue

Test Environment

The Challenge

main

feature/1-foo

feature/2-bar

feature/3-baz

Test Environment 1

Test Environment 2

Test Environment 3

Frontend

Backend

Database

Cache

Search Engine

Queue

Test Environment 1

Test Environment 2

Test Environment 3

https://1-foo.my-project.com

https://2-bar.my-project.com

https://3-baz.my-project.com

Frontend

Backend

Database

Cache

Search Engine

Queue

Frontend

Backend

Database

Cache

Search Engine

Queue

Frontend

Backend

Database

Cache

Search Engine

Queue

Test Environment 1

Test Environment 2

Test Environment 3

https://1-foo.my-project.com

https://2-bar.my-project.com

https://3-baz.my-project.com

One Server?

x Servers?

Bootstrap?

Tear Down?

Seeding?

DNS?

Questions:

Hen or Egg first?

Remember the Test Pyramid?

Unit Tests

Integration Tests

Acceptance Tests

Automated

Automated

Manually

Teddy Tester

Answer:

Automate Acceptance Tests with

codeception.com

Written in PHP

Write your tests in PHP

Gherkin compatible

BDD user centric style

Installs via Composer

Modular

Extendable

Example Scenario

SourceCode: gitlab.com/oroessner/acceptance-tests-with-codeception-and-gitlab-ci

Our First Test

<?php

namespace Tests\Acceptance;

final class TodoCest
{
    public function tryToSeeTheDefaultTask(\AcceptanceTester $I): void
    {
        $I->amOnPage('/');
        $I->see('test this!');
    }
}

Our Second Test

<?php

namespace Tests\Acceptance;

final class TodoCest
{
	// ...other tests
    
    public function tryToCreateATask(\AcceptanceTester $I): void
    {
        $task = 'Task from Codeception';

        $I->amOnPage('/');
        $I->fillField('Add task:', $task);
        $I->click('Submit');

        $I->see($task);
    }
}

Application "Architecture"

Application Development

version: '3'
services:
  db:
    image: mariadb:10.4.17
    environment:
      MYSQL_ROOT_PASSWORD: 123
      MYSQL_USER: &dbUser 'todo'
      MYSQL_PASSWORD: &dbPass 'pas3w0rd'
      MYSQL_DATABASE: 'todo'
    volumes:
    - ./.docker/mariadb:/docker-entrypoint-initdb.d
    ports:
    - "127.0.0.1:3306:3306"

  todo:
    build:
      context: .
    environment:
      DB_DSN: 'mysql:dbname=todo;host=db'
      DB_USER: *dbUser
      DB_PASSWORD: *dbPass
    links:
      - db
    ports:
    - "80:80"
    volumes:
    - ./public:/var/www/html

Application Dockerfile

FROM php:8-apache

COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/

RUN set -eux; \
    install-php-extensions pdo_mysql;

WORKDIR /var/www/html/

COPY public/ /var/www/html/

Automate with Docker and

-CI

Manually use Docker?

  • Not very user friendly
  • Networking?
  • A lot of scripting
  • Who's cleaning up?

Use Docker-Compose like in dev?

  • Reuse existing environment!
  • Not so much scripting.
  • Who's cleaning up?

Sidecars!

  • Configure in your CI-config!
  • Almost no scripting!
  • CI-Agent is cleaning up for you!

* AFAIK Gitlab-CI, Jenkins, TeamCity and GitHub Actions have a Sidecars feature!

Make a plan!

Change your app

Build an Image

Use Image as Sidecar

Run tests against the Sidecar

stages:
  - build
  - test

variables:
  # Example: "registry.gitlab.com/oroessner/my-project:ci-feature-1-foobar-V1234"
  BUILD_IMAGE: "registry.gitlab.com/oroessner/my-project:ci-$CI_COMMIT_REF_SLUG-V$CI_PIPELINE_IID"

build-test-image:
  stage: build
  image: docker:stable
  services:
    - docker:stable-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $BUILD_IMAGE .
    - docker push $BUILD_IMAGE

Gitlab-CI Predefined environment variables

.gitlab-ci.yml

stages:
  - build
  - test

variables:
  BUILD_IMAGE: "registry.gitlab.com/oroessner/my-project:ci-$CI_COMMIT_REF_SLUG-V$CI_PIPELINE_IID"
  FF_NETWORK_PER_BUILD: 1

build-test-image: [...]

acceptance-tests:
  stage: test
  image: registry.gitlab.com/oroessner/my-project/ci-runner:latest
  services:
    - name: $BUILD_IMAGE
      alias: website
    - name: mariadb:10.4.17
      alias: db
  variables:
    # for the mariadb image
    MYSQL_ROOT_PASSWORD: 123
    MYSQL_USER: &dbUser 'todo'
    MYSQL_PASSWORD: &dbPass 'pas3w0rd'
    MYSQL_DATABASE: 'todo'
    # for the application image
    DB_DSN: 'mysql:dbname=todo;host=db'
    DB_USER: *dbUser
    DB_PASSWORD: *dbPass
  before_script:
    # Install composer dependencies
    - composer install --no-progress --classmap-authoritative
    # wait 60 seconds for the database to be ready.
    - wait-for-it db:3306 -t 60
  script:
    - vendor/bin/codecept run --env=ci --steps
  cache:
    key: $CI_COMMIT_REF_SLUG
    paths:
      - vendor

.gitlab-ci.yml

Our plan in detail!

Change your app

Build an image

Use image as sidecar

Run tests against the Sidecar(s)

Push image to registry

Create CI job

Define other service sidecars

1

2

3

Create simple CI image

My simple ci-runner image for Codeception

FROM php:8-cli

COPY --from=composer /usr/bin/composer /usr/bin/composer
COPY --from=djbasster/wait-for-it:latest /usr/bin/wait-for-it /usr/bin/wait-for-it
COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/

RUN set -eux; \
    apt-get update; \
    apt-get install -y --no-install-recommends unzip; \ # for composer
    install-php-extensions pdo_mysql zip; # for codeception

# create.sh

docker build --no-cache --pull -t registry.gitlab.com/oroessner/my-project/ci-runner:latest .
docker push registry.gitlab.com/oroessner/my-project/ci-runner:latest

Codeception config

# tests/acceptance.suite.yml
actor: AcceptanceTester
suite_namespace: Tests\Acceptance
modules:
    enabled:
      - PhpBrowser:
          url: http://localhost/
      - Db:
          # for local docker-compose
          dsn: 'mysql:host=127.0.0.1;dbname=todo' 
          user: 'root'
          password: '123'
          dump: .docker/mariadb/dump.sql
          populate: true
          cleanup: true
          reconnect: true
      - \Helper\Acceptance
    step_decorators: ~

# tests/_envs/ci.yml
modules:
  config:
    PhpBrowser:
      url: http://website
    Db:
      # for gitlab-ci
      dsn: 'mysql:host=db;dbname=todo'

Hello Pipeline

PhpBrowser?

Uses Guzzle to interact with your application over CURL. Module works over CURL and requires PHP CURL extension to be enabled.

Use to perform web acceptance tests with non-javascript browser.

Codeception: PhpBrowser Module

What about JavaScript support?

Selenium & WebDriver

Selenium Server

A real Browser

WebDriver

My Testcode

Codeception Tests with WebDriver

<?php 
namespace Tests\Acceptance;
final class TodoCest
{
    public function tryToSeeTheDefaultTask(\AcceptanceTester $I): void
    {
        $I->amOnPage('/');
        $I->see('test this!');
    }

    public function tryToCreateATask(\AcceptanceTester $I): void
    {
        $task = 'Task from Codeception';

        $I->amOnPage('/');
        $I->fillField('Add task:', $task);
        $I->click('Submit');

        $I->see($task);
    }
}

?!

Codeception config with Selenium

# tests/acceptance.suite.yml
actor: AcceptanceTester
suite_namespace: Tests\Acceptance
modules:
    enabled:
      - WebDriver:
          url: http://todo/
          browser: chrome
          host: localhost
      - Db: [...] # same
      - \Helper\Acceptance
    step_decorators: ~

# tests/_envs/ci.yml
modules:
  config:
    WebDriver:
      url: http://website/
      browser: chrome
      host: selenium
    Db:
      # for gitlab-ci
      dsn: 'mysql:host=db;dbname=todo'

stages:
  - build
  - test

variables:
  BUILD_IMAGE: "registry.gitlab.com/oroessner/my-project:ci-$CI_COMMIT_REF_SLUG-V$CI_PIPELINE_IID"
  FF_NETWORK_PER_BUILD: 1

build-test-image: [...]

acceptance-tests:
  stage: test
  image: registry.gitlab.com/oroessner/my-project/ci-runner:latest
  services:
    - name: $BUILD_IMAGE
      alias: website
    - name: mariadb:10.4.17
      alias: db
    - name: selenium/standalone-chrome
      alias: selenium
  variables:
    # for the mariadb image
    MYSQL_ROOT_PASSWORD: 123
    MYSQL_USER: &dbUser 'todo'
    MYSQL_PASSWORD: &dbPass 'pas3w0rd'
    MYSQL_DATABASE: 'todo'
    # for the application image
    DB_DSN: 'mysql:dbname=todo;host=db'
    DB_USER: *dbUser
    DB_PASSWORD: *dbPass
  before_script:
    # Install composer dependencies
    - composer install --no-progress --classmap-authoritative
    # wait 60 seconds for the database to be ready.
    - wait-for-it db:3306 -t 60
    - wait-for-it selenium:4444 -t 60
  script:
    - vendor/bin/codecept run --env=ci --steps
  cache:
    key: $CI_COMMIT_REF_SLUG
    paths:
      - vendor

.gitlab-ci.yml for Selenium Tests

Hello Pipeline

Special Case: PHP-FPM

or

The Internet

Special Case: Test Files

Rules and Issues:

  • Don't build a separate image for testing!
  • Don't packages test files in your (prod) image!
  • You cannot mount volumes into sidecars!*

*at least on Gitlab-CI

I lied!     (sort of...)

on-the-fly

Special Case: Test Files

ARG BASE_IMAGE="registry.gitlab.com/oroessner/my-project"

FROM ${BASE_IMAGE}

COPY cat.png /var/www/html/img/

Special Case: Test Files

variables:
  BUILD_IMAGE: "registry.gitlab.com/oroessner/acceptance-tests-with-codeception-and-gitlab-ci:ci-$CI_COMMIT_REF_SLUG-V$CI_PIPELINE_IID"
  TEST_IMAGE: "registry.gitlab.com/oroessner/acceptance-tests-with-codeception-and-gitlab-ci:ci-test-$CI_COMMIT_REF_SLUG-V$CI_PIPELINE_IID"

build-test-image:
  stage: build
  image: docker:stable
  services:
    - docker:stable-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $BUILD_IMAGE .
    - docker build -t $TEST_IMAGE --build-arg BASE_IMAGE="${BUILD_IMAGE}" .docker/build-image-with-test-files
    - docker push $BUILD_IMAGE
    - docker push $TEST_IMAGE
    
 acceptance-tests:
  stage: test
  image: registry.gitlab.com/oroessner/my-project/ci-runner:latest
  services:
    - name: $TEST_IMAGE
      alias: website
    - name: mariadb:10.4.17
      alias: db
  [...]

Special Case: Test Files

<?php

namespace Tests\Acceptance;

class TodoCest
{
    // [...]

    public function tryToSeeTheCatImage(\AcceptanceTester $I):void
    {
        $I->amOnPage('/');
        $I->seeElement('img[alt="cats rule the internet"]');
    }
}

Thank you!

Questions?

Slides

slides.com/neusta/acceptance-tests-with-codeception-and-gitlab-ci

Code Examples

gitlab.com/oroessner/acceptance-tests-with-codeception-and-gitlab-ci

Ole Rößner | neusta GmbH | o.roessner@neusta.de |      @djbasster