Testing Magento 2 extensions in a continuous integration pipeline

As you might know, I like to write code that is tested. Now, writing tests in Magento can be quite hard, but running your tests in a continuous integration pipeline can be even harder. Even harder is it to test standalone Magento 2 extensions in a continuous integration pipeline.

Especially with standalone extensions, there is a lot of value in running your tests against multiple Magento 2 versions. You get information like:

• Is my code compliant with Magento 2.1, 2.2, 2.3, and 2.4?

• Is my code compliant with PHP 7.0, 7.1, and 7.1?

• Does setup:di:compile run on every version?

In the past, I've literally spent several hundred hours to get integration tests up & running for the extensions I was working on. Most of those problems existed because I needed a working Magento 2 installation to test my extensions against.

Recently I needed to repeat this process so I decided to get some help from Docker to skip the complicated part.

How does this all work?

The Composer image contains a complete Magento 2 installation, with a database and tools like Composer, Xdebug, and Blackfire on board in case you need it. It is also already configured to run unit and integration tests.

When the Github Action runs it will download this image. Next, it will:

• Copy your code into the docker container.

• Runs composer require <VENDOR>/<MODULE> so your extension including all dependencies is installed.

• Executes bin/magento setup:di:compile. This step is not required but is recommended.

• Developer mode is enabled (bin/magento deploy:mode:set developer). I'm not sure why this is required but if you skip this you might get a lot of errors.

• Run the unit/integration tests.

How to get started?

First of all, you need to decide if you want to run Unit tests, Integration tests, or both. For both cases I've created a starter template your can find here:

• Integration tests

• Unit tests

Copy the file(s) in your repository into the .github/workflows directory. If it doesn't exists yet just create it. Chance the <VENDOR> and <MODULE> placeholders in the file with your vendor and module name. Now, commit the files and push the commit. The Github actions should just run and you should be informed when something fails.

Do you support Travis CI/(Gitlab|)Bitbucket Pipelines/Circle CI/etc?

I only tested this on Github Actions, but as everything is in a Docker container the same idea could be applied to other systems too. 

Need help implementing this in your project? Send me an e-mail ([email protected]) or reach out on Twitter and I will see what I can do.

How do I know this works for my code?

My workflow for adding CI to an existing code base is pretty straightforward: create a new branch of the stable branch usually called CI, copy the config from the .github/workflows directory into the .github/workflows directory of my local project and push the new files to Github.

Now, you might get some error. Some are clear, some are not, but most of the times I end up changing little things, commit, push, and wait for the result. Sometimes it's an error in a specific Magento version (yay, that's what we want), sometimes it's a dependency problem (no access to Satis for example) or sometimes it's something completely else. It's always a surprise.

Need help with your project? You can always drop me a line at [email protected].

I can't get integration tests to run

Integration tests are great when they work, but can be a pain in the ass when there broken. Luckily for you, I have created Revive a while back, which exactly helps you with this. It helps you solve known bugs a gives you hints on what to do when Revive can't help you in an automatic way.