– Angular e2e Protractor Tests on Systems without GUI applied to static and dynamic Web Pages –

In this blog post, we will show how to perform end-to-end (e2e) tests with Angular 4:

• first, we will apply Protractor end to end tests on a little Angular CLI hello world application with static HTML content
• second, we will perform end to end tests for a dynamic web application, a REST client application that retrieves and displays content received via the WordPress API.

For that, we will use a Docker Protractor Image that can be used on systems with or without graphical user interface (e.g. a Jenkins CI system).

In a classical installation situation, you often use a machine with a graphical window system (e.g. X11 for Linux) so you can run a real Chrome or Firefox Browser on the system. In such a situation, you often install Jasmine, Protractor, Selenium and a Chrome or Firefox browser on the test machine. However, in this blog post, we prefer to use a pre-installed Docker image provided by webnicer instead, which allows us to run the e2e tests on Docker hosts without a graphical system. The missing need for a graphical interface makes this an ideal deployment option for continuous integration purposes, e.g. for Jenkins systems.

Why End to End Tests?

My motivation to write a blog about end-to-end tests is, that I have made a very good experience with „tests first“ and more specifically „behavior driven development“ (BDD) principles with previous projects. And end-to-end tests are the main ingredient needed for BDD. In my experience, projects that follow „test first“ or BDD principles benefit from a better customer view and higher quality, together with higher motivated developers, who start their work with a set of failed (red) tests and are rewarded for their work with successful (green) tests.

Even if we are far from following behavior driven principles yet, let us perform our first tiny steps towards this principle by looking more closely at end-to-end testing now:

Step 1: Install and start an Angular CLI Hello World Application

We are closely following the phase 1 instructions on a previous popular blog post of mine Consuming a RESTful Web Service with Angular 4. Since I do not want to copy and paste that part of the other blog post, let me just summarize the commands after you have got access to a Docker host (e.g. by following the instructions found here; search for the term “Install a Docker Host”):

• start Docker container

docker run -it -p 4200:4200 -v $(pwd):/localdir centos bash

• install Angular CLI

(container)# yum install -y epel-release
(container)# yum install -y https://kojipkgs.fedoraproject.org//packages/http-parser/2.7.1/3.el7/x86_64/http-parser-2.7.1-3.el7.x86_64.rpm
(container)# yum install -y nodejs 
(container)# npm install -g @angular/cli

• create a project

(container)# cd /localdir
(container)# ng new my-project-name
(container)# cd my-project-name

• we do not yet start the service in order to see a certain error message, but in step 3, we will start the service with this command:

(container)# ng serve --host 0.0.0.0

For more detailed information about Step 1, see phase 1 of this blog post.

Step 2: Prepare Docker Host for Protractor Usage

For a convenient handling of the protractor test, let us open a new terminal and create a shell script on the Docker host like follows (see the readme of the Docker image page webnicer/protractor-headless):

If your Docker host does not allow the usage of sudo, then try the same commands without sudo.

(dockerhost)$ sudo cat - << END | sudo tee /usr/local/bin/protractor-headless
#!/bin/bash
docker run -it --privileged --rm --net=host -v /dev/shm:/dev/shm -v \$(pwd):/protractor webnicer/protractor-headless \$@
END

Then we make sure the file is executable:

(dockerhost)$ sudo chmod +x /usr/local/bin/protractor-headless
(dockerhost)$ which /usr/local/bin/protractor-headless
/usr/local/bin/protractor-headless

Step 3: Download and run Protractor in a Docker Container

Let us download the protractor Docker image webnicer/protractor-headless so the container will be started immediately from the image later:

(dockerhost)$ docker pull webnicer/protractor-headless 

Then we enter the project root folder we have created in step 1:

(dockerhost)$ cd my-project-name

and run protractor via:

(dockerhost)$ protractor-headless

[23:43:08] I/direct - Using ChromeDriver directly...
[23:43:08] I/launcher - Running 1 instances of WebDriver
Spec started
[23:43:24] E/protractor - Could not find Angular on page http://localhost:4200/ : retries looking for angular exceeded

  my-project-name App
    ? should display welcome message
      - Failed: Angular could not be found on the page http://localhost:4200/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load
          at /usr/local/lib/node_modules/protractor/built/browser.js:506:23
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)
      From: Task: Run it("should display welcome message") in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:16:5
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:10:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:3:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:382:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:385:12)

**************************************************
*                    Failures                    *
**************************************************

1) my-project-name App should display welcome message
  - Failed: Angular could not be found on the page http://localhost:4200/. If this is not an Angular application, you may need to turn off waiting for Angular. Please see https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular-on-page-load

Executed 1 of 1 spec (1 FAILED) in 10 secs.
[23:43:24] I/launcher - 0 instance(s) of WebDriver still running
[23:43:24] I/launcher - chrome #01 failed 1 test(s)
[23:43:24] I/launcher - overall: 1 failed spec(s)
[23:43:24] E/launcher - Process exited with error code 1

Okay, this is expected, because no angular is running on port 4200 yet.

Step 4: Start Angular CLI Application and run Protractor again

As anticipated, let us start the Angular application as describes in step 1:

(container)# ng serve --host 0.0.0.0

Then we get following output, if we try to run the e2e tests:

(dockerhost)$ protractor-headless

[00:19:21] I/direct - Using ChromeDriver directly...
[00:19:21] I/launcher - Running 1 instances of WebDriver
Spec started

  my-project-name App
    ? should display welcome message
      - Failed: Error: Error while waiting for Protractor to sync with the page: "Could not find testability for element."
          at proxyDone.fail (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:87:34)
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)
      From: Task: Run it("should display welcome message") in control flow
          at Object. (/usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:79:14)
          at /usr/local/lib/node_modules/protractor/node_modules/jasminewd2/index.js:16:5
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
      From asynchronous test:
      Error
          at Suite. (/protractor/e2e/app.e2e-spec.ts:10:3)
          at Object. (/protractor/e2e/app.e2e-spec.ts:3:1)
          at Module._compile (module.js:570:32)
          at Module.m._compile (/protractor/node_modules/ts-node/src/index.ts:382:23)
          at Module._extensions..js (module.js:579:10)
          at Object.require.extensions.(anonymous function) [as .ts] (/protractor/node_modules/ts-node/src/index.ts:385:12)

**************************************************
*                    Failures                    *
**************************************************

1) my-project-name App should display welcome message
  - Failed: Error: Error while waiting for Protractor to sync with the page: "Could not find testability for element."

Executed 1 of 1 spec (1 FAILED) in 0.565 sec.
[00:19:26] I/launcher - 0 instance(s) of WebDriver still running
[00:19:26] I/launcher - chrome #01 failed 1 test(s)
[00:19:26] I/launcher - overall: 1 failed spec(s)
[00:19:26] E/launcher - Process exited with error code 1

Step 5: Fix the „waiting for Protractor to sync“ Problem and run Protractor again

I have found a solution on this StackOverflow Q&A: we need to add the additional configuration line into protractor.conf.js:

useAllAngular2AppRoots: true

In my case, the protractor configuration file looks like follows:

// Protractor configuration file, see link for more information
// https://github.com/angular/protractor/blob/master/lib/config.ts

const { SpecReporter } = require('jasmine-spec-reporter');

exports.config = {
  allScriptsTimeout: 11000,
  specs: [
    './e2e/**/*.e2e-spec.ts'
  ],
  capabilities: {
    'browserName': 'chrome'
  },
  directConnect: true,
  baseUrl: 'http://localhost:4200/',
  useAllAngular2AppRoots: true,
  framework: 'jasmine',
  jasmineNodeOpts: {
    showColors: true,
    defaultTimeoutInterval: 30000,
    print: function() {}
  },
  onPrepare() {
    require('ts-node').register({
      project: 'e2e/tsconfig.e2e.json'
    });
    jasmine.getEnv().addReporter(new SpecReporter({ spec: { displayStacktrace: true } }));
  }
};

After that, the e2e test via Protractor Docker Container is successful:

protractor-headless
[00:46:32] I/direct - Using ChromeDriver directly...
[00:46:32] I/launcher - Running 1 instances of WebDriver
Spec started
[00:46:38] W/element - more than one element found for locator By(css selector, app-root h1) - the first result will be used

  my-project-name App
    ? should display welcme message

Executed 1 of 1 spec SUCCESS in 1 sec.
[00:46:38] I/launcher - 0 instance(s) of WebDriver still running
[00:46:38] I/launcher - chrome #01 passed

Step 6: Review the Protractor Spec File

Now, we want to understand in more detail, what happened. For that, let us analyze the e2e specification file (e2e/app.e2e-spec.ts):

import { MyProjectNamePage } from './app.po';

describe('my-project-name App', () => {
  let page: MyProjectNamePage;

  beforeEach(() => {
    page = new MyProjectNamePage();
  });

  it('should display welcome message', done => {
    page.navigateTo();
    page.getParagraphText()
      .then(msg => expect(msg).toEqual('Welcome to app!!'))
      .then(done, done.fail);
  });
});

Now let us compare that with the browser content on http://localhost:4200:

There, it is: the e2e test is loading the page, retrieving the first paragraph and will compare it with the text „Welcome to app!!“. Since the text of the first paragraph matches this text, the test is successful.

Step 7 (optional): Review the Error Message of a failed Test

Now let us see, what happens, if we change the expected text in e2e/app.e2e-spec.ts to „Welcome to ppa!!“ instead:

$ protractor-headless

[00:32:30] I/direct - Using ChromeDriver directly...
[00:32:30] I/launcher - Running 1 instances of WebDriver
Spec started
[00:32:36] W/element - more than one element found for locator By(css selector, app-root h1) - the first result will be used

  my-project-name App
    ? should display welcome message
      - Expected 'Angular 4 Hello World Quickstart' to equal 'Welcome to ppa!!'.
          at /protractor/e2e/app.e2e-spec.ts:13:32
          at /usr/local/lib/node_modules/protractor/built/element.js:798:32
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)

**************************************************
*                    Failures                    *
**************************************************

1) my-project-name App should display welcome message
  - Expected 'Angular 4 Hello World Quickstart' to equal 'Welcome to ppa!!'.

Executed 1 of 1 spec (1 FAILED) in 1 sec.
[00:32:36] I/launcher - 0 instance(s) of WebDriver still running
[00:32:36] I/launcher - chrome #01 failed 1 test(s)
[00:32:36] I/launcher - overall: 1 failed spec(s)
[00:32:36] E/launcher - Process exited with error code 1

So, this is, how a failed test looks like.

Step 8 (advanced): Apply the e2e Test to a dynamic Web Page

Up to now, we have seen a simple test, which is comparing a static pattern with a static web page. We now want to apply the principle to a dynamic page as we have created in my blog post Consuming a RESTful Web Service with Angular 4. For that, we reverse the change in Step 7, so we get a successful e2e test again. Then we need to follow the steps in the corresponding blog post and run the resulting service on localhost port 4200. After that, the e2e test will fail:

(dockerhost)$ protractor-headless

[00:32:30] I/direct - Using ChromeDriver directly...
[00:32:30] I/launcher - Running 1 instances of WebDriver
Spec started
[00:32:36] W/element - more than one element found for locator By(css selector, app-root h1) - the first result will be used

  my-project-name App
    ? should display welcome message
      - Expected 'Angular 4 Hello World Quickstart' to equal 'Welcome to app!!'.
          at /protractor/e2e/app.e2e-spec.ts:13:32
          at /usr/local/lib/node_modules/protractor/built/element.js:798:32
          at ManagedPromise.invokeCallback_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1379:14)
          at TaskQueue.execute_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2913:14)
          at TaskQueue.executeNext_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2896:21)
          at asyncRun (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2775:27)
          at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:639:7
          at process._tickCallback (internal/process/next_tick.js:103:7)

**************************************************
*                    Failures                    *
**************************************************

1) my-project-name App should display welcome message
  - Expected 'Angular 4 Hello World Quickstart' to equal 'Welcome to app!!'.

Executed 1 of 1 spec (1 FAILED) in 1 sec.
[00:32:36] I/launcher - 0 instance(s) of WebDriver still running
[00:32:36] I/launcher - chrome #01 failed 1 test(s)
[00:32:36] I/launcher - overall: 1 failed spec(s)
[00:32:36] E/launcher - Process exited with error code 1

But what is the expected result we want to see? Let us head to http://localhost:4200 again, and we will see:

The blue part is static content again. However, the Title „Angular 4 Hello World Quickstart“ is dynamic content retrieved from the WordPress API. We can easily see that this pattern is missing in the source HTML code:

Nevertheless, we want to use our Protractor Docker Container to test, whether the dynamic content is visible in a browser. For that, let us adapt the e2e test file on (e2e/app.e2e-spec.ts) to the new situation:

import { MyProjectNamePage } from './app.po';

describe('my-project-name App', () => {
  let page: MyProjectNamePage;

  beforeEach(() => {
    page = new MyProjectNamePage();
  });

  it('should display title', done => {
    page.navigateTo();
    page.getParagraphText()
      .then(msg => expect(msg).toContain('Angular 4 Hello World Quickstart'))
      .then(done, done.fail);
  });
});

the e2e test is successful:

(dockerhost)$ protractor-headless
[00:46:32] I/direct - Using ChromeDriver directly...
[00:46:32] I/launcher - Running 1 instances of WebDriver
Spec started
[00:46:38] W/element - more than one element found for locator By(css selector, app-root h1) - the first result will be used

  my-project-name App
    ? should display title

Executed 1 of 1 spec SUCCESS in 1 sec.
[00:46:38] I/launcher - 0 instance(s) of WebDriver still running
[00:46:38] I/launcher - chrome #01 passed

We have achieved our goal: we had intended to perform a successful end to end test for a dynamic web page that retrieves its content from an external resource like the WordPress API.

Summary

In this blog post, we

• have applied a Protractor Docker Container on an existing Angular CLI Hello World.
  For this to work, we had to adapt the Protractor configuration file to circumvent an „Error while waiting for Protractor to sync with the page“
• have applied a Protractor Docker Container with the patched Protractor configuration on an app that is dynamically downloading and displaying content retrieved from the WordPress API. This has worked as expected although the content we have looked for was not visible in the HTML source code.

With the latter, we have successfully verified that the Protractor Docker container can handle dynamic content retrieved and altered via javascript.

Next Steps

• Write more complete test cases for the dynamic web page
• Refactor the dynamic web page application by dividing the HTTP GET into a separate service. The Protractor tests will help me to verify that the refactored application will keep its previously achieved features.

References:

• Thanks to webnicer a.k.a. jciolek, who has offered his Docker image and corresponding Git repo jciolek/docker-protractor-headless to the open source community
• Blog post describing the steps towards a dynamic Web application that is retrieving content from the WordPress API