How to do end-to-end testing with Cypress?

As the name implies, we will focus on E2E tests using the cypress tool. Why did we do it? Within our company, we have been thinking for a long time how to improve internal processes, increase efficiency in case of major changes in the application, and at the same time bring added value for our customers in the field of automated testing. In the article, we will focus on basic information about E2E tests and the subsequent practical use of cypress.

Integration tests

Integration testing is a technique where individual software modules are tested as a group. Applications are made up of different modules, for example in the case of React, components that can be implemented by different members of the software team, which also leads to a higher code error rate. The purpose of the testing level is to detect errors in the interaction between these modules. It focuses on checking data communication and integrity between components.

End to End tests

E2E testing is a type of testing in which you test your web application using a web browser up to the back-end itself, you can also test integrations with third-party API services. This kind of testing is great for making sure your app is working as a whole. At the same time, these tests can also be used as integration tests. As we can see in the image below, with these tests, the complexity, time requirement increases and the speed also decreases. On the other hand, they are certainly significantly more efficient than manual testing. After the initial hassles and setup of the test environment, all the disadvantages mentioned above are minimized if there are no major changes during development. The recommendation is to write the tests only at the end of the application development, if major changes are expected during this process that would significantly affect the already written tests, and thus prolong the time of closing the testing and deploying the application to production.

‍

At the beginning, it is more difficult to set up and prepare the entire infrastructure so that everything works correctly. It is necessary to pay attention to the chosen strategy and the quality of the tests themselves. In the case of complex test scenarios, I recommend thinking carefully about the structure and organization. If you have already completed all these initial settings, the advantage is that you have the application tested automatically if you have chosen integration into your CI/CD infrastructure, also from a UX point of view.

‍

Examples of scenarios:

1. Validation of critical application screens such as login and registration.
2. Validation of data so that it is persistent and displayed consistently across multiple pages of your application.
3. Automating tests within pipelines and subsequent validation before application deployment.

🧪 Cypress

Cypress is a next-generation testing tool for modern front-end applications. It primarily addresses key issues that developers and QA engineers face when testing applications. It allows writing, running, managing and debugging unit, integration and e2e tests.

‍

Why did we choose cypress?

As the first tool, we chose selenium, with which cypress is often compared. We had planned to try another tool so that we could compare all the advantages and disadvantages. That's why we also reached for cypress. As a result, cypress provides more options, while having fewer limits than selenium. Cypress is also architecturally different from Selenia and therefore allows you to write faster, simpler and more reliable tests.

Statement on the official website: "Until now, comprehensive testing has not been easy. It was the part the developers hated. Cypress makes it easy to set up, write, run and debug tests." so I can only confirm. I recommend Cypress, it makes testing easy and works very well. 🙂

🔧 Installation and initial settings

Information on basic settings and installation can be found here.

Testing strategies

Before each test, it is necessary to have the test data stored, for example, in a database. There are many procedures for preparing this data before the test. In this article, we will further show basic information about two of them, namely "Seeding data" and "Stubbing server".

Seeding data

The classic way when using e2e. Dynamic or static generation of data and associations. In cypress, all static test data can be found in the fixtures folder in JSON format. You can logically divide this data into individual files based on the context of the tests. Using this data, you can create, for example, a user entity and send a POST request to your server, which will process it and store it in the database. Subsequently, you can query the page, depending for example on the identifier.

cy.visit(`/cars/${carID}`)

‍

Three functions are available to fill in this data:

1. cy.exec() - to run system commands
2. cy.task() - to run code in Node via the setupNodeEvents function
3. cy.request() - for making server HTTP requests

The easiest way is to use the cy.request() function, which creates a request with test data and sends it to the server. See example below…

describe('Update car', () =>{
  cy.request ('POST', '/test-api/cars/${carID}', {
    brand: Volvo
    model: V60
    yearOfProduction: 2020
  })
})

‍

This approach increases complexity. You will struggle with synchronizing state between your server and frontend application. You will always have to set or delete this state before the tests, which slows down the actual execution of the tests, so you have to think about it and not forget it.

Stubbing server - aka ignoring responses from the server

Another valid approach that is the opposite of "seeding data". This is an approach where you replace the response from the server with your generated data, thus ignoring the data you received from the server, bypassing your backend server entirely. This means that instead of resetting or populating the database to the desired state, you can force the server to respond with whatever you want. In this way, we not only avoid the need to synchronize the state between the server and the frontend, but also prevent the state from mutating between individual tests. This means that the tests will not create a state that could affect other tests. One of the other benefits is that it allows you to build your application without needing to have the server running, which is more useful for integration tests. Even though stubbing is a faster technique, it also has its disadvantages, for example, that you have no guarantees whether the data you "push" corresponds to what the server actually sends. Another option within this technique is to generate test data in advance on the server for each test, and then receive this data, where you avoid the problem described above.

A more balanced approach is to integrate both strategies. In the official documentation, they recommend using the "seeding data" technique for the main scenario. Once you find that this test works, you can use the "stubbing" technique to test all edge cases and other scenarios. Using real data in the vast majority of cases does not bring any benefits. The recommendation is that the vast majority of tests use static data. Test execution will be orders of magnitude faster and much less complex.

‍

Best practices

1. 🗃️ Organization of tests, 🔐 Login/Authorization, 🎛️ Controlling state

❌ Share objects between individual pages/tests, use UI to login. 
✅Test individual pages/elements in isolation, use API service to login to your app. Check individual states and data only within the given test.

‍

If you want to know more check out this video.

2. Elements selection

❌ Using non-specific "tags" to select individual elements that may be subject to change, for example in the case of a UI library update. 
✅ Use the data-* attribute to provide context to your selectors and isolate them from changes.

In the official cypress documentation, they state that these selectors data-cy, data-test, data-testid are standard in the case of a unique element identifier.

<button
  id="main"
  calss="btn btn-large"
  name="submission"
  role="button"
  data-cy="submit"
>
  Submit
</button>

Above is an example of a button element. Individual cases of cypress commands for selecting the button element can be seen in the table below.

‍

3. 🏢Test external third-party services

❌ As part of the test, try to "visit" and interact with a third-party application that you do not have under your control. 
✅ Only test what you have under your control. Try to avoid testing third-party services. If necessary, always use cy.request() to communicate with third-party servers through their APIs.

If it will be necessary to access third-party servers as part of the tests, for example in the following cases:

1. Testing login/registration when your application uses third-party services for authorization
2. Checking whether, for example, the data sent by your application to a third-party server has been updated
3. To check the sending of email notifications for which you use third-party services

as mentioned above, always use the cypress function cy.request() for these purposes.

📧  Verification of sent emails/notifications

If your application sends emails directly through your SMTP server, you can use a temporary local test SMTP server running in Cypress. See the blog post "Testing HTML Emails with Cypress" for details.

‍

In this case, it is necessary to add the SMTP server settings to the "plugins" index file. Example setup below.

const ms = require('smtp-tester')

/**
 * @type {Cypress.PluginConfig}
 */
export default (on: any, config: any) => {
	// starts the SMTP server at localhost:7777
	const port = 7777
	const mailServer = ms.init(port)
	console.log('mail server at port %d', port)

	// process all emails
	mailServer.bind((addr, id, email) => {
		console.log(addr, id, email)
	})
...

If your application uses a third-party email service, you can use the test mailbox by accessing the API. For details, see the blog post "Full HTML Email Testing Using SendGrid and Ethereal Accounts".

Automation and integration into bitbucket pipelines

For the integration and automatic launch of e2e tests, we have chosen the procedure of launching the entire infrastructure of the BE, FE project and all necessary services, such as the database, using the docker compose tool. One command docker-compose up -d starts the build and configuration of the entire application infrastructure in the background. All the instructions for this process are in the docker-compose.yml file. More information about docker compose settings and options can be found here. Subsequently, tests are run on this instance using the cypress tool. You can run these tests at timed intervals. We run them once a day for the development branch. You can see an example of the output of the test results in the bitbucket console below.

If the tests reveal an error or problem, subsequent debugging and debugging will be facilitated by videos or screenshots that you can download and view in the "Artifacts" folder, see picture below. But it is necessary to set this file saving in your docker-compose.yml file. This way you can also save statements from your BE service. In our case, you have access to the saved files 14 days after the execution of the pipeline.

‍

...
artifacts: # store cypress images, videos and BE logs
  - cypress/screenshots/**
  - cypress/videos/**
  - backend/logs/**
...

Practical demonstration

An example of a test suite for registering and authorizing users to the application. As part of the tests, we use our own selectors, in which the necessary logic for component selection and other necessary operations is implemented. You can find these selectors in our company library for form components, which you can find here 📘.

context('Auth', () => {
	it('Sign up', () => {
		cy.clearLocalStorage()
		cy.intercept({
			method: 'POST',
			url: '/test-api/registration'
		}).as('registration')
		cy.visit('/signup')
		cy.setInputValue(FORM.REGISTRATION, 'email', `${generateRandomString(5)}_${user.emailSuffix}`)
		cy.setInputValue(FORM.REGISTRATION, 'password', user.password)
		cy.setInputValue(FORM.REGISTRATION, 'phone', user.phone)
		cy.clickButton('gdpr', FORM.REGISTRATION, true)
		cy.clickButton('marketing', FORM.REGISTRATION, true)
		cy.get('form').submit()
		cy.wait('@registration').then((interception: any) => {
			// check status code of registration request
			expect(interception.response.statusCode).to.equal(200)
			// take local storage snapshot
			cy.saveLocalStorage()
		})
		// check redirect to activation page
		cy.location('pathname').should('eq', '/activation')
	})

	it('Sign out', () => {
		cy.restoreLocalStorage()
		cy.intercept({
			method: 'POST',
			url: '/test-api/logout'
		}).as('authLogout')
		cy.visit('/')
		cy.get('.noti-my-account').click()
		cy.get('#logOut').click()
		cy.wait('@authLogout').then((interception: any) => {
			// check status code of logout request
			expect(interception.response.statusCode).to.equal(200)
			// check if tokens are erased
			assert.isNull(localStorage.getItem('refresh_token'))
			assert.isNull(localStorage.getItem('access_token'))
		})
		// check redirect to login page
		cy.location('pathname').should('eq', '/login')
	})

	it('Sign in', () => {
		cy.clearLocalStorage()
		cy.intercept({
			method: 'POST',
			url: '/test-api/login'
		}).as('authLogin')
		cy.visit('/login')
		cy.setInputValue(FORM.LOGIN, 'email', Cypress.env('auth_email'))
		cy.setInputValue(FORM.LOGIN, 'password', Cypress.env('auth_password'))
		cy.get('form').submit()
		cy.wait('@authLogin').then((interception: any) => {
			// check status code of login request
			expect(interception.response.statusCode).to.equal(200)
			// take local storage snapshot
			cy.saveLocalStorage()
		})
		// check redirect to home page
		cy.location('pathname').should('eq', '/')
	})
})

Conclusion

I believe this article has introduced you to E2E testing with cypress and will make your work easier if you decide to do E2E tests.​​