h05 - Integration and E2E Testing

This homework assignment is designed to introduce you to the ideas of integration tests and end-to-end tests, as illustrated in this diagram, often called the “testing pyramid”:

testing pyramid

Briefly, the idea is that:

Unit tests test only a single unit, so in principle, absolutely everything outside that unit should be mocked (including network services, database calls, etc.)
For an integration test, you are testing the integration of multiple units, so it’s ok to not mock some parts of the interaction. For example, an integration test of a controller might look exactly like the unit test of a controller, except that instead of mocking the database calls, you actually let the database calls happen to a real instance of the database (albeit one that is carefully controlled), and then you do assertions on the before and after state of the database tables.
For an end-to-end test, you treat the entire application (to the extent possible) as a “black box”, meaning that the tests typically have no knowledge of “what’s inside the box”. You use browser automation software to interact with an instance of a real browser (Firefox, Chrome, Edge, Safari, etc.) and to act like a “robot” that is clicking buttons and entering data on real web pages, and then making assertions about the content of the page that shows up.

You can read more about the testing pyramid in a dedicated article here:

https://ucsb-cs156.github.io/topics/testing/testing_pyramid.html

If you were aware of the testing pyramid prior to taking this course, you might have noticed that unit tests exist in every assignment given in this course, but integration and end-to-end tests do not…yet. While the rolling in of the rest of the testing pyramid is in progress and while we decide how to best integrate these topics directly into the course material, this assignment is designed to be a brief introduction to these testing topics that you will likely encounter in real-world software development.

In this homework exercise, you’ll do five things, working in the team repo from team03. While you are working in the team repo, this is an individual homework.

Run an existing integration test and look at the code.
Run an existing end-to-end test in headless mode (the normal way)
Run an existing end-to-end test in not headless mode (for development and debugging).
Develop an integration test of your own for just one of the controller methods from the controller you built during the team02 phase of the project that changes the database in some way (either create, update or delete; your choice).
Develop an end-to-end test for either create, update or delete for the user interface code you developed for a CRUD operations that changes the database (either create, update or delete, your choice).
Make a pull request containing these two tests.
Submit a link to the pull request on Canvas.

Part 1.1: Getting Started

Again, for this assignment you will be working in your team03 repos, but is an individual homework assignment.

Start by going into the directory where you cloned your team03 repo.

Click triangle for links to the repos:

4pm	5pm
team03-s24-4pm-1	team03-s24-5pm-1
team03-s24-4pm-2	team03-s24-5pm-2
team03-s24-4pm-3	team03-s24-5pm-3
team03-s24-4pm-4	team03-s24-5pm-4
team03-s24-4pm-5	team03-s24-5pm-5
team03-s24-4pm-6	team03-s24-5pm-6
team03-s24-4pm-7	team03-s24-5pm-7
team03-s24-4pm-8	team03-s24-5pm-8

Begin by making a branch for this homework assignment.
Then, update that branch by merging in the code in https://github.com/ucsb-cs156-s24/STARTER-team03 again. You will need to do this because we made a few bug fixes to the code that supports integration and end-to-end testing after most of your teams cloned the code for team03.

You may already have a remote for this called starter; you can check by typing: git remote -v and seeing if you have a remote called starter. If not, define one:
```
git remote add starter https://github.com/ucsb-cs156-s24/STARTER-team03
```
Then do:
```
git fetch starter
git merge starter/main
```
It is possible that there may be merge conflicts at this stage that you have to resolve; if so resolve those.

Then push to your new branch:
```
git push origin your-branch-name
```

Now you are ready to try running the integration and end-to-end tests that are already in the starter code (instructions in next step.)

Part 1.2: Understanding integration and end-to-end tests

Before you begin working on the integration and end-to-end tests for the code you wrote in team02 and team03, you’ll need to know how to run the tests that already exist in the codebase.

The starter code contains a number of Java test files, under the integration and web directories. These files use a couple of new tools that are explained in the testing pyramid article here. All of the integration and end-to-end tests contain the keyword IT which is similar to the keyword Test that appears at the end of each unit test file, and we use Web to distinguish end-to-end test from integration tests.

The RestaurantIT integration test class which contains the integration tests for the Restaurants backend. The structure of these tests are incredibly similar to the structure of the tests in RestaurantControllerTests, with the difference being that the integration test uses a real RestaurantRepository instead of a mocked one.
The HomePageWebIT end-to-end test class that contains the minimum configuration necessary for running an end-to-end test within our stack. This test uses Playwright to assert whether or not the home page of our application contains the text that we expect.
The OauthWebIT which tests our mocked authentication endpoint so that we can login like we normally do with Google Oauth but with fake user information.
The RestaurantWebIT which contains the example of an end-to-end test for Restaurants.
WebTestCase which is the parent class of OauthWebIt and RestaurantWebIT and contains much of the shared code and setup that is needed for the end-to-end tests.

These test files just contain a selection of examples of what kind of tests you could write for the backend and frontend that you worked on in team02 and team03. This assignment is relatively open-ended and any test that satisfies the submission criteria will be sufficient.

Part 1.3: Running the tests

You may be used to running mvn test in order to run the test suite for the application, but integration and end-to-end tests run with a seperate command.

In order to run the integration and end-to-end test suite you should use the following series of commands.

mvn clean

To make sure that we do not have anything lingering from previous test runs. Running mvn clean is important because the tests are highly sensitive and can fail if this is not done before the next steps.

INTEGRATION=true mvn test-compile

This step is this test compile command that has this INTEGRATION=true command at the front. What this does is it specifies that the program should run in the profile for integration tests. This command may take a while because it compiles the frontend with the backend.

INTEGRATION=true mvn failsafe:integration-test

This command actually runs the test suite. The test suite should pass.

If you have previously gone through these three commands and have ONLY modified the test cases then you may just use the last command,INTEGRATION=true mvn failsafe:integration-test. Otherwise, you may need to recompile.

If you want to just run a single integration/end-to-end test (e.g. only HomePageWebIT.java) use -Dit.test=ClassName, for example:

INTEGRATION=true mvn test-compile failsafe:integration-test -Dit.test=HomePageWebIT

A note about end-to-end tests: they can typically be done in two modes:

“headless”, where there is no real browser being rendered on a screen; it’s all just simulated in memory. This is the usual way of running end-to-end tests because its a lot faster.
“not headless” where the tester can actually watch all of the interactions happen on screen (albeit very quickly) as the tests are being run. This is typically only used when developing or debugging the end-to-end tests.

Our tests run “headless” by default and you can configure the tests to run “not headless” with the following:

INTEGRATION=true HEADLESS=false mvn failsafe:integration-test

Part 2: Your task

After understanding how to run the tests, you can begin working on the tests using the examples for the parts of the application that you work on in team01 and team02.

Integration Test	End-to-end Test
`UCSBDiningCommonsMenuItem.java`	`UCSBDiningCommonsMenuItemWebIT.java`
`UCSBOrganizationIT.java`	`UCSBOrganizationWebIT.java`
`RecommendationRequestIT.java`	`RecommendationRequestWebIT.java`
`MenuItemReviewIT.java`	`MenuItemReviewWebIT.java`
`HelpRequestIT.java`	`HelpRequestWebIT.java`
`ArticleIT.java`	`ArticleWebIT.java`

There are two test files that you must write as seen in the table above. It is okay to emulate the examples in RestaurantIT.java and RestaurantWebIT.java but it is not a requirement to follow them one-to-one, and depending on your implementation you may not be able to.

Create a new branch from main (assuming that all PRs from team03 are merged into main) and begin working on the tests.

For both tests, you may choose any controller method you implemented in team02 to test or any frontend feature from team03. You are not required to cover every method or every frontend feature.

NOTE: For the end-to-end test, if your frontend form contains a date picker, you may have trouble selecting a date. If you can figure out how to do it with the Playwright Actions, great. If not, you can try to do something similar to the integration test, where you save something to the database using the repository directly.

Part 3: Submission

Once you have developed both an _IT.test and a _WebIT.test, make a pull request containing both, but you do not need to merge it or obtain reviews.

There are no additional metrics like coverage or mutation testing, as long as the tests passed with the commands in Step #1.3 then that is sufficient.

When the PR passes all the checks, you can submit a link to the PR on canvas at https://ucsb.instructure.com/courses/20058/assignments/243482.