Puff

Open source smoke testing platform

Puff is an open source smoke testing platform for students to collaboratively write and run tests on their assignment or project code for quick and easy sanity testing.

Getting started | Motivation | Supported Languages | Contributors

Motivation

Let's face it. Everyone writes their test cases last.

Unless you're someone who lives by TDD, you're like the rest of us lazy developers and write your test cases as the last part of your assignment. But before you start writing you'll run several rounds of sanity checks (smoke tests) to make sure your project works as expected.

Now imagine instead of only having the four quick cases you thought up, you also had the ones from your friends also working on the project. Or the ones from the entire class. These quick and dirty smoke tests can help you rat out bugs before you start writing out the fancy test suite you're going to pretend you used to test your assignment.

We all want to make sure our assignment actually performs according to the requirements before we submit it. After all, most of your marks come from the behavior of your code, not the test case writeup.

Getting started

Puff's backend is designed using a microservice architecture with Spring Boot written in Java. Microservices can be built using Maven.

Puff has 3 microservices that make up its backend:

• course-management manages courses, assignments and user actions
• test-runner runs test cases and manages user code uploads
• user-management manages user permissions and access controls

See each repo for instructions on setting up and running the microservice locally.

Setup Keycloak

Keycloak is an open source Identity and Access Management solution aimed at modern applications and services. It makes it easy to secure applications and services with little to no code.

Puff uses keycloak as a user management and authentication solution. More information about Keycloak can be found on their offical docs page. In order to authenticate and send requests locally you'll need to setup your own keycloak server:

As a Docker container:

• Download the realm settings, configuration and clients using the realm.json file located in the keycloak folder.

docker run -p 8180:8080 \
 --name keycloak \
 -e KEYCLOAK_USER=admin \
 -e KEYCLOAK_PASSWORD=pass \
 -e KEYCLOAK_IMPORT=/tmp/realm.json \
 -v "$(pwd)/keycloak/realm.json:/tmp/realm.json:ro" \
 jboss/keycloak

This will set your admin credentials to be admin and pass and import the Puff realm.

• If you run into problems, make sure you have the proper path to the realm.json file when running the Docker command.

Locally with the standalone server:

• Download and install the standalone server from https://www.keycloak.org/downloads.html.
• Navigate to your installed keycloak directory and run the following command:

Linux/Mac

./bin/standalone.sh -Djboss.socket.binding.port-offset=100

Windows

./bin/standalone.bat -Djboss.socket.binding.port-offset=100

• Setup your admin account by navigating to http://localhost:8180.
• Navigate to the admin portal from http://localhost:8180/auth/admin
• When you log in you should be on the Master realm. Hover over the dropdown arrow and then click Add realm.
• Import the realm settings, configuration and clients using the realm.json file located in the keycloak folder.

Setup test accounts

• Create at least two test user accounts. They will automatically be added to the Users group. You'll want to add at least one use to the Administrators group to test. You will need to reset their passwords once created.
• You can optionally add the sys role to a user to test system-secured endpoints.
• View the account page of a test user at http://localhost:8180/auth/realms/puff/account/.
• If you have already generated a secret for both user-auth and puff-service-acc skip this step. Otherwise, navigate to the admin portal from http://localhost:8180/auth/admin and log in. Once logged in, navigate to Clients > user-auth > Credentials and click Regenerate Secret. Repeat this for Clients > puff-service-acc, replacing the new secrets in your application-local.yml files as needed.
• Generate an authentication token by making the following curl call replacing TEST_USER_USERNAME, TEST_USER_PASSWORD and USER_AUTH_CLIENT_SECRET with the credentials for the test accounts you created and the client-secret for user-auth.

curl -X POST 'http://localhost:8180/auth/realms/puff/protocol/openid-connect/token' \
 --header 'Content-Type: application/x-www-form-urlencoded' \
 --data-urlencode 'grant_type=password' \
 --data-urlencode 'client_id=user-auth' \
 --data-urlencode 'client_secret=USER_AUTH_CLIENT_SECRET' \
 --data-urlencode 'username=TEST_USER_USERNAME' \
 --data-urlencode 'password=TEST_USER_PASSWORD'

Development

Swagger

Puff's Spring-Boot backend exposes a REST API. The project utilizes Swagger to document and keep a consistent REST interface.

Once you have a microservice running visit the following urls to see the REST APIs for the respective microservices:

• Course-management: http://localhost:8080/swagger-ui.html
• Test-runner: http://locahost:8083/swagger-ui.html
• User-management: TBD

A json api version to be consumed and used to generate client libraries can be accessed at http://localhost:XXXX/v2/api-docs.

Select Authorize and login with a test user account to try out any of the endpoints.

H2 Database

Puff's Spring-boot backend uses a H2 runtime database to simulate a database connection for local development. Once the microservice is running its database can be accessed at

• Course-management: http://localhost:8080/h2
• Test-runner: http://localhost:8083/h2
• User-management: TBD

The credentials for the database are as follows:

Driver Class: org.h2.Driver
JDBC URL: jdbc:h2:mem:testdb
User Name: admin
Password:

For more information about H2 databases see the H2 Database Engine.

Docs

Included in the docs repository are documents with the planning for API endpoints specifications and entity relation diagrams for UnityTest's data model.

More information will be added as the project matures.

Supported languages

The Puff testing platform plans to support projects written in the following languages:

Haskell

Quickcheck and simple GHCi cases.

Python

Tests written using pytest.

Java

Tests written using JUnit.

To come

C, C++, MySQL

Contributors

The Puff project is looking for contributors to join the initiative! For information about progress, features under construction and opportunities to contribute see our project board.

If you're interested in helping please read our CONTRIBUTING.md for details like our Code of Conduct and contact Benjamin Kostiuk for more information.