Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added contents for testing of wasmedge #197

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/contribute/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ To help new contributors understand WasmEdge development workflow, this guide wi

- [Build WasmEdge and WasmEdge plug-in from source on different platforms](/category/build-wasmedge-from-source)
- [WasmEdge Plug-in system introduction](/category/wasmedge-plugin-system)
- [Test WasmEdge](test.md)
- [Test WasmEdge](./test/testing.md)
- [WasmEdge Fuzzing](fuzzing.md)
- [WasmEdge internal explanation](internal.md)
- [WasmEdge installer system explanation](installer.md)
Expand Down
10 changes: 0 additions & 10 deletions docs/contribute/test.md

This file was deleted.

8 changes: 8 additions & 0 deletions docs/contribute/test/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"label": "Testing WasmEdge",
"position": 2,
"link": {
"type": "generated-index",
"description": "Here's the guide focused on testing WasmEdge"
}
}
72 changes: 72 additions & 0 deletions docs/contribute/test/example.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
sidebar_position: 2
---


# Example Test Explaination

The provided code snippet is a C++ test suite for a WebAssembly module, specifically focusing on a logging functionality within the WasmEdge runtime environment. It uses Google Test (gtest) for structuring and executing the tests. Let's break down the code into its key components for a clearer understanding:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the code source?


## Includes and Namespace

The code begins with including necessary headers for the WasmEdge runtime, logging functionality, Google Test, and other standard libraries.

```cpp
Copy code
#include "common/defines.h"
#include "runtime/instance/module.h"
#include "wasi_logging/func.h"
#include "wasi_logging/module.h"

#include <gtest/gtest.h>
#include <iostream>
#include <spdlog/sinks/ostream_sink.h>
#include <spdlog/spdlog.h>
#include <sstream>

namespace {
// Helper functions and test cases are defined here.
}
```
### Helper Functions

Two helper functions are defined within an unnamed namespace, making them local to this file.

1. `createModule`: This function loads a WasmEdge plugin for the wasi_logging module and creates an instance of it. If successful, it returns a pointer to the module instance; otherwise, it returns nullptr.

2. `fillMemContent`: These overloaded functions are used to fill the memory of a WebAssembly module instance. One fills a specified memory area with a byte value, and the other fills it with a string.

### Test Case: WasiLoggingTests

The `TEST` block defines a test case for the logging functionality of the WasmEdge runtime.

1. Module Instance Creation: It creates an instance of the WasiLoggingModule using the createModule function.

2. Memory and Calling Frame Setup: The test sets up a memory instance and a calling frame, which are necessary for executing WebAssembly functions.

3. Memory Preparation: The memory is prepared with specific content using fillMemContent, setting up the required state for the test.

4. Function Retrieval and Execution: The test retrieves the log function from the module and executes it with various parameters to test different logging levels and contexts.

5. Assertions: Throughout the test, EXPECT_TRUE and EXPECT_FALSE assertions from Google Test are used to check if the outcomes of the operations are as expected.

### Main Function

The main function initializes the Google Test framework and runs all the tests defined in the test suite.

```cpp
Copy code
GTEST_API_ int main(int argc, char **argv) {
testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
```

adithyaakrishna marked this conversation as resolved.
Show resolved Hide resolved
### Summary

This code is a comprehensive test suite for a WebAssembly logging module in the WasmEdge runtime. It demonstrates how to set up a WebAssembly environment in C++, load modules, manipulate memory, execute module functions, and validate their behavior using the Google Test framework. The focus is on testing the functionality of a specific log function within the WasiLoggingModule, ensuring it behaves correctly across different scenarios.





138 changes: 138 additions & 0 deletions docs/contribute/test/testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
---
sidebar_position: 1
---

# Testing

This module will teach you about writing tests for Web-Assembly (Wasm) Code for adding test suites for new plugins or modules for WasmEdge Software. This is an important aspect if you want to submit plugins to the WasmEdge community and we would require you to write test suites to cater to the plugin functionalities.

## Writing Tests for Wasm Code Targeting WebAssembly

Testing is a crucial part of software development, ensuring that code behaves as expected. In this guide, we'll explore how to write tests for Wasm code, using Google Test (gtest) and a specific example related to the WasmEdge runtime.

### Prerequisites
- Familiarity with C++ and WebAssembly.
- Basic understanding of Google Test (gtest) framework.
- WasmEdge runtime environment set up for testing.

### Code Explanation and Comments

The provided code snippet demonstrates how to set up and execute tests for a WebAssembly code or the plugins you create, specifically using the WasmEdge runtime. The code is structured into two main parts: the test setup and the test cases themselves.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From here, what's the difference between example.md and testing.md

```cpp
#include "common/defines.h"
#include "runtime/instance/module.h"
#include "wasi_logging/func.h"
#include "wasi_logging/module.h"

#include <gtest/gtest.h>
#include <iostream>
#include <spdlog/sinks/ostream_sink.h>
#include <spdlog/spdlog.h>
#include <sstream>

namespace {

// Function to create a module instance for testing.
WasmEdge::Runtime::Instance::ModuleInstance *createModule() {
// ... Code omitted for brevity ...
}

// Helper function to fill memory with a byte value.
void fillMemContent(WasmEdge::Runtime::Instance::MemoryInstance &MemInst,
uint32_t Offset, uint32_t Cnt, uint8_t C = 0) noexcept {
// ... Code omitted for brevity ...
}

// Helper function to fill memory with a string.
void fillMemContent(WasmEdge::Runtime::Instance::MemoryInstance &MemInst,
uint32_t Offset, const std::string &Str) noexcept {
// ... Code omitted for brevity ...
}

} // namespace
```

### Test Cases

```cpp
TEST(WasiLoggingTests, func_log) {
// ... Code omitted for brevity ...

// Test cases for different logging levels and conditions.
// Each test case uses the `EXPECT_TRUE` or `EXPECT_FALSE` macros
// to validate the behavior of the `HostFuncInst.run` method.

// ... Code omitted for brevity ...
}
```

You can call the main function to run all the tests using the below given example

```cpp
GTEST_API_ int main(int argc, char **argv) {
testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
```

## Where to add the tests?

After writing tests for your specific modules or plugins, you can add them under `https://github.com/WasmEdge/WasmEdge/tree/master/test` and for more specific placement of the tests,

For example,

- You can add the tests for new plugins or update existing ones under `https://github.com/WasmEdge/WasmEdge/tree/master/test/plugins`

## Documenting the Tests in Markdown
adithyaakrishna marked this conversation as resolved.
Show resolved Hide resolved

When documenting these tests in Markdown, you should focus on explaining the purpose of each test, the setup required, and the expected outcomes. Here's an example of how you might structure this documentation:


```md

# WasmEdge Module Tests

## Overview

This document outlines the tests for the `WasiLogging` module in a WebAssembly environment using the WasmEdge runtime.

## Test Setup

- **Module Creation**: A module instance is created for testing purposes using the `createModule` function.
- **Memory Manipulation**: The `fillMemContent` functions are used to manipulate memory content, essential for setting up test scenarios.

## Test Cases

### `func_log` Tests

- **Purpose**: These tests validate the logging functionality at different levels and conditions.
- **Test Scenarios**:
- **Clear Memory**: Ensures memory is correctly cleared before each test.
- **Set Strings in Memory**: Tests the ability to write strings to memory.
- **Logging at Different Levels**: Validates logging behavior at various levels (info, warning, error, etc.).
- **Stderr Context**: Checks if the logging correctly handles the stderr context.
- **Unknown Level**: Ensures that an unknown logging level is handled appropriately.

## Running Tests

- Compile the test code with your C++ compiler targeting WebAssembly.
- Run the tests using a command-line tool that supports Google Test.

## Conclusion

These tests ensure the robustness and reliability of the module in a WebAssembly environment

```

> Note: You can add the documentation in the plugin test folders you create

### Blogs
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I don't think we need this.


- [Google Test GitHub Repository](https://github.com/google/googletest): While not a traditional blog, the Google Test GitHub repository is a rich resource for learning about testing in C++
- [Modern C++ Testing with Catch2](https://www.jetbrains.com/help/clion/unit-testing-tutorial.html)
- [C++ Testing - Catch2 and Google Test](https://www.learncpp.com/cpp-tutorial/introduction-to-testing-your-code/): LearnCpp.com provides a tutorial on testing in C++ using frameworks like Catch2 and Google Test.

### Further Reading

- Here is an example test case explaination for [test/plugins/wasi_logging](./example.md)
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ To help new contributors understand WasmEdge development workflow, this guide wi

- [Build WasmEdge and WasmEdge plug-in from source on different platforms](/category/build-wasmedge-from-source)
- [WasmEdge Plug-in system introduction](/category/wasmedge-plugin-system)
- [Test WasmEdge](test.md)
- [Test WasmEdge](./test/testing.md)
- [WasmEdge Fuzzing](fuzzing.md)
- [WasmEdge internal explanation](internal.md)
- [WasmEdge installer system explanation](installer.md)
Expand Down
10 changes: 0 additions & 10 deletions i18n/zh/docusaurus-plugin-content-docs/current/contribute/test.md

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"label": "Testing WasmEdge",
"position": 2,
"link": {
"type": "generated-index",
"description": "Here's the guide focused on testing WasmEdge"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
sidebar_position: 2
---


# Example Test Explaination

The provided code snippet is a C++ test suite for a WebAssembly module, specifically focusing on a logging functionality within the WasmEdge runtime environment. It uses Google Test (gtest) for structuring and executing the tests. Let's break down the code into its key components for a clearer understanding:

## Includes and Namespace

The code begins with including necessary headers for the WasmEdge runtime, logging functionality, Google Test, and other standard libraries.

```cpp
Copy code
#include "common/defines.h"
#include "runtime/instance/module.h"
#include "wasi_logging/func.h"
#include "wasi_logging/module.h"

#include <gtest/gtest.h>
#include <iostream>
#include <spdlog/sinks/ostream_sink.h>
#include <spdlog/spdlog.h>
#include <sstream>

namespace {
// Helper functions and test cases are defined here.
}
```
### Helper Functions

Two helper functions are defined within an unnamed namespace, making them local to this file.

1. `createModule`: This function loads a WasmEdge plugin for the wasi_logging module and creates an instance of it. If successful, it returns a pointer to the module instance; otherwise, it returns nullptr.

2. `fillMemContent`: These overloaded functions are used to fill the memory of a WebAssembly module instance. One fills a specified memory area with a byte value, and the other fills it with a string.

### Test Case: WasiLoggingTests

The `TEST` block defines a test case for the logging functionality of the WasmEdge runtime.

1. Module Instance Creation: It creates an instance of the WasiLoggingModule using the createModule function.

2. Memory and Calling Frame Setup: The test sets up a memory instance and a calling frame, which are necessary for executing WebAssembly functions.

3. Memory Preparation: The memory is prepared with specific content using fillMemContent, setting up the required state for the test.

4. Function Retrieval and Execution: The test retrieves the log function from the module and executes it with various parameters to test different logging levels and contexts.

5. Assertions: Throughout the test, EXPECT_TRUE and EXPECT_FALSE assertions from Google Test are used to check if the outcomes of the operations are as expected.

### Main Function

The main function initializes the Google Test framework and runs all the tests defined in the test suite.

```cpp
Copy code
GTEST_API_ int main(int argc, char **argv) {
testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
```

### Summary

This code is a comprehensive test suite for a WebAssembly logging module in the WasmEdge runtime. It demonstrates how to set up a WebAssembly environment in C++, load modules, manipulate memory, execute module functions, and validate their behavior using the Google Test framework. The focus is on testing the functionality of a specific log function within the WasiLoggingModule, ensuring it behaves correctly across different scenarios.





Loading