diff --git a/docs/contribute/overview.md b/docs/contribute/overview.md index 524d85e1..0614802e 100644 --- a/docs/contribute/overview.md +++ b/docs/contribute/overview.md @@ -11,7 +11,7 @@ To help new contributors understand WasmEdge development workflow, this guide wi - [Build WasmEdge and WasmEdge plug-in from source on different platforms](/category/build-wasmedge-from-source) - [WasmEdge Plug-in system introduction](/category/wasmedge-plugin-system) -- [Test WasmEdge](test.md) +- [Test WasmEdge](./test/testing.md) - [WasmEdge Fuzzing](fuzzing.md) - [WasmEdge internal explanation](internal.md) - [WasmEdge installer system explanation](installer.md) diff --git a/docs/contribute/test.md b/docs/contribute/test.md deleted file mode 100644 index ad778399..00000000 --- a/docs/contribute/test.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -sidebar_position: 4 ---- - -# Testing - - -:::info -Work in Progress -::: diff --git a/docs/contribute/test/_category_.json b/docs/contribute/test/_category_.json new file mode 100644 index 00000000..6d9a49b3 --- /dev/null +++ b/docs/contribute/test/_category_.json @@ -0,0 +1,8 @@ +{ + "label": "Testing WasmEdge", + "position": 2, + "link": { + "type": "generated-index", + "description": "Here's the guide focused on testing WasmEdge" + } +} diff --git a/docs/contribute/test/example.md b/docs/contribute/test/example.md new file mode 100644 index 00000000..e35a4ede --- /dev/null +++ b/docs/contribute/test/example.md @@ -0,0 +1,72 @@ +--- +sidebar_position: 2 +--- + + +# Example Test Explaination + +The provided code snippet is a C++ test suite for a WebAssembly module, specifically focusing on a logging functionality within the WasmEdge runtime environment. It uses Google Test (gtest) for structuring and executing the tests. Let's break down the code into its key components for a clearer understanding: + +## Includes and Namespace + +The code begins with including necessary headers for the WasmEdge runtime, logging functionality, Google Test, and other standard libraries. + +```cpp +Copy code +#include "common/defines.h" +#include "runtime/instance/module.h" +#include "wasi_logging/func.h" +#include "wasi_logging/module.h" + +#include +#include +#include +#include +#include + +namespace { + // Helper functions and test cases are defined here. +} +``` +### Helper Functions + +Two helper functions are defined within an unnamed namespace, making them local to this file. + +1. `createModule`: This function loads a WasmEdge plugin for the wasi_logging module and creates an instance of it. If successful, it returns a pointer to the module instance; otherwise, it returns nullptr. + +2. `fillMemContent`: These overloaded functions are used to fill the memory of a WebAssembly module instance. One fills a specified memory area with a byte value, and the other fills it with a string. + +### Test Case: WasiLoggingTests + +The `TEST` block defines a test case for the logging functionality of the WasmEdge runtime. + +1. Module Instance Creation: It creates an instance of the WasiLoggingModule using the createModule function. + +2. Memory and Calling Frame Setup: The test sets up a memory instance and a calling frame, which are necessary for executing WebAssembly functions. + +3. Memory Preparation: The memory is prepared with specific content using fillMemContent, setting up the required state for the test. + +4. Function Retrieval and Execution: The test retrieves the log function from the module and executes it with various parameters to test different logging levels and contexts. + +5. Assertions: Throughout the test, EXPECT_TRUE and EXPECT_FALSE assertions from Google Test are used to check if the outcomes of the operations are as expected. + +### Main Function + +The main function initializes the Google Test framework and runs all the tests defined in the test suite. + +```cpp +Copy code +GTEST_API_ int main(int argc, char **argv) { + testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} +``` + +### Summary + +This code is a comprehensive test suite for a WebAssembly logging module in the WasmEdge runtime. It demonstrates how to set up a WebAssembly environment in C++, load modules, manipulate memory, execute module functions, and validate their behavior using the Google Test framework. The focus is on testing the functionality of a specific log function within the WasiLoggingModule, ensuring it behaves correctly across different scenarios. + + + + + diff --git a/docs/contribute/test/testing.md b/docs/contribute/test/testing.md new file mode 100644 index 00000000..af011313 --- /dev/null +++ b/docs/contribute/test/testing.md @@ -0,0 +1,138 @@ +--- +sidebar_position: 1 +--- + +# Testing + +This module will teach you about writing tests for Web-Assembly (Wasm) Code for adding test suites for new plugins or modules for WasmEdge Software. This is an important aspect if you want to submit plugins to the WasmEdge community and we would require you to write test suites to cater to the plugin functionalities. + +## Writing Tests for Wasm Code Targeting WebAssembly + +Testing is a crucial part of software development, ensuring that code behaves as expected. In this guide, we'll explore how to write tests for Wasm code, using Google Test (gtest) and a specific example related to the WasmEdge runtime. + +### Prerequisites +- Familiarity with C++ and WebAssembly. +- Basic understanding of Google Test (gtest) framework. +- WasmEdge runtime environment set up for testing. + +### Code Explanation and Comments + +The provided code snippet demonstrates how to set up and execute tests for a WebAssembly code or the plugins you create, specifically using the WasmEdge runtime. The code is structured into two main parts: the test setup and the test cases themselves. + +```cpp +#include "common/defines.h" +#include "runtime/instance/module.h" +#include "wasi_logging/func.h" +#include "wasi_logging/module.h" + +#include +#include +#include +#include +#include + +namespace { + +// Function to create a module instance for testing. +WasmEdge::Runtime::Instance::ModuleInstance *createModule() { + // ... Code omitted for brevity ... +} + +// Helper function to fill memory with a byte value. +void fillMemContent(WasmEdge::Runtime::Instance::MemoryInstance &MemInst, + uint32_t Offset, uint32_t Cnt, uint8_t C = 0) noexcept { + // ... Code omitted for brevity ... +} + +// Helper function to fill memory with a string. +void fillMemContent(WasmEdge::Runtime::Instance::MemoryInstance &MemInst, + uint32_t Offset, const std::string &Str) noexcept { + // ... Code omitted for brevity ... +} + +} // namespace +``` + +### Test Cases + +```cpp +TEST(WasiLoggingTests, func_log) { + // ... Code omitted for brevity ... + + // Test cases for different logging levels and conditions. + // Each test case uses the `EXPECT_TRUE` or `EXPECT_FALSE` macros + // to validate the behavior of the `HostFuncInst.run` method. + + // ... Code omitted for brevity ... +} +``` + +You can call the main function to run all the tests using the below given example + +```cpp +GTEST_API_ int main(int argc, char **argv) { + testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} +``` + +## Where to add the tests? + +After writing tests for your specific modules or plugins, you can add them under `https://github.com/WasmEdge/WasmEdge/tree/master/test` and for more specific placement of the tests, + +For example, + +- You can add the tests for new plugins or update existing ones under `https://github.com/WasmEdge/WasmEdge/tree/master/test/plugins` + +## Documenting the Tests in Markdown + +When documenting these tests in Markdown, you should focus on explaining the purpose of each test, the setup required, and the expected outcomes. Here's an example of how you might structure this documentation: + + +```md + +# WasmEdge Module Tests + +## Overview + +This document outlines the tests for the `WasiLogging` module in a WebAssembly environment using the WasmEdge runtime. + +## Test Setup + +- **Module Creation**: A module instance is created for testing purposes using the `createModule` function. +- **Memory Manipulation**: The `fillMemContent` functions are used to manipulate memory content, essential for setting up test scenarios. + +## Test Cases + +### `func_log` Tests + +- **Purpose**: These tests validate the logging functionality at different levels and conditions. +- **Test Scenarios**: + - **Clear Memory**: Ensures memory is correctly cleared before each test. + - **Set Strings in Memory**: Tests the ability to write strings to memory. + - **Logging at Different Levels**: Validates logging behavior at various levels (info, warning, error, etc.). + - **Stderr Context**: Checks if the logging correctly handles the stderr context. + - **Unknown Level**: Ensures that an unknown logging level is handled appropriately. + +## Running Tests + +- Compile the test code with your C++ compiler targeting WebAssembly. +- Run the tests using a command-line tool that supports Google Test. + +## Conclusion + +These tests ensure the robustness and reliability of the module in a WebAssembly environment + +``` + +> Note: You can add the documentation in the plugin test folders you create + +### Blogs + +- [Google Test GitHub Repository](https://github.com/google/googletest): While not a traditional blog, the Google Test GitHub repository is a rich resource for learning about testing in C++ +- [Modern C++ Testing with Catch2](https://www.jetbrains.com/help/clion/unit-testing-tutorial.html) +- [C++ Testing - Catch2 and Google Test](https://www.learncpp.com/cpp-tutorial/introduction-to-testing-your-code/): LearnCpp.com provides a tutorial on testing in C++ using frameworks like Catch2 and Google Test. + +### Further Reading + +- Here is an example test case explaination for [test/plugins/wasi_logging](./example.md) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/overview.md index 524d85e1..0614802e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/overview.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/overview.md @@ -11,7 +11,7 @@ To help new contributors understand WasmEdge development workflow, this guide wi - [Build WasmEdge and WasmEdge plug-in from source on different platforms](/category/build-wasmedge-from-source) - [WasmEdge Plug-in system introduction](/category/wasmedge-plugin-system) -- [Test WasmEdge](test.md) +- [Test WasmEdge](./test/testing.md) - [WasmEdge Fuzzing](fuzzing.md) - [WasmEdge internal explanation](internal.md) - [WasmEdge installer system explanation](installer.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test.md deleted file mode 100644 index ad778399..00000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -sidebar_position: 4 ---- - -# Testing - - -:::info -Work in Progress -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test/_category_.json b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test/_category_.json new file mode 100644 index 00000000..6d9a49b3 --- /dev/null +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test/_category_.json @@ -0,0 +1,8 @@ +{ + "label": "Testing WasmEdge", + "position": 2, + "link": { + "type": "generated-index", + "description": "Here's the guide focused on testing WasmEdge" + } +} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test/example.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test/example.md new file mode 100644 index 00000000..e35a4ede --- /dev/null +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test/example.md @@ -0,0 +1,72 @@ +--- +sidebar_position: 2 +--- + + +# Example Test Explaination + +The provided code snippet is a C++ test suite for a WebAssembly module, specifically focusing on a logging functionality within the WasmEdge runtime environment. It uses Google Test (gtest) for structuring and executing the tests. Let's break down the code into its key components for a clearer understanding: + +## Includes and Namespace + +The code begins with including necessary headers for the WasmEdge runtime, logging functionality, Google Test, and other standard libraries. + +```cpp +Copy code +#include "common/defines.h" +#include "runtime/instance/module.h" +#include "wasi_logging/func.h" +#include "wasi_logging/module.h" + +#include +#include +#include +#include +#include + +namespace { + // Helper functions and test cases are defined here. +} +``` +### Helper Functions + +Two helper functions are defined within an unnamed namespace, making them local to this file. + +1. `createModule`: This function loads a WasmEdge plugin for the wasi_logging module and creates an instance of it. If successful, it returns a pointer to the module instance; otherwise, it returns nullptr. + +2. `fillMemContent`: These overloaded functions are used to fill the memory of a WebAssembly module instance. One fills a specified memory area with a byte value, and the other fills it with a string. + +### Test Case: WasiLoggingTests + +The `TEST` block defines a test case for the logging functionality of the WasmEdge runtime. + +1. Module Instance Creation: It creates an instance of the WasiLoggingModule using the createModule function. + +2. Memory and Calling Frame Setup: The test sets up a memory instance and a calling frame, which are necessary for executing WebAssembly functions. + +3. Memory Preparation: The memory is prepared with specific content using fillMemContent, setting up the required state for the test. + +4. Function Retrieval and Execution: The test retrieves the log function from the module and executes it with various parameters to test different logging levels and contexts. + +5. Assertions: Throughout the test, EXPECT_TRUE and EXPECT_FALSE assertions from Google Test are used to check if the outcomes of the operations are as expected. + +### Main Function + +The main function initializes the Google Test framework and runs all the tests defined in the test suite. + +```cpp +Copy code +GTEST_API_ int main(int argc, char **argv) { + testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} +``` + +### Summary + +This code is a comprehensive test suite for a WebAssembly logging module in the WasmEdge runtime. It demonstrates how to set up a WebAssembly environment in C++, load modules, manipulate memory, execute module functions, and validate their behavior using the Google Test framework. The focus is on testing the functionality of a specific log function within the WasiLoggingModule, ensuring it behaves correctly across different scenarios. + + + + + diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test/testing.md b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test/testing.md new file mode 100644 index 00000000..af011313 --- /dev/null +++ b/i18n/zh/docusaurus-plugin-content-docs/current/contribute/test/testing.md @@ -0,0 +1,138 @@ +--- +sidebar_position: 1 +--- + +# Testing + +This module will teach you about writing tests for Web-Assembly (Wasm) Code for adding test suites for new plugins or modules for WasmEdge Software. This is an important aspect if you want to submit plugins to the WasmEdge community and we would require you to write test suites to cater to the plugin functionalities. + +## Writing Tests for Wasm Code Targeting WebAssembly + +Testing is a crucial part of software development, ensuring that code behaves as expected. In this guide, we'll explore how to write tests for Wasm code, using Google Test (gtest) and a specific example related to the WasmEdge runtime. + +### Prerequisites +- Familiarity with C++ and WebAssembly. +- Basic understanding of Google Test (gtest) framework. +- WasmEdge runtime environment set up for testing. + +### Code Explanation and Comments + +The provided code snippet demonstrates how to set up and execute tests for a WebAssembly code or the plugins you create, specifically using the WasmEdge runtime. The code is structured into two main parts: the test setup and the test cases themselves. + +```cpp +#include "common/defines.h" +#include "runtime/instance/module.h" +#include "wasi_logging/func.h" +#include "wasi_logging/module.h" + +#include +#include +#include +#include +#include + +namespace { + +// Function to create a module instance for testing. +WasmEdge::Runtime::Instance::ModuleInstance *createModule() { + // ... Code omitted for brevity ... +} + +// Helper function to fill memory with a byte value. +void fillMemContent(WasmEdge::Runtime::Instance::MemoryInstance &MemInst, + uint32_t Offset, uint32_t Cnt, uint8_t C = 0) noexcept { + // ... Code omitted for brevity ... +} + +// Helper function to fill memory with a string. +void fillMemContent(WasmEdge::Runtime::Instance::MemoryInstance &MemInst, + uint32_t Offset, const std::string &Str) noexcept { + // ... Code omitted for brevity ... +} + +} // namespace +``` + +### Test Cases + +```cpp +TEST(WasiLoggingTests, func_log) { + // ... Code omitted for brevity ... + + // Test cases for different logging levels and conditions. + // Each test case uses the `EXPECT_TRUE` or `EXPECT_FALSE` macros + // to validate the behavior of the `HostFuncInst.run` method. + + // ... Code omitted for brevity ... +} +``` + +You can call the main function to run all the tests using the below given example + +```cpp +GTEST_API_ int main(int argc, char **argv) { + testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} +``` + +## Where to add the tests? + +After writing tests for your specific modules or plugins, you can add them under `https://github.com/WasmEdge/WasmEdge/tree/master/test` and for more specific placement of the tests, + +For example, + +- You can add the tests for new plugins or update existing ones under `https://github.com/WasmEdge/WasmEdge/tree/master/test/plugins` + +## Documenting the Tests in Markdown + +When documenting these tests in Markdown, you should focus on explaining the purpose of each test, the setup required, and the expected outcomes. Here's an example of how you might structure this documentation: + + +```md + +# WasmEdge Module Tests + +## Overview + +This document outlines the tests for the `WasiLogging` module in a WebAssembly environment using the WasmEdge runtime. + +## Test Setup + +- **Module Creation**: A module instance is created for testing purposes using the `createModule` function. +- **Memory Manipulation**: The `fillMemContent` functions are used to manipulate memory content, essential for setting up test scenarios. + +## Test Cases + +### `func_log` Tests + +- **Purpose**: These tests validate the logging functionality at different levels and conditions. +- **Test Scenarios**: + - **Clear Memory**: Ensures memory is correctly cleared before each test. + - **Set Strings in Memory**: Tests the ability to write strings to memory. + - **Logging at Different Levels**: Validates logging behavior at various levels (info, warning, error, etc.). + - **Stderr Context**: Checks if the logging correctly handles the stderr context. + - **Unknown Level**: Ensures that an unknown logging level is handled appropriately. + +## Running Tests + +- Compile the test code with your C++ compiler targeting WebAssembly. +- Run the tests using a command-line tool that supports Google Test. + +## Conclusion + +These tests ensure the robustness and reliability of the module in a WebAssembly environment + +``` + +> Note: You can add the documentation in the plugin test folders you create + +### Blogs + +- [Google Test GitHub Repository](https://github.com/google/googletest): While not a traditional blog, the Google Test GitHub repository is a rich resource for learning about testing in C++ +- [Modern C++ Testing with Catch2](https://www.jetbrains.com/help/clion/unit-testing-tutorial.html) +- [C++ Testing - Catch2 and Google Test](https://www.learncpp.com/cpp-tutorial/introduction-to-testing-your-code/): LearnCpp.com provides a tutorial on testing in C++ using frameworks like Catch2 and Google Test. + +### Further Reading + +- Here is an example test case explaination for [test/plugins/wasi_logging](./example.md) \ No newline at end of file