v0.5.3

Released on 14 May, 2024

What's Changed

Features

• Add socket time out environment variable by @mellis13 in #474

Bug Fixes

• Fix widths field for list-table in install documentation by @al-rigazzi in #468
• Bump redis dep to 7.2.4 by @ankona in #470
• Enforce matching type for dataset::unpack_tensor by @mellis13 in #478
• Fix minor warnings in C++ client by @ashao in #476

API Breaks

• Discontinue SmartRedis support for python 3.8 by @AlyssaCote in #482

Miscellaneous Improvements

• Fix inconsistency with function parameter variable names by @mellis13 in #471
• Remove a vestigial requirements.txt file by @ashao in #462
• Update CI for Intel by @ashao in #475
• Remove unused parameter model name in _get_model_script_db() by @mellis13 in #477
• Remove unused memory layout constructor parameter for TensorBase by @mellis13 in #479
• Addition of changelog edit check github action by @amandarichardsonn in #480
• Upgrade ubuntu to version 22.04 and gcc to version 11 by @AlyssaCote in #484
• Auto-post release PR to develop by @amandarichardsonn in #486
• Automate release notes by @amandarichardsonn in #487
• Fix build wheel by @mellis13 in #491
• Pin pylint to fix regression error by @ankona in #492
• Ensure all reply errors are logged by @ankona in #485

Full Changelog: v0.5.2...v0.5.3

v0.5.2

v0.5.2

Released on February 16, 2024

Description

• Fixed bug which was sending tensors to the database twice (Python Client)

Detailed Notes

• A previous bug fix for the Python client which addressed a problem when sending numpy views inadvertently kept the original put_tensor call in place. This essentially doubles the cost of the operation. (PR464)

v0.5.1

Released on February 15, 2024

Description

• Fix bug when sending an array view
• Add concurrency groups for Github Action testing
• Update license to include 2024
• Increase build space for Github Actions
• Update README python versions
• Expose Typehints
• Update supported python versions [Add 3.11, remove 3.7]
• Tweak the build system to enable building SmartRedis with Nvidia's NVHPC toolchain
• Improvements/upgrades to the container used for Github actions
• Code updates to avoid compiler warnings
• Added developer documentation on how to run a single test case and eliminated duplicative environment variables
• Resolve a linting issue with pybind-to-python error propagation
• Use mutable fields to enable Dataset get methods that store memory to be marked const

Detailed Notes

• Detect whether the tensor the user is sending is a view and if so, make an explicit copy. (PR453)
• Add support to concurrency groups in the run_tests workflow. (PR456)
• Update license to include 2024. (PR454)
• Add new Github Action that removes unneeded packages and resizes the root disk space. (PR455)
• Update developer documentation to reflect newly supported versions of Python (PR450) (PR452)
• Add and ship py.typed marker to expose inline type hints (PR451)
• Deprecate support for Python 3.7 by removing from the allowed Python versions (PR450)
• Update Python package dependencies to add support for Python 3.11 (PR450)
• Change the order of arguments in our MakeFile to ensure that all dependencies are compiled with GCC (PR448)
• Add new user-configurable parameters DEP_CC, DEP_CXX to control which compiler is used to build dependencies (PR448)
• Ameliorate some compiler warnings related that were flagged in GCC 12 (unreachable code blocks, signed/unsigned mismatches) (PR448)
• CI/CD: Bump the container version used in Github Actions Ubuntu 22.04 to be able to start testing GCC 12 (PR448)
• CI/CD: Bump the versions of GCC used in testing to the currently maintained versions (PR448)
• CI/CD: Add NVHPC to the testing matrix (PR448)
• CI/CD: Test the shared/static compilations and examples with all compilers (PR448)
• CI/CD: Compile Redis and RedisAI and use those versions in testing instead of extracting from a container (PR448)
• CI/CD: Bump the version of Redis used in testing to 7.0.5, the same version as we use with SmartSim (PR448)
• CI/CD: Pin the Torch version to 1.11.0, the same as supported in SmartSim (PR448)
• Added developer documentation on how to run a single test case with the new test/build system and eliminated use of SMARTREDIS_TEST_DEVICE and SMARTREDIS_TEST_CLUSTER environment variables (PR445)
• Resolve a linting issue with pybind-to-python error propagation by changing import format and narrowing the lookup of pybind error names to the error module (PR444)
• Use mutable fields to enable Dataset get methods that store memory to be marked const (PR443)

v0.5.0

Released on December 18, 2023

Description

• Unpin the Intel Fortran compiler in CI/CD
• Added a missing space in an error message
• Improved consistency of namespace declarations for C++ pybind
  interface
• Improved const correctness of C++ Client
• Improved const correctness of C++ Dataset
• Updated documentation
• Added test cases for all Client construction parameter combinations
• Centralized dependency tracking to setup.cfg
• Improved robustness of Python client construction
• Updated Client and Dataset documentation
• Expanded list of allowed characters in the SSDB address
• Added coverage to SmartRedis Python API functions
• Improved responsiveness of library when attempting connection to
  missing backend database
• Moved testing of examples to on-commit testing in CI/CD pipeline
• Added name retrieval function to the DataSet object
• Updated RedisAI version used in post-commit check-in testing in
  Github pipeline
• Allow strings in Python interface for Client.run_script,
  Client.run_script_multiGPU
• Improved support for model execution batching
• Added support for model chunking
• Updated the third-party RedisAI component
• Updated the third-party lcov component
• Add link to contributing guidelines
• Added link to contributing guidelines
• Added support for multiple backend databases via a new Client
  constructor that accepts a ConfigOptions object

Detailed Notes

• Unpin the Intel Fortran compiler in CI/CD. This requires running the
  compiler setup script twice, once for Fortran and once for other
  languages, since they're on different releases
  (PR436)
• Added a missing space in an error message
  (PR435)
• Made the declaration of the py namespace in py*.h consistently
  outside the SmartRedis namespace declaration
  (PR434)
• Fields in several C++ API methods are now properly marked as const
  (PR430)
• The Dataset add_tensor method is now const correct, as are all
  internal the methods it calls
  (PR427)
• Some broken links in the documentation were fixed, and the
  instructions to run the tests were updated
  (PR423)
• Added test cases for all Client construction parameter combinations
  (PR422)
• Merged dependency lists from requirements.txt and
  requirements-dev.txt into setup.cfg to have only one set of
  dependencies going forward
  (PR420)
• Improved robustness of Python client construction by adding
  detection of invalid kwargs
  (PR419),
  (PR421)
• Updated the Client and Dataset API documentation to clarify which
  interacts with the backend db
  (PR416)
• The SSDB address can now include '-' and '_' as special characters
  in the name. This gives users more options for naming the UDS socket
  file (PR415)
• Added tests to increase Python code coverage
• Employed a Redis++ ConnectionsObject in the connection process to
  establish a TCP timeout of 100ms during connection attempts
  (PR413)
• Moved testing of examples to on-commit testing in CI/CD pipeline
  (PR412)
• Added a function to the DataSet class and added a test
• Updated RedisAI version used in post-commit check-in testing in
  Github pipeline to a version that supports fetch of model chunking
  size (PR408)
• Allow users to pass single keys for the inputs and outputs
  parameters as a string for Python run_script and run_script_multigpu
• Exposed access to the Redis.AI MINBATCHTIMEOUT parameter, which
  limits the delay in model execution when trying to accumulate
  multiple executions in a batch
  (PR406)
• Models will now be automatically chunked when sent to/received from
  the backed database. This allows use of models greater than 511MB in
  size. (PR404)
• Updated from RedisAI v1.2.3 (test target)/v1.2.4 and v1.2.5 (CI/CD
  pipeline) to v1.2.7
  (PR402)
• Updated lcov from version 1.15 to 2.0
  (PR396)
• Create CONTRIBUTIONS.md file that points to the contribution
  guideline for both SmartSim and SmartRedis
  (PR395)
• Migrated to ConfigOptions-based Client construction, adding multiple
  database support
  (PR353)

v0.4.2

What's Changed

• Cleanup obsolete build and test files by @billschereriii in #366
• Update language support matrix by @billschereriii in #379
• Fix bad kwarg when launching server via UDS by @ankona in #378
• Update URI for installation instructions by @ankona in #380
• Make testing more robust by @billschereriii in #381
• Mitigate lint errors, apply formatters by @ankona in #382
• Add support for static builds by @billschereriii in #386
• Fix Postmerge tests by @billschereriii in #388
• Update third-party software by @billschereriii in #389
• Extended section on dataset aggregation by @billschereriii in #390
• Resolve pylint error by @ankona in #393
• Update cibuildwheel version by @MattToast in #397
• Update changelog for release by @MattToast in #398
• Bump SR version number to 0.4.2 by @MattToast in #399
• Update to v0.4.2 by @MattToast in #400

Full Changelog: v0.4.1...v0.4.2

v0.4.1

Released on July 5, 2023

Description

This release revamps the build and test systems for SmartRedis as well
as improving compatibility with different Fortran compilers and laying
the groundwork for future support for interacting with multiple
concurrent backend databases:

• Documentation improvements
• Improved compatibility of type hints with third-party software
• Added type hints to the Python interface layer
• Add support for Python 3.10
• Updated setup.py to work with the new build system
• Remove unneeded method from Python SRObject class
• Fixed a memory leak in the C layer
• Revamp SmartRedis test system
• Remove debug output in pybind layer
• Update Hiredis version to 1.1.0
• Enable parallel build for the SmartRedis examples
• Experimental support for Nvidia toolchain
• Major revamp of build and test systems for SmartRedis
• Refactor Fortran methods to return default logical kind
• Update CI/CD tests to use a modern version of MacOS
• Fix the spelling of the Dataset destructor's C interface (now
  DeallocateDataSet)
• Update Redis++ version to 1.3.8
• Refactor third-party software dependency installation
• Add pip-install target to Makefile to automate this process going
  forward (note: this was later removed)
• Added infrastructure for multiDB support

Detailed Notes

• Assorted updates and clarifications to the documentation (PR367)
• Turn ParamSpec usage into forward
  references to not require typing-extensions at runtime (PR365)
• Added type hints to the Python interface layer (PR361)
• List Python 3.10 support and loosen PyTorch requirement to allow for
  versions support Python 3.10 (PR360)
• Streamlined setup.py to simplify Python install (PR359)
• Remove from_pybind() from Python SRObject class as it's not needed
  and didn't work properly anyway (PR358)
• Fixed memory leaked from the C layer when calling
  get_string_option() (PR357)
• Major revamp to simplify use of SmartRedis test system, automating
  most test processes (PR356)
• Remove debug output in pybind layer associated with put_dataset (PR352)
• Updated to the latest version of Hiredis (1.1.0) (PR351)
• Enable parallel build for the SmartRedis examples by moving utility
  Fortran code into a small static library (PR349)
• For the NVidia toolchain only: Replaces the assumed rank feature of
  F2018 used in the Fortran client with assumed shape arrays, making
  it possible to compile SmartRedis with the Nvidia toolchain. (PR346)
• Rework the build and test system to improve maintainability of the
  library. There have been several significant changes, including that
  Python and Fortran clients are no longer built by defaults and that
  there are Make variables that customize the build process. Please
  review the build documentation and make help to see all that has
  changed. (PR341)
• Many Fortran routines were returning logical kind = c_bool which
  turns out not to be the same default kind of most Fortran compilers.
  These have now been refactored so that users need not import
  iso_c_binding in their own applications (PR340)
• Update MacOS version in CI/CD tests from 10.15 to 12.0 (PR339)
• Correct the spelling of the C DataSet destruction interface from
  DeallocateeDataSet to DeallocateDataSet (PR338)
• Updated the version of Redis++ to v1.3.8 to pull in a change that
  ensures the redis++.pc file properly points to the generated
  libraries (PR334)
• Third-party software dependency installation is now handled in the
  Makefile instead of separate scripts
• New pip-install target in Makefile will be a dependency of the lib
  target going forward so that users don't have to manually pip
  install SmartRedis in the future (PR330)
• Added ConfigOptions class and API, which will form the backbone of
  multiDB support (PR303)

v0.4.0

Released on April 12, 2023

Description

This release provides a variety of features to improve usability and
debugging of the SmartRedis library, notably including Unix domain
socket support, logging, the ability to print a textual representation
of a string or dataset, dataset inspection, documentation updates, fixes
to the multi-GPU support, and much more:

• Prepare 0.4.0 release
• Disable codecov CI tests
• Improved error message in to_string methods in C interface
• Streamlined PyBind interface layer
• Updated Python API documentation
• Streamlined C interface layer
• Improved performance of get, put, and copy dataset methods
• Fix a bug which prevented multi-GPU model set in some cases
• Streamline pipelined execution of tasks for backend database
• Enhance code coverage to include all 4 languages supported by
  SmartRedis
• Fix a bug which resulted in wrong key prefixing when retrieving
  aggregation lists in ensembles
• Correct assorted API documentation errors
• Improve documentation of exception handling in Redis server classes
• Improve error handling for setting of scripts and models
• Add support to inspect the dimensions of a tensor via
  get_tensor_dims()
• Split dataset prefixing control from use_tensor_ensemble_prefix() to
  use_dataset_ensemble_prefix()
• Update to the latest version of redis-plus-plus
• Update to the latest version of PyBind
• Change documentation theme to sphinx_book_theme and fix doc strings
• Add print capability for Client and DataSet
• Add support for inspection of tensors and metadata inside datasets
• Add support for user-directed logging for Python clients, using
  Client, Dataset, or LogContext logging methods
• Add support for user-directed logging for C and Fortran clients
  without a Client or Dataset context
• Additional error reporting for connections to and commands run
  against Redis databases
• Improved error reporting capabilities for Fortran clients
• Python error messages from SmartRedis contain more information
• Added logging functionality to the SmartRedis library
• A bug related to thread pool initialization was fixed.
• This version adds new functionality in the form of support for Unix
  Domain Sockets.
• Fortran client can now be optionally built with the rest of the
  library
• Initial support for dataset conversions, specifically Xarray.

Detailed Notes

• Update docs and version numbers in preparation for version 0.4.0.
  Clean up duplicate marking of numpy dependency
  (PR321)
• Remove codecov thresholds to avoid commits being marked as
  'failed' due to coverage variance
  (PR317)
• Corrected the error message in to_string methods in C interface to
  not overwrite the returned error message and to name the function
  (PR320)
• Streamlined PyBind interface layer to reduce repetitive boilerplate
  code (PR315)
• Updated Python API summary table to include new methods
  (PR313)
• Streamlined C interface layer to reduce repetitive boilerplate code
  (PR312)
• Leveraged Redis pipelining to improve performance of get, put, and
  copy dataset methods
  (PR311)
• Redis::set_model_multigpu() will now upload the correct model to
  all GPUs (PR310)
• RedisCluster::_run_pipeline() will no longer unconditionally apply
  a retry wait before returning
  (PR309)
• Expand code coverage to all four languages and make the CI/CD more
  efficent (PR308)
• An internal flag was set incorrectly, it resulted in wrong key
  prefixing when accessing (retrieving or querying) lists created in
  ensembles (PR306)
• Corrected a variety of Doxygen errors and omissions in the API
  documentation
  (PR305)
• Added throw documentation for exception handling in redis.h,
  redisserver.h, rediscluster.h
  (PR301)
• Added error handling for a rare edge condition when setting scripts
  and models
  (PR300)
• Added support to inspect the dimensions of a tensor via new
  get_tensor_dims() method
  (PR299)
• The use_tensor_ensemble_prefix() API method no longer controls
  whether datasets are prefixed. A new API method,
  use_dataset_ensemble_prefix() now manages this.
  (PR298)
• Updated from redis-plus-plus v1.3.2 to v1.3.5
  (PR296)
• Updated from PyBind v2.6.2 to v2.10.3
  (PR295)
• Change documentation theme to sphinx_book_theme to match SmartSim
  documentation theme and fix Python API doc string errors
  (PR294)
• Added print capability for Client and DataSet to give details
  diagnostic information for debugging
  (PR293)
• Added support for retrieval of names and types of tensors and
  metadata inside datasets
  (PR291)
• Added support for user-directed logging for Python clients via
  {Client, Dataset, LogContext}.{log_data, log_warning, log_error}
  methods (PR289)
• Added support for user-directed logging without a Client or Dataset
  context to C and Fortran clients via _string() methods
  (PR288)
• Added logging to capture transient errors that arise in the _run()
  and _connect() methods of the Redis and RedisCluster classes
  (PR287)
• Tweak direct testing of Redis and RedisCluster classes
  (PR286)
• Resolve a disparity in the construction of Python client and
  database classes
  (PR285)
• Fortran clients can now access error text and source location
  (PR284)
• Add exception location information from CPP code to Python
  exceptions
  (PR283)
• Added client activity and manual logging for developer use
  (PR281)
• Fix thread pool error
  (PR280)
• Update library linking instructions and update Fortran tester build
  process (PR277)
• Added [add_metadata_for_xarray]{.title-ref} and
  [transform_to_xarray]{.title-ref} methods in
  [DatasetConverter]{.title-ref} class for initial support with Xarray
  (PR262)
• Change Dockerfile to use Ubuntu 20.04 LTS image
  (PR276)
• Implemented support for Unix Domain Sockets, including
  refactorization of server address code, test cases, and check-in
  tests. (PR252)
• A new make target [make lib-with-fortran]{.title-ref} now compiles
  the Fortran client and dataset into its own library which
  applications can link against
  (PR245)

v0.3.1

Released on June 24, 2022

Description

Version 0.3.1 adds new functionality in the form of DataSet aggregation lists for pipelined retrieval of data, convenient support for multiple GPUs, and the ability to delete scripts and models from the backend database. It also introduces multithreaded execution for certain tasks that span multiple shards of a clustered database, and it incorporates a variety of internal improvements that will enhance the library going forward.

Detailed Notes

• Implemented DataSet aggregation lists in all client languages, for pipelined retrieval of data across clustered and non-clustered backend databases. (PR258) (PR257) (PR256) (PR248) New commands are:

  • append_to_list()
  • delete_list()
  • copy_list()
  • rename_list()
  • get_list_length()
  • poll_list_length()
  • poll_list_length_gte()
  • poll_list_length_lte()
  • get_datasets_from_list()
  • get_dataset_list_range()
  • use_list_ensemble_prefix()

• Implemented multithreaded execution for parallel dataset list retrieval on clustered databases. The number of threads devoted for this purpose is controlled by the new environment variable SR_THERAD_COUNT. The value defaults to 4, but may be any positive integer or special value zero, which will cause the SmartRedis runtime to allocate one thread for each available hardware context. (PR251) (PR246)

• Augment support for GPUs by implementing multi-GPU convenience functions for all client languages. (PR254) (PR250) (PR244) New commands are:

  • set_model_from_file_multigpu()
  • set_model_multigpu()
  • set_script_from_file_multigpu()
  • set_script_multigpu()
  • run_model_multigpu()
  • run_script_multigpu()
  • delete_model_multigpu()
  • delete_script_multigpu()

• Added API calls for all clients to delete models and scripts from the backend database. (PR240) New commands are:

  • delete_script()
  • delete_model()

• Updated the use of backend RedisAI API calls to discontinue use of deprecated methods for model selection (AI.MODELSET) and execution (AI.MODELRUN) in favor of current methods AI.MODELSTORE and AI.MODELEXECUTE, respectively. (PR234)

• SmartRedis will no longer call the C runtime method srand() to ensure that it does not interfere with random number generation in client code. It now uses a separate instance of the C++ random number generator. (PR233)

• Updated the way that the Fortran enum_kind type defined in the fortran_c_interop module is defined in order to better comply with Fortran standard and not interfere with GCC 6.3.0. (PR231)

• Corrected the spelling of the word "command" in a few error message strings. (PR221)

• SmartRedis now requires a CMake version 3.13 or later in order to utilize the add_link_options CMake command. (PR217)

• Updated and improved the documentation of the SmartRedis library. In particular, a new SmartRedis Integration Guide provides an introduction to using the SmartRedis library and integrating it with existing software. (PR261) (PR260) (PR259) (SSPR214)

• Added clustered Redis testing to automated GitHub check-in testing. (PR239)

• Updated the SmartRedis internal API for building commands for the backend database. (PR223) This change should not be visible to clients.

• The SmartRedis example code is now validated through the automated GitHub checkin process. This will help ensure that the examples do not fall out of date. (PR220)

• Added missing copyright statements to CMakeLists.txt and the SmartRedis examples. (PR219)

• Updated the C++ test coverage to ensure that all test files are properly executed when running "make test". (PR218)

• Fixed an internal naming conflict between a local variable and a class member variable in the DataSet class. (PR215) This should not be visible to clients.

• Updated the internal documentation of methods in SmartRedis C++ classes with the override keyword to improve compliance with the latest C++ standards. (PR214) This change should not be visible to clients.

• Renamed variables internally to more cleanly differentiate between names that are given to clients for tensors, models, scripts, datasets, etc., and the keys that are used when storing them in the backend database. (PR213) This change should not be visible to clients.

v0.3.0

Release on Febuary 11, 2022

Description

• Improve error handling across all SmartRedis clients (PR159) (PR191) (PR199) (PR205) (PR206)

  • Includes changes to C and Fortran function prototypes that are not backwards compatible
  • Includes changes to error class names and enum type names that are not backwards compatible

• Add poll_dataset functionality to all SmartRedis clients (PR184)

  • Due to other breaking changes made in this release, applications
    using methods other than poll_dataset to check for the
    existence of a dataset should now use poll_dataset

• Add environment variables to control client connection and command
  timeout behavior (PR194)

• Add AI.INFO command to retrieve statistics on scripts and models
  via Python and C++ clients (PR197)

• Create a Dockerfile for SmartRedis (PR180)

• Update redis-plus-plus version to 1.3.2 (PR162)

• Internal client performance and API improvements (PR138) (PR141) (PR163) (PR203)

• Expose Redis FLUSHDB, CONFIG GET, CONFIG SET, and SAVE
  commands to the Python client (PR139) (PR160)

• Extend inverse CRC16 prefixing to all hash slots (PR161)

• Improve backend dataset representation to enable performance
  optimization (PR195)

• Simplify SmartRedis build proccess (PR189)

• Fix zero-length array transfer in Fortran
  convert_char_array_to_c (PR170)

• Add continuous integration for all SmartRedis tests (PR165) (PR173) (PR177)

• Update SmartRedis docstrings (PR200) (PR207)

• Update SmartRedis documentation and examples (PR202) (PR208) (PR210)

v0.2.0

Release on August, 9, 2021

Description

• Improved tensor memory management in the Python client (PR70)
• Improved metadata serialization and removed protobuf dependency (PR61)
• Added unit testing infrastructure for the C++ client (PR96)
• Improve command execution fault handling (PR65) (PR97) (PR105)
• Bug fixes (PR52) (PR72) (PR76) (PR84)
• Added copy, rename, and delete tensor and DataSet commands in the Python client (PR66)
• Upgrade to RedisAI 1.2.3 (PR101)
• Fortran and C interface improvements (PR93) (PR94) (PR95) (PR99)
• Add Redis INFO command execution to the Python client (PR83)
• Add Redis CLUSTER INFO command execution to the Python client (PR105)