LanguageExt v5 first beta

I'm now moving the v5 release from alpha to beta. Not because I'm feature complete, but because from my real-world testing of v5 (with my new startup project) it is much more stable than I expected. In fact I haven't hit any issues at all outside of missing functionality.

So, this is more of a 'soft launch beta', primarily for those who were perhaps not ready to use language-ext in alpha form but are more likely to in beta form.

Removed ResourceT transformer / integrated into IO

This release removes the ResourceT<M, A> monad-transformer from language-ext and instead moves the functionality into the IO<A> monad. ResourceT required IO to be in the transformer stack and so it really was adding complexity to a feature that's closely linked. This adds a tiny overhead to the IO monad -- the IO monad already carried an environment through its computations, so this doesn't change much -- in the big scheme of things it's likely to bring performance benefits.

Some big improvements because of this:

• use and release are now available in the Prelude, which makes them easier to work with (no need for any manual generic arguments), everything is inferable from usage.
• Forking an IO computation (launching it on a new thread) automatically creates a local resource environment for the fork and cleans it up when the forked operation is complete.
• Repeating an IO computation (repeat(computation)) - will automatically clean up any resources acquired by use in the computation (on each iteration).
• Retrying an IO computation (retry(computation)) - will automatically clean up any resources (acquired with use) when the computation fails, that mean retries don't accumulate resources unnecessarily.
• New function local(computation) works as a 'super using' -- in that it will automatically clean up any resources acquired with use in the computation. This allows you to create local scopes where you just freely acquire resources and then have a clean-up happen automatically.
  • By the way, I am open to different names for this, as we already have IO.local for local cancellation contexts and Reader.local for local environments. I'm also open to changing the names of the others. Ideally any name would be a single word so it's easy on the eye. So, nothing like localResource or cleanUp.
• New functions bracket(Acq, Use, Err, Fin) and bracket(Acq, Use, Fin) - these are like try \ catch \ finally blocks for more explicit resource usage:
  • Acq - acquires the resource
  • Use - uses the resource
  • Err - is the catch block
  • Fin - is the finally block

All the usual caveats apply: this is an alpha, this isn't fully tested, use at your own risk.

New Try monad and updated TryT transformer

I've re-added a Try<A> monad (I always intended to re-add it, just hadn't got around to it). And I've and reimplemented the TryT<M, A> monad-transformer in terms of Try (K<M, Try<A>>), previously it was implemented in terms of Fin (Func<K<M, Fin<A>>>).

I have also:

• Added Match and IfFail methods for pattern matching on the success/fail state.
• Added | operator @catch overrides to allow for easy error handling.

The IO monad also has a .Try() method that will run the IO monad in a try/catch block returning IO<Fin<A>> for more manual handling of IO errors.

Still needs some in-depth testing to make sure all exceptions are captured, but it's effectively feature complete.

Language-Ext 5.0 alpha-3

WARNING: THIS IS AN ALPHA RELEASE AND SHOULD BE CONSUMED WITH CARE! NOT FOR PRODUCTION.

Bug fixing and TODO resolving release, with some minor featurettes!

For those that don't know yet (and there's no reason to think you should, because I haven't announced it yet) -- the Pipes Effect system now has the ability to lift any monad into its stack (previously it only allowed Aff to be lifted). It is now a general monad transformer like ReaderT, OptionT, EitherT, etc.

As, with all monad-transfomers, when you 'run' the transformer, it generates the lifted monad. You can think of this being like a mini-compiler that takes the monad stack and compiles down to the inner-most monad, which can then just be run as normal.

The problem for Pipes is that there's usually lots of recursion, repetition (using repeat, retry), or iteration (using yieldAll, etc.). This is problematic when you don't know anything about the inner monad. The transformer can't run the inner monad, because it only has access to the Monad interface (Bind) and the inherited interfaces of Applicative and Functor (Apply, Action, Map, and Pure). So, doing iteration requires recursion, and recursion blows the stack in C#.

Previously Pipes were able to directly Run the Aff because the Pipe system knew it was working only with Aff. This allowed it to flatten the recursion.

Anyway, now Pipes has internal support for any Foldable. That means yieldAll(...) can take a sequence from any foldable (Range, EnumerableM, HashMap, HashSet, Lst, Map, Seq, Either, Option, Validation, Identity, ... and any you write) and yield the values within the structure through the pipe. Functions like repeat(ma) - which continually repeat an operation until it fails - have also been implemented internally as something that iterates over an infinite foldable.

This functionality has been enabled by adding a new method to the Applicative trait: Actions. You might know the existing Action(K<M, A> ma, K<M, B> mb) method that runs the first applicative (ma), ignores its result, and then runs the second applicative mb, returning its result.

Actions instead takes an IEnumerable<K<M, A>>:

K<F, A> Actions<A>(IEnumerable<K<F, A>> fas)

It runs each applicative action and ignores its result, returning the result of the last item. That means a sequence of Proxy values (Proxy is the monad-transformer for pipes) can be mapped - the map will just run (using RunEffect) the Proxy - producing a sequence of whatever the lifted inner-monad is for the Proxy. This lazy sequence of monads can then be invoked by calling Actions on it, which will lazily walk the sequence, evaluating the inner-monad one-by-one.

There is a default implementation, but it has the same lack of knowledge that Pipes had, so it should be overridden for computation based applicatives (that usually need invoking with without an argument). Here's the override for Eff<RT, A>:

static K<Eff<RT>, A> Applicative<Eff<RT>>.Actions<A>(IEnumerable<K<Eff<RT>, A>> fas) =>
    from s in getState<A>()
    from r in Eff<RT, A>.Lift(
        rt =>
        {
            Fin<A> rs = Errors.SequenceEmpty;
            foreach (var kfa in fas)
            {
                var fa = kfa.As();
                rs = fa.Run(rt, s.Resources, s.EnvIO);
                if (rs.IsFail) return rs;
            }
            return rs;
        })
    select r;

You can see how:

• It's able to gather information, like the runtime, resources, and IO environment.
• It knows how to run itself, whereas the generic transformer can't.
• It can shortcut the operation when any effect fails.

And so, if you want to use your own monads with Pipes then you should implement Actions.

There's still more to do with Pipes, but all of the examples in EffectsExamples now work, which is a good sign!

WARNING: THIS IS AN ALPHA RELEASE AND SHOULD BE CONSUMED WITH CARE! NOT FOR PRODUCTION.

Language-Ext 5.0 alpha-2

WARNING: THIS IS AN ALPHA RELEASE AND SHOULD BE CONSUMED WITH CARE! NOT FOR PRODUCTION.

General updates

• Free monad doesn't need Alternative trait: removed
• All semigroup and monoid-like types have their Append operator renamed to Combine. 'Combine' works semantically for more of the monoidal associative operations than Append (which really only makes sense with collections).
• Added new SemigroupK and MonoidK -- these are like the Semigroup and Monoid traits except they work on K<M, A> instead of A. These are almost identical to SemiAlternative and Alternative execept they don't require the underlying value to an an Applicative. The idea here is that SemigroupK and MonoidK would be used on types like collections that 'sum' when the Combine operator is applied, whereas SemiAlternative and Alternative provide an alternative value when the Combine operator is applied (coalescing).
• Added missing repeat variants, retry variants, and timeout for the IO monad
• Added IO.yieldFor(TimeSpan). This is like Task.Delay but for the IO monad. The war against async means that this does the thread-yielding internally, no need to call await. I figured yieldFor is more meaningful than Delay, it indicates that the thread is yielding, not simply blocking.
• Added support for guards in the IO monad
• Initial pass at a continuation-monad transformer: ContT<R, M, A> -- just the raw type for now.
• HeadOrNone, HeadOrInvalid, HeadOrLeft, LastOrNone, etc. have been removed.
• Head and Last are now Option returning. This is a breaking change. Can be mitigated by either matching, casting, or invocation of .ValueUnsafe() extension.
• Improved Range type -- previously there were several types (IntegerRange, CharRange, etc.) -- now there's just one: Range<A>. It leverages the new traits built into .NET (IComparisonOperators, IAddtionOperators, etc.)
• Started adding support for more of the .NET traits in the collection types and various other places like Foldable.Sum, Foldable.Max, etc.: (IComparisonOperators, IAddtionOperators, IAdditiveIdentity etc.) -- these are Microsoft's ugly named versions of monoids etc.

War on Extension Methods

I'm rapidly coming to the conclusion that extension-methods are a terrible idea. Especially in a library like language-ext where I am trying to present a consistent set of interfaces to types that share common traits. It's just impossible to enforce consistency and relies on the human eye -- and that errs regularly!

The latest move toward using traits is really starting to help reduce the extension methods, or at least mean the extension methods are hanging off traits rather than individual instance-types.

One change that I have made recently is to change Foldable to require implementation of FoldWhile and FoldWhileBack instead of Fold and FoldBack. This means that so many more default behaviours can hang off of Foldable -- and most of them are optimal. For example, Exists -- which can stop processing as soon as its predicate returns true -- couldn't early-out before.

And so, the foldable trait is now growing to have a ton of functionality. Also nested foldables!

However, quite a lot of those methods, like Sum, Count, etc. also exist on IEnumerable. And so, for a type like Seq which derives from both IEnumerable and K<Seq, A>, there will be extension method resolution issues.

So, the choice is to provide extension methods for IEnumerable (an ill defined type) or for Foldable - a super featureful type with the opportunity for implementers to provide bespoke optimised overrides.

Really, the choice should be easy: extensions for Foldable are just better than extensions for IEnumerable. So, I have done that. The downside is that this will be another breaking change (because the IEnumerable extensions have been removed). The fix is to convert from IEnumerable<A> to EnumerableM<A> using .AsEnumerableM(). EnumerableM<A> supports Foldable (and other traits).

Conclusion

So, I've been working to remove as many non-trait extension methods as I can -- and I will continue to do so leading up to the beta. This will bring consistency to the code-base, reduce the amount of code, and provide ample opportunities for bespoke optimisations. Just be aware that this is another fix-up job.

WARNING: THIS IS AN ALPHA RELEASE AND SHOULD BE CONSUMED WITH CARE! NOT FOR PRODUCTION.

Language-Ext 5.0 alpha-1

This release should only be consumed by those who are interested in the new features coming in the monster v5 release.

Just to give you an idea of the scale of this change:

• 193 commits
• 1,836 files changed
• 135,000 lines of code added (!)
• 415,000 lines of code deleted (!!)

It is a monster and should be treated with caution...

• It is not ready for production
• It is not feature complete
• The new features don't have unit tests yet and so are probably full of bugs
• I haven't yet dogfooded all the new functionality, so it may not seem as awesome as it will eventually become!

If you add it to a production project, you should only do so to see (potentially) how many breaking changes there are. I would not advise migrating a production code-base until I get close to the final release.

I am also not going to go into huge detail about the changes here, I will simply list them as headings. I will do a full set of release notes for the beta release. You can however follow the series of articles I am writing to help you all prep for v5 -- it goes (and will go) into much more detail about the features.

New Features

• Higher-kinded traits
  • K<F, A> - higher-kinds enabling interface
  • Includes:
    • Defintions (interfaces listed below)
    • Static modules (Functor.map, Alternative.or, StateM.get, ...)
    • Extension methods (.Map, .Or, Bind, etc.),
    • Extension methods that replace LanguageExt.Transformers (BindT, MapT, etc. ), now fully generic.
    • Trait implementations for all Language-Ext types (Option, Either<L>, etc.)
  • Functor<F>
  • Applicative<F>
  • Monad<M>
  • Foldable<F>
  • Traversable<T>
  • Alternative<F>
  • SemiAlternative<F>
  • Has<M, TRAIT>
  • Reads<M, OUTER_STATE, INNER_STATE>
  • Mutates<M, OUTER_STATE, INNER_STATE>
  • ReaderM<M, Env>
  • StateM<M, S>
  • WriterM<M, OUT>
  • MonadT<M, N> - Monad transformers
    • ReaderT<Env, M, A>
    • WriterT<Out, M, A>
    • StateT<S, M, A>
    • IdentityT<M, A>
    • EitherT<L, M, R>
    • ValidationT<F, M, S>
    • OptionT<M, A>
    • TryT<M, A>
    • IdentityT<M, A>
    • ResourceT<M, A>
• Free<F, A> - Free monads
• IO<A> - new IO monad that is the base for all IO
• Eff<RT, A> monad rewritten to use monad-transformers (StateT<RT, ResourceT<IO>, A>)
• Eff<RT, A> doesn't need HasCancel trait (or any trait)
• Transducers
• Pure / Fail monads
• Lifting
• Improved guards, when, unless
• Nullable annotations - still WIP, mostly complete on Core)
• Collection initialisers

Breaking changes

• netstandard2.0 no longer supported (.NET 8.0+ only)
• Seq1 made [Obsolete]
• 'Trait' types now use static interface methods
• The 'higher-kind' trait types have all been refactored
• The Semigroup<A> and Monoid<A> types have been refactored
• The static TypeClass class has been renamed Trait
• Apply extensions that use raw Func removed
• Manually written Sequence extension methods have been removed
• Manually written Traverse extension methods have been removed
• ToComparer doesn't exist on the Ord<A> trait any more
• Renamed LanguageExt.ClassInstances.Sum
• Guard<E> has become Guard<E, A>
• UnitsOfMeasaure namespace converted to a static class
• Either doesn't support IEnumerable<EitherData> any more
• Either 'bi' functions have their arguments flipped
• Nullable (struct) extensions removed
• Support for Tuple and KeyValuePair removed
• Types removed outright
  • Some<A>
  • OptionNone
  • EitherUnsafe<L, R>
  • EitherLeft<L>
  • EitherRight<L>
  • Validation<MFail, Fail, A>
  • Try<A>
  • TryOption<A>
  • TryAsync<A>
  • TryOptionAsync<A>
  • Result<A>
  • OptionalResult<A>
  • Async extensions for Option<A>
  • ExceptionMatch, ExceptionMatchAsync, ExceptionMatchOptionalAsync
• Libraries removed outright
  • LanguageExt.SysX
  • LanguageExt.CodeGen
  • LanguageExt.Transformers

Big fixes and minor improvements release

New:

• Support for IfFail in Try, TryOption, TryAsync, and TryOptionAsync - Thanks @mark-pro 👍
• Update prelude for Arithmetic typeclass - Thanks @benjstephenson 👍
• Improvements to the Effects samples

Bug fixes:

• Fix for EitherUnsafe MapLeft throws ResultIsNullException
• Fix for Stack overflow with MapT on T
• Fix for CancellationTokenSource not disposed

Breaking change: OptionAsync await + Producer.merge

This is a fixes release.

OptionAsync

I have brought forward a change to OptionAsync that I was saving for v5: the removal of the async-awaiter. You can't now await an OptionAsync. The resulting value wasn't clear, and honestly the async/await machinery is really quite shonky outside of using it for Tasks.

I have made the OptionAsync implementation aware of nullable references, and so you can now await the Value property instead:

    public Task<A?> Value

That will reproduce the same behaviour as before. You can still await the ToOption() method, which returns a Task<Option<A>>, if you want to do matching on the underlying option. Or call the various Match* methods.

This release fixes the following issues:

• #1159
• #1184

Producer.merge error handling

Producer merging was silently ignoring errors. They now exit and return the first error and shutdown other producers they were merged with. Merged producers also listen for cancellation correctly now.

Finally, you can only merge produces with a bound value of Unit. This is to stop the silent dropping of their return value as well as the need to provide a final (bound) value for merged producers, which doesn't really make sense. That also means the + operator can't work any more because it can't be defined for the Producer<..., A> type. So you must use Producer.merge.

This fixes an issue mentioned in: #1177

repeatM doesn't cause a stack-overflow

Certain elements of the Pipes capability of language-ext are direct ports from the Haskell Pipes library, which uses recursion everywhere. repeatM was causing a stack-overflow on usage, this is now fixed.

Example usage:

public static Effect<Runtime, Unit> effect =>
    Producer.repeatM(Time<Runtime>.nowUTC) | writeLine<DateTime>(); 

static Consumer<Runtime, X, Unit> writeLine<X>() =>
    from x in awaiting<X>()
    from _ in Console<Runtime>.writeLine($"{x}")
    from r in writeLine<X>()
    select r;

repeat improvements

Removed the Repeat case from the Pipes DSL which simplifies it and brings it closer to the Haskell version. Updated the repeat combinator function to use the same Enumerate case that yieldAll uses. This has benefits that it doesn't spread out when composed with other Proxy types. This is should mean it's easier to pick bits of the expression to repeat, rather than the whole effect being repeated due to the spread.

Trampoline

Added trampolining functionality. It's relatively light at the moment, I am considering approaches to enable genuine recursion in the effects system. Don't rely on this, it may be removed if it doesn't prove useful and almost certainly will have API changes if it stays.

Breaking Change: Pipes enumerate

There Pipes functions: enumerate, enumerate2, observe, observe2 have been deleted and replaced with yieldAll (that accepts IEnumerable, IAsyncEnumerable, or IObservable).

The previous implementation had mixed behaviours, some that always yielded the values, some that turned the remainder of the pipes expression into a enumeration. This wasn't entirely clear from the name and so now there is a single set of yieldAll functions that always yield all the values in the collection downstream.

The behaviour of the always yield enumerate functions was also buggy, and didn't result in the remainder of a Producer or Pipe being invoked after the yield. :

    public static Effect<Runtime, Unit> effect => 
        repeat(producer) | consumer;

    static Producer<Runtime, int, Unit> producer =>
        from _1 in Console<Runtime>.writeLine("before")
        from _2 in yieldAll(Range(1, 5))
        from _3 in Console<Runtime>.writeLine("after")
        select unit;
    
    static Consumer<Runtime, int, Unit> consumer =>
        from i in awaiting<int>()
        from _ in Console<Runtime>.writeLine(i.ToString())
        select unit;

In the example above, "after" would never be called, this is now fixed.

There is also a new & operator overload for Pipes which performs the operations in series. This has the effect of concatenating Producers (for example), but will work for Pipe, Consumer, Client, and Server.

    // yields [1..10]
    static Producer<Runtime, int, Unit> producer =>
        yieldAll(Range(1, 5)) & yieldAll(Range(6, 5));

There's still work to do on repeat, but this was quite a difficult change, so I'll leave that for now.

Other fixes:

• #1171
• #1180

Fixes and improvements release

This release puts out the 4.3.* beta changes:

• Refactored Error type + Eff and Aff applicative functors

And contains a number of contributed improvements:

• Added LanguageExt Either and F# Result Converters
• Added Prism optics for manipulating Option fields
• Eq.ToEqualityComparer (Ord.ToComparer)

And a number of contributed bug fixes:

• Fix for broken Task.Cast and ValueTask.Cast
• Fix incorrect docstring for two LeftToSeq implementations.
• Fix Distinct with custom comparer (hash bug)
• Fix for SearchOptions.TopDirectoryOnly option not working
• Fix for missing implementations of FileIO (test)
• Fix for Fin.GetHashCode Throws an Exception when Fin.Succ is False

Thanks to all those who contributed. I am still super busy with other projects right now, and I don't always get to PRs as quickly as I would like, but It's always appreciated.

Any problems, please report in the Issues.