Skip to content

Task/ValueTuple/ReadOnlyMemory/Channels migration. Add .MemoryStore, .DynamoStore, .MessageDb; Support non-event sourced pipelines

Latest

Choose a tag to compare

@bartelink bartelink released this 24 Sep 11:45
· 14 commits to master since this release

Added

  • Feed: Monitor.AwaitCompletion enables quasi deterministic waiting for the processing of async reactions within integration tests #170
  • Feed: Checkpoint enables committing progress (and obtaining the achieved positions) without stopping the Sink #162
  • Feed.SinglePassFeedSource: Coordinates reads of a set of tranches until each reaches its Tail #179
  • Streams.Stats.abendThreshold: Abends Sink's processing with a HealthCheckException if a stream continually fails to progress, or continually errors with non-transient exceptions #246
  • Streams.Stats.Categorize: Break down ok handler outcomes by stream category #258
  • Streams.Stats.Latency.Record(Ok|Outcom|Exn): Gather custom Handler latency metrics #258
  • StreamFilter: Generic logic for filtering source events based on Category name, Stream name or Event type #252
  • Ingester, Sinks: Expose optional ingesterStateInterval and commitInterval control on Sink factories #154 #239
  • Scheduler: Split out stats re rateLimited and timedOut vs exceptions #194
  • Scheduler: Added index, eventType to error logging #237
  • Scheduler: purgeInterval to control memory usage #97
  • Scheduler: wakeForResults option to maximize throughput (without having to drop sleep interval to zero) #161
  • Internal.Async.AwaitKeyboardInterruptAsTaskCanceledException: Honored cancellation; moved to Propulsion.Internal #160
  • Propulsion.EventStoreDb: Ported EventStore to target Equinox.EventStore >= 4.0.0 (using the gRPC interface) #139
  • Propulsion.CosmosStore3: Special cased version of Propulsion.CosmosStore to target Equinox.CosmosStore v [3.0.7-3.99.0] Deprecated; Please migrate to Propulsion.CosmosStore by updating Equinox.CosmosStore dependencies to 4.0.0 #139
  • Propulsion.DynamoStore: Equinox.CosmosStore-equivalent functionality for Equinox.DynamoStore. Combines elements of CosmosStore, SqlStreamStore, Feed #140 #140 #177
  • Propulsion.MessageDb: FeedSource and CheckpointStore for MessageDb #181 🙏 @nordfjord
  • Propulsion.MemoryStore: MemoryStoreSource to align with other sources for integration testing. Includes deterministic AwaitCompletion as per Propulsion.Feed-based Sources #165
  • Propulsion.SqlStreamStore: Added startFromTail #173
  • Propulsion.Tool: checkpoint commandline option; enables viewing or overriding checkpoints #141
  • Propulsion.Tool: sync <kafka|stats> supports from json source option #250
  • Propulsion.Tool: Add support for autoscaling throughput of Cosmos containers and databases #142 🙏 @brihadish
  • Propulsion.Tool: sync cosmos from <cosmos|json> #252 #263

Changed

  • Targeted net6.0 with 6.0.300 SDK, FSharp.Core v 6.0.0, FSharp.Control.TaskSeq v 0.4.0, MathNet.Numerics v 4.15.0
  • Changed all Tuple types to struct tuples (System.ValueTuple) #169
  • Changed all Async to Task #192
  • Dispatcher: Replaced GetConsumingEnumerable with System.Threading.Channels #155
  • Ingester,Submitter: Replaced Async.Sleep with Task.WhenAny; Condensed logging #154
  • Pipeline: Renamed AwaitShutdown to Await, added Wait #192
  • Projector: Replaced Async with task for projector supervision #156
  • Scheduler: Replaced Concurrent* with System.Threading.Channels #156
  • Scheduler: Struct/voption conversions; buffer reuse #157
  • Scheduler: Replaced Thead.Sleep with Task.WhenAny; Added Sleep time logging #161
  • Streams: Changed dominant ITimelineEvent EventBody type from byte[] to System.ReadOnlyMemory<byte> (Sinks.EventBody) #169 #208
  • Propulsion.CosmosStore: Changed to target Equinox.CosmosStore v 4.0.0 #139
  • Propulsion.CosmosStore.CosmosSource: Changed parsing to use System.Text.Json #139
  • Propulsion.CosmosStore.CosmosStoreSink+CosmosStorePruner: Exposed *Stats #226
  • Propulsion.EventStore: Pinned to target Equinox.EventStore v [3.0.7-3.99.0] Deprecated; Please migrate to Propulsion.EventStoreDb #139
  • Propulsion.EventStoreDb.EventStoreSource: Changed API to matchPropulsion.SqlStreamStore API rather thanPropulsion.EventStore #139
  • Propulsion.Feed: Moved implementations into main Propulsion library. While this adds a FSharp.Control.TaskSeq dependency, it makes maintenance and navigation easier #265
  • Propulsion.Feed,Kafka: Replaced Async with task for supervision #158, #159
  • Propulsion.Kafka: Target FsCodec.NewtonsoftJson v 3.0.0 #139
  • Propulsion.Prometheus: Extracted Propulsion.Prometheus and Propulsion.Feed.Prometheus in order to remove Prometheus dependency from core package #265
  • Propulsion.Tool: project renamed to sync; sources now have a from prefix #252

Removed

  • Streams.StreamSpan: Changed from a record to individual arguments of FsCodec.StreamName and Sinks.Event[] #169 #208
  • Streams.SpanResult: Replaced with int64 to reflect the updated position #264 #208
  • Streams: statsInterval is obtained from the Stats wherever one is supplied #208
  • Propulsion.Cosmos: Should not be in general use - users should port to Propulsion.CosmosStore3, then Propulsion.CosmosStore #193
  • Destructurama.FSharp dependency #152
  • Ingester,Submitter: Removed requirement to specify sleep intervals #154
  • Scheduler: Removed need to tune idleDelay except in extreme circumstances #161

Fixed

  • Propulsion.CosmosStore.CosmosStoreSource: Propagate correlationId/causationId #260
  • Propulsion.SqlStreamStore: Replaced incorrect/meaningless stream name for SqlStreamStoreSource #139
  • Synced AwaitTaskCorrect with official version 3c11142
  • Projector,Ingester,Submitter, Scheduler: Deterministic shutdown via Cancellation and/or unhandled exceptions #154

New Contributors

Full Changelog: 2.12.2...3.0.0