Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
d7ca5c3
Sketched `DatasetUpdateFlow` aggregate
zaychenko-sergei Nov 16, 2023
8b12280
Separated Update flow on Ingest and ExecuetQuery
zaychenko-sergei Dec 18, 2023
9348267
Implemented systematic tracing for flow manager services/repors and d…
zaychenko-sergei Dec 18, 2023
4ec1b59
Tests for dependency graph
zaychenko-sergei Dec 19, 2023
ec82cb7
EventBus tests
zaychenko-sergei Dec 19, 2023
2cb8387
Unit tests for flow time wheel
zaychenko-sergei Dec 20, 2023
bb48ef4
Tests for flow configurations service
zaychenko-sergei Dec 20, 2023
9dcd386
Candidate flow system test structure
zaychenko-sergei Dec 21, 2023
0ef4665
Rounding time to step in flow system to get stable activation times
zaychenko-sergei Dec 21, 2023
2d994c6
Tested and fixed manual flow triggers
zaychenko-sergei Dec 22, 2023
0e1a94e
Release + minor deps + merge correction
zaychenko-sergei Dec 22, 2023
08ea032
Minimizing fow tests randomness with fixed planned start time and flo…
zaychenko-sergei Dec 22, 2023
859cfad
Tested deleting dataset with queued/scheduled flow.
zaychenko-sergei Dec 22, 2023
978f056
New test: pause/resume/modify flow schedule when queued already.
zaychenko-sergei Dec 22, 2023
762d256
New test: flow finishes with different task outcomes, enqueueing next…
zaychenko-sergei Dec 22, 2023
5cfbe8b
New test: root update triggers derived dataset + fixes
zaychenko-sergei Dec 22, 2023
68ccb92
Disabled flow tests on Windows
zaychenko-sergei Dec 23, 2023
fc57e10
Changelog updated
zaychenko-sergei Dec 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,27 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased
## [0.149.0] - 2023-12-23
### Added
- Added `KAMU_WORKSPACE` env var to handle custom workspace path if needed
- Added `event-bus` crate: an utility component based on Observer design pattern,
which allows event producers and event consumers not to know about each other
- Applied `event-bus` component to inform consumers of dataset removal, dependency changes,
task completions
- Added in-memory dataset dependency graph instead of continous rescanning of all datasets:
- the initial dependencies are computed on demand on first request
- using `petgraph` project to represent dataset dependencies in the form of directed acyclic graph
- further events like new/removed dependency or dataset removal update the graph
- simplified GraphQL APIs and dataset removal check using new dependency graph
- Added prototype of flow management system:
- flows are automatically launched activities, which are either dataset related or represent system process
- flows can have a schedule configuration, using time delta or CRON expressions
- flows system manages activation of flows according to the dynamically changing configuration
- flows system manages triggering of dependent dataset flows, when their inputs have events
- derived flows may have throttling settings
### Changed
- Integrated latest `dill=0.8` version, which removes a need in registering simple dependency binds
- Using new `dill=0.8` to organize bindings of structs to implemented traits via declarative attributes

## [0.148.0] - 2023-12-20
### Added
Expand Down
Loading