I constructed Sleuth (GitHub – ruth-tirusew/sleuth: experimental… ยท GitHub), a Go CLI instrument for working integration and situation flows outlined in declarative YAML.
I constructed it to unravel a number of particular complications I bumped into with API integration testing:
-
Silent API Drift: Endpoints altering payload shapes with out breaking normal assertions.
-
Complicated Step Dependencies: Having to manually sequence steps in YAML as an alternative of letting a graph deal with execution order.
-
Database Verification: Eager to confirm database side-effects mid-flow with out risking unintentional writes or state air pollution.
Iโm nonetheless experimenting with the thought and open to collaborations please do let me know what you suppose
I had a glance by means of the repo, particularly the behavioural snapshot checks. I believe that half is fairly attention-grabbing, since API adjustments that donโt break present assertions might be surprisingly straightforward to overlook.
One factor Iโm inquisitive about is the way youโd advocate dealing with snapshot merging in a bigger venture.
From what I perceive, -merge can increase the accepted response form primarily based on a number of profitable runs. That is smart for fields which might be genuinely optionally available, however I ponder if it may additionally make surprising API adjustments more durable to catch over time, particularly when totally different environments or API variations are concerned.
Possibly preserving separate baselines per surroundings or API model would assist? Iโm not fully certain what one of the best method could be right here.
I havenโt run the CLI but, so that is extra of a query from studying the implementation and checks than a bug report.
Iโd have an interest to listen to the way you see groups managing these snapshots in CI as soon as they’ve fairly a number of flows.
