Testing Automation – Website Suggestions

I constructed Sleuth (GitHub – ruth-tirusew/sleuth: experimental… · GitHub), a Go CLI device for operating integration and state of affairs flows outlined in declarative YAML.

I constructed it to resolve just a few particular complications I bumped into with API integration testing:

• Silent API Drift: Endpoints altering payload shapes with out breaking commonplace assertions.

• Advanced Step Dependencies: Having to manually sequence steps in YAML as an alternative of letting a graph deal with execution order.

• Database Verification: Desirous to confirm database side-effects mid-flow with out risking unintended writes or state air pollution.

I’m nonetheless experimenting with the concept and open to collaborations please do let me know what you suppose

I had a glance by way of the repo, particularly the behavioural snapshot checks. I feel that half is fairly attention-grabbing, since API modifications that don’t break current assertions may be surprisingly simple to overlook.

One factor I’m interested by is the way you’d suggest dealing with snapshot merging in a bigger undertaking.

From what I perceive, -merge can increase the accepted response form based mostly on a number of profitable runs. That is smart for fields which can be genuinely non-obligatory, however I ponder if it might additionally make sudden API modifications more durable to catch over time, particularly when totally different environments or API variations are concerned.

Possibly conserving separate baselines per surroundings or API model would assist? I’m not solely positive what the most effective strategy could be right here.

I haven’t run the CLI but, so that is extra of a query from studying the implementation and checks than a bug report.

I’d have an interest to listen to the way you see groups managing these snapshots in CI as soon as they’ve fairly just a few flows.

Thanks for the shut learn. You’re proper about -merge: it solely ever loosens a snapshot, so merging in a run from a nasty surroundings or an previous API model means the snapshot then accepts that form.

What I’d suggest:

• Hold one snapshot per surroundings and API model, for instance snapshots//checkout.json, and examine every surroundings in opposition to its personal baseline. At the moment that’s executed by file naming. Native per-environment help isn’t constructed but past recording the surroundings within the file.
• Use -merge just for a number of known-good runs of the identical stream in opposition to the identical goal. That’s what it’s for: studying which fields are actually non-obligatory. Don’t merge throughout environments that ought to behave in a different way.
• Hold recording and merging out of CI. CI ought to solely examine, with -shapes FILE -shape-drift-fail breaking. Recording refuses to overwrite a snapshot that differs, and -force retains a .prev copy. Snapshot updates then present up as a reviewed git diff.
• Deal with required-to-optional modifications as one thing to assessment. Use assertions and contracts for the fields that should at all times be current.

I feel your “type” subject is superfluous. type: “db” is immediately adopted by “db: …” so it’s redundant and susceptible to interrupt on copy & pasting or enhancing.

Thanks for beginning Testing Automation — it’s useful seeing how others strategy testing and automation. I just lately up to date my very own bookmarks for testing after testing a number of neighborhood suggestions.

You possibly can try the one I switched to over at guide-index-hub.pages.dev/r/forum_golangbridge_org