Looking for Suggestions for gojob: A Job Scheduler Library for GoLang – Code Assessment

Hey guys,

Excited to share my venture, gojob! It’s a easy Job Scheduler library for GoLang designed for activity sharding, retry on failure, logging, and standing reporting.

Whether or not you’re constructing an online app, a microservice, or an information processing pipeline, gojob makes activity administration a breeze.

I’m all ears for suggestions on design, performance, or usability. Professionals’ insights will assist refine this venture and my expertise to satisfy business requirements.

Options:

1. Concurrent Job Execution: Handles a number of duties concurrently.
2. Logging Assist: Tracks activity execution and errors seamlessly.
3. Standing Reporting: Actual-time monitoring with customizable choices.
4. Ease of Use: Easy API and intuitive design for straightforward integration.
5. Monitoring Integration: Works easily with Prometheus, Grafana, and extra.

Take a look at this fast instance (a concurrent HTTP crawler) to see how easy gojob can used:

Mainly you simply must implement the Job interface and submit it to the scheduler. Right here’s an instance of a concurrent http crawler utilizing gojob:

// Job is an interface that defines a activity
sort Job interface {
	// Do begins the duty, returns error if failed
	// If an error is returned, the duty might be retried till MaxRetries
	// You'll be able to set MaxRetries by calling SetMaxRetries on the scheduler
	Do() error
}

package deal foremost

import (
	"fmt"
	"time"

	"github.com/WangYihang/gojob"
)

sort MyTask struct {
	Url        string `json:"url"`
	StatusCode int    `json:"status_code"`
}

func New(url string) *MyTask {
	return &MyTask{
		Url: url,
	}
}

func (t *MyTask) Do() error {
	response, err := http.Get(t.Url)
	if err != nil {
		return err
	}
	t.StatusCode = response.StatusCode
	defer response.Physique.Shut()
	return nil
}

func foremost() {
	var numTotalTasks int64 = 256
	scheduler := gojob.New(
		gojob.WithNumWorkers(8),
		gojob.WithMaxRetries(4),
		gojob.WithMaxRuntimePerTaskSeconds(16),
		gojob.WithNumShards(4),
		gojob.WithShard(0),
		gojob.WithTotalTasks(numTotalTasks),
		gojob.WithStatusFilePath("standing.json"),
		gojob.WithResultFilePath("consequence.json"),
		gojob.WithMetadataFilePath("metadata.json"),
	).
		Begin()
	for i := vary numTotalTasks {
		scheduler.Submit(New(fmt.Sprintf("https://httpbin.org/activity/%d", i)))
	}
	scheduler.Wait()
}

Discover extra data, examples (e.g. concurrent tcp port scanner), and the supply code on GitHub. Your suggestions and options are welcome. Let’s make coding extra pleasurable collectively!