This operate is to fetch some xml content material and put it aside. One subject i encounterd is that the right way to ship case s.updates ← pending[0]: (pending Merchandise) separately. the code presently has an error of accessing index out of certain.
right here’s the hyperlink to the complete code : Higher Go Playground
func(s *sub) loop(){
var pending []Merchandise // appended by fetch; consumed by ship
var subsequent time.Time
var err error
var seen = make(map[string]bool) // set of merchandise.GUIDS
for{
// var updates chan Merchandise
var fetchDone chan fetchResult// if non-nil, fetch is working
var fetchDelay time.Period // initally 0
if now := time.Now(); subsequent.After(now) {
fetchDelay = subsequent.Sub(now)
}
var startFetch <-chan time.Time
if fetchDone == nil && len(pending) < maxPending {
startFetch = time.After(fetchDelay)
}
choose{
case errc := <-s.closing:
errc <- err
shut(s.updates)// tells receiver we're achieved
return
case <-startFetch:
fmt.Println("begin fetch")
var fetched []Merchandise
fetchDone = make(chan fetchResult, 1)
go func(){
fetched, subsequent, err = s.fetcher.Fetch()
fetchDone <- fetchResult{fetched, subsequent, err}
}()
case s.updates <- pending[0]:
fmt.Println("add pending")
if len(pending) > 0 {
pending = pending[1:]
}
case consequence := <-fetchDone:
fmt.Println("fetch achieved")
fetchDone = nil
if consequence.err != nil {
subsequent = time.Now().Add(10 * time.Second)
break
}
for _, merchandise := vary consequence.fetched {
if !seen[item.Channel.Link] {
pending = append(pending, merchandise)
seen[item.Channel.Title] = true
}
}
}
}
}
Certain, I’d be glad to assist enhance your Go concurrency code. May you please present the code you’re working with? Then, I can provide options for optimization or any enhancements primarily based in your particular necessities.
Hey there,
The code within the hyperlink Higher Go Playground
The code is about fetching .xml information on the net, and retailer it, I needed to fetch 10 objects at a time. and retailer it separately by passing to a channel named updates.
Thanks
There are a variety of issues with this code. First is scoping of variables: The variable fetchDone is asserted contained in the for loop, so it’s reset after every iteration. This breaks most of your logic.
General your code could be very complicated for what you are attempting to perform. There are established sample for writing parallel capabilities (fan out, fan in) with sync.WaitGroup or related mechanisms.
Attempt to construct your code very straightforward to learn: A gaggle of employee go-routines, which all learn from a single channel the place work comes and and push their consequence right into a single channel the place the outcomes come out. And use the for w := vary over channel to deal with closing of channels gracefully
1 Like
