A Golang primarily based structured programming and distributed processing platform that allows extremely concurrent, scalable, and maintainable workflows for Machine Studying and Knowledge Processing. It’s a material that connects disparate computation backends utilizing a type-safe knowledge dependency graph. It information all adjustments to a pipeline, making it attainable to rewind time. It additionally shops a historical past of all executions and offers an intuitive UI, CLI, and REST/gRPC API to work together with the computation.
Flyte is greater than a workflow engine — it makes use of workflow as a core idea, and process (a single unit of execution) as a top-level idea. A number of duties organized in a knowledge producer-consumer order create a workflow.
Workflows and Duties could be written in any language, with out-of-the-box help for Golang, Python, Java, and Scala. Flyte was designed to handle the complexity of Knowledge and ML groups and guarantee they sustain their excessive velocity of delivering business-impacting options. A method it achieves that is by separating the management aircraft from the consumer aircraft. Thus, each group can provide Flyte as a service to end-users the place extra infrastructure-focused people handle the service. On the similar time, the customers use the intuitive interface of Flytekit.
Why utilizing the Golang primarily based Flyte
- Kubernetes-native workflow automation platform
- Knowledge-aware and strongly-typed
- Ergonomic SDKs in Python, Java, and Scala
- Versioned, auditable, and reproducible pipelines
- Useful resource-aware and deployments at scale
