Utilizing Conda? You may not want Docker

Docker packaging is beneficial, however doing it effectively will not be straightforward.
Even limiting the scope of dialogue to manufacturing use of Python functions, the variety of particulars to cowl is in depth sufficient that I’ve written over 50 articles on the subject, and created quite a lot of merchandise to hurry up the packaging course of.

In a greater universe, none of this is able to be essential.

So whereas Docker is usually helpful sufficient to benefit this effort, in some conditions you is likely to be higher off avoiding Docker altogether.
Particularly, Conda affords a number of the advantages of Docker.
And whereas Conda actually has its personal points, utilizing Conda by itself will contain much less work than a mixture of Conda on Docker.

This text will cowl:

• Improvement environments:
  • The advantages of Docker packaging.
  • The downsides.
  • Why Conda would possibly allow you to keep away from the downsides and nonetheless have the identical advantages.
• Manufacturing, the place Docker’s advantages is likely to be changed by a mixture of framework performance and Conda.

Docker for improvement environments

Whenever you’re working in your software program regionally, throughout improvement, Docker has a number of advantages.

First, you would possibly have to run further providers like PostgreSQL; Docker, and particularly Docker Compose, make this rather a lot simpler.
Conda doesn’t actually assist a lot with this, so if this can be a requirement Docker could be the appropriate resolution.

Second, you might need builders utilizing a number of working techniques, which raises some difficulties:

• The runtime setting will differ for every developer primarily based on their working system.
  By working on Docker, you may run the applying persistently on the identical working system, no matter variant of Linux your picture relies on.
• Set up totally different instruments persistently throughout a number of working techniques will be troublesome.
  You possible want quite a lot of utilities accessible: git, black, flake8, maybe a compiler.
  On Linux you would possibly use apt or dnf or different instruments, on macOS you’ll need brew, and on Home windows, good luck, not one of the package deal repositories appear all that good.
  Docker may give you entry to all these instruments by placing them in a Linux picture, so that you solely want to put in them a method.

There are caveats, nevertheless: whereas Docker photos do run persistently throughout totally different working techniques, they’re not precisely the identical, and these variations change into extra vital in improvement environments.

• When mounting a bunch listing as a quantity, a typical state of affairs in improvement environments, on macOS and Home windows any writes shall be owned by the host’s person. On Linux, the created information could have a distinct UID, so that you must take steps to run the container because the host person’s UID, which will be tough if the picture wasn’t designed for this.
• And if you wish to use host.docker.inside to entry processes working on the host, that gained’t work in Linux out of the field.

Python and Conda: cross-platform logic, cross-platform dependencies

If you happen to’re writing a Python venture and also you’re counting on Conda, and particularly Conda-Forge, the advantages of Docker for a improvement setting change into rather less compelling.

In the case of the runtime setting, Python already creates a reasonably cross-platform abstraction layer.
Whether or not it’s opening information or networking, the Python APIs and libraries that construct on them normally summary away many of the platform-specific particulars.

That doesn’t assist with command-line utilities (at the very least non-Python ones), compilers, and native libraries you would possibly rely upon… however that is the place Conda is available in.
Conda packages every thing however the usual C library, from C libraries to the Python interpreter to command-line instruments to compilers.
And Conda-Forge has an enormous number of instruments, libraries, and common packages accessible, pre-compiled for Home windows, macOS, and Linux.

So as a substitute of apt + brew + no matter you’re doing on Home windows, you may have a single constant packaging config: setting.yml, optionally pinned with conda-lock’s multi-OS lock information for additional consistency.
It’s going to record all of the instruments, libraries, compilers, Python dependencies, and so forth that that you must run and develo your code, and it’ll set up persistently throughout working techniques.
No want for Docker!

title: yourapp
channels:
  - conda-forge
dependencies:
  - python=3.9
  - flake8
  - black
  - git
  - compilers

To summarize, for the use case of Python improvement environments, right here’s the way you would possibly select options to Docker:

Conda as a Docker various for manufacturing

Can Conda operate as an alternative choice to Docker in manufacturing?

Given a manufacturing picture will sometimes be working on Linux solely, cross-platform consistency turns into much less of a problem.
However a Docker picture combines two issues we do need:

• A constant set of information that may run on Linux, together with:
  • All dependencies.
  • The precise software code.
• A selected command to run on startup.

As we mentioned for improvement environments, getting constant dependencies through Conda is usually an inexpensive various to Docker.
The one factor that may rely upon the host working system is glibc, just about every thing else shall be packaged by Conda.
So a pinned setting.yml or conda-lock.yml file is an inexpensive various to a Docker picture so far as having constant dependencies.

That also leaves getting your software code, and working it.

In lots of circumstances the framework you might be utilizing to run your code can set up Conda packages and distribute your code to regardless of the runtime setting is, and also will management what will get run.
This implies you don’t really want a Docker picture.
For instance:

All of those frameworks will allow you to set what command or code will get run, relying on their abstraction stage.

Some limitations to avoiding Docker

First, in the event you’re not utilizing a Docker picture it’s much more essential than ordinary to document the model of the code you’re utilizing within the output, so you may reproduce all the mandatory code later.

Second, there’s the bootstrapping drawback: relying on the framework you’re utilizing, you would possibly want to put in Conda and the framework driver earlier than you will get something going.
A Docker picture would come prepackaged with each, along with your code and its dependencies.
So even when your framework helps Conda immediately, you would possibly wish to use Docker anyway.

Third, relying on how good the framework is, you would possibly end up putting in Conda packages over and over on each run.
That is inefficient, even when utilizing a quicker installer like Mamba.

You don’t want containers for every thing

Nevertheless helpful they’re, containers are simply one resolution out of many, and in some conditions Conda may give you lots of the advantages with much less complexity.
Each instrument has its candy spot, and virtually each instrument has options, so it’s price spending at the very least a bit of time contemplating in case your default instrument is the perfect one for the job.