JuliaCon2020: Julia is production ready!

By: Blog by Bogumił Kamiński

Re-posted from: https://bkamins.github.io/julialang/2020/08/07/production-ready.html

Introduction

Last time I have posted about the take-aways for DataFrames.jl from
JuliaCon 2020. This time I wanted to share my general conclusions
from attending different talks at this extremely successful event.

I have spent 20 years now deploying data science related projects in corporate
environments (back then it was not called data science, but we were already
training neural networks to make predictions) and have many colleagues who are
deeply into enterprise software development. Quoting a discussion with
Tomasz Olczak I had some time ago, who is a genuine one-man army in
delivering complex enterprise projects:

Julia is fast, and has a very nice syntax, but its ecosystem is not mature
enough for use in serious production projects.

For many years I would agree with it, but after JuliaCon 2020
I believe we can confidently announce that

Julia is production ready!

Let me now give a list of key (in my opinion) presentations given during
JuliaCon 2020 that make me draw this conclusion.

I will not comment here on functionalities related to number crunching, as it
is clear that Julia shines here, but rather I want to focus on the things that
make Julia a great tool for deployment in production (still I skip many
interesting talks in this area — check out the detailed agenda to
learn more).

Building microservices and applications

In this talk Jacob Quinn gives an end to end tutorial how
to build and deploy in an enterprise setting a microservice using Julia.
He gives ready recipes how to solve typical tasks that need to be handled in
such contexts: logging, context management, middleware setup, authentication,
caching, connection pooling, dockerization, and many other, that
are bread and butter of enterprise projects.

As an addition be sure to check out:

  • the shippable apps talk, where Kristoffer Carlsson guides you
    through creating executables which can be run on machines that do not have Julia
    installed.
  • the Julia for scripting presentation, during which
    Fredrik Ekre discusses the best practices for using Julia in contexts
    where you need to execute short code snippets many times.
  • the Genie.jl talk, in which Adrian Salceanu shows that it is currently
    a mature, stable, performant, and feature-rich Julia web development framework.

Dependency management

The two talks Pkg.update() and What’s new in Pkg show that
currently Julia has best in class functionalities for enterprise grade dependency
management for your projects. The list of provided functionalities is so long
that it is hard to list them all here.

Let me just mention one particular tool in this ecosystem, that is presented in
BinaryBuilder.jl talk that explains how to take software written in compiled
languages such as C, C++, Fortran, Go or Rust, and build precompiled artifacts
that can be used from Julia packages with ease (which means that no compilation
has to take place on client side when you install packages having such dependencies).

Integration with external libraries

A natural topic related to dependency management is how to integrate Julia with
external tools. This area of functionality is really mature. Here is a list of
talks that cover this topic:

Here it is worth to add Julia has had a great integration with Python for many
years now, see JuliaPy.

A good end to end example of doing some real work in Julia that requires integration
is Creating a multichannel wireless speaker setup with Julia
talk that show how to easily stitch things together (and in particular featuring
ZMQ.jl, Opus.jl, PortAudio.jl, and DSP.jl).

Another interesting talk showcasing integration capabilities is
JSServe: Websites & Dashboards in Julia
that shows a high performance framework to easily combine interactive plots,
markdown, widgets and plain HTML/Javascript in Jupyter / Atom / Nextjournal and
on websites.

Developer tooling

The two great talks Juno 1.0 and Using VS Code that current IDE
support for Julia in VS Code is first class. You have all tools that normally you
would expect to get: code analysis (static and dynamic), debugger, workspaces,
integration with Jupyter Notebooks, and remote capabilities.

Managing ML workflows

I do not want to cover many different ML algorithms that are available in Julia
natively, as there are just too many of them (and if something is missing you
can easily integrate it — see the integration capabilities section above).

However, on top of particular models you need frameworks that let you manage
ML workflows. In this area there are two interesting talks, one about
MLJ: a machine learning toolbox for Julia and the other showing
AutoMLPipeline: A ToolBox for Building ML Pipelines. From my experience
such tools are crucial when you want to move with your ML models from data scientist’s
sandbox to a real production usage.

Conclusion

Obviously, I have omitted many interesting things that were shown during
JuliaCon 2020. However, I hope that the aspects I have covered here,
that is:

  • enterprise grade patterns to create microservices and applications in Julia,
  • robust dependency management tools,
  • very flexible and powerful capabilities to integrate Julia with existing code
    bases that were not written in Julia,
  • excellent developer tooling in VSCode,
  • mature packages that help you to create production-grade code for ML solutions
    deployment,

show that already now Julia can (and should) be considered as a serious option
for your next project in enterprise environment.

What I believe is crucially important is that not only we have required tools
ready but also we have great practical showcases how they can be used to build
robust production code with.