Re-posted from: https://bkamins.github.io/julialang/2022/03/18/manning.html
Introduction
One of the projects I am currently working on is writing the
Julia for Data Analysis book. In this post I want to give you a brief
overview of the contents of the book and its current status.
What is the objective of the book?
The book is aimed at data scientists with some programming experience wanting
to learn how to do data analysis in Julia. Some previous experience with Julia
would be a plus, but it is not strictly required.
Since the book is aimed as an entry-level introduction to data analysis in Julia
I have divided it into two parts:
- Part 1 is teaching you the essential Julia skills you need to learn to
confidently use the language. - Part 2 is focused on data analysis.
I do not assume that you know the Julia language and in Part 1 explain the basic
components of the language from the very beginning. However, I assume that you
have some experience with programming (e.g. in R or Python).
Part 2 focuses on data analysis in Julia. In it I concentrate a lot on
the functionalities of the DataFrames.jl package.
It is impossible to cover every aspect of data analysis in Julia in a single
book. Therefore, my goal was to discuss all essential material that will
allow you to later confidently learn more advanced things on your own, while
being convenient that you have a firm grasp of the fundamental concepts.
What is the scope of the book?
Most of the chapters in the book are project oriented. I introduce new concepts
and functionalities of the Julia ecosystem by showing how they can be used to
solve practical problems.
Since the scope of the book is quite wide let me here list selected technical
topics that I discuss in it which, I think, are useful even for people who
already know Julia:
- working with source data in various formats: CSV, Apache Arrow, SQLite, JSON,
serialization, parsing data that has non-standard format; - getting data from external sources (like: downloading data, unpacking
compressed data, getting data using HTTP requests, finishing with writing
a web service on your own); - solving typical issues encountered when pre-processing data: efficient
handling of strings, using categorical values,
working with time-series, understanding missing values. - coverage of key functionalities of the DataFrames.jl package and
DataFramesMeta.jl package (creating data frames, transforming them,
split-apply-combine strategy, joining, sorting, subsetting, reshaping); - creating plots that visualize the results of your analysis using the Plots.jl
package; - building simple predictive models (linear regression, logistic regression,
LOESS) and assessment of the model performance; - integrating Julia code with R and Python.
Notably, I have not covered in this book more advanced topics on machine
learning. The reason is that there are too many options available, so
this would be a material for another book. However, as I have already mentioned,
the book is written in a way so that after reading it you should be able to
easily learn the functionalities of the concrete packages that provide such
functionalities, like e.g. MLJ.jl, yourself.
For example, after reading this book you might want to check out my
Hands-on Data Science with Julia live project that gives you an
introduction to machine learning with Julia.
Where to get the book?
The book will be published by Manning. This week its first two chapters
have been made available in MEAP (Manning Early Access Platform).
You can find a preview of the book under this link.
If you are interested in the topic I encourage you to visit the website,
as MEAP program gives the readers an opportunity to give feedback about the
material I have prepared before the final version of the book is published.
What is the status of the book?
All core chapters have been already written. Now we are going through a review
and publishing process. I will write another post when the book is finalized.
However, since all the technical material is already prepared, you can get the
source codes of all chapters from
GitHub.
Codes are shared there are under MIT license so you can freely reuse them.
Conclusions
Today, as a conclusion, let me pick one example code from the book (codes are
also available on GitHub).
The example code produces a plot that compares speed and code size of Julia,
Python, Java, and C on 10 standard problems taken from the
The Computer Language Benchmarks Game website.
The example is self-contained and was tested under Julia 1.7.0,
DataFrames.jl 1.3.2, CSV.jl 0.10.3, and Plots.jl 1.27.1.
data = """
problem,language,time,size
n-body,c,2.13,1633
mandelbrot,c,1.3,1135
spectral norm,c,0.41,1197
fannkuch-redux,c,7.58,910
fasta,c,0.78,1463
k-nucleotide,c,3.96,1506
binary-trees,c,1.58,809
reverse-complement,c,0.41,1965
pidigits,c,0.56,1090
regex-redux,c,0.8,1397
n-body,Java,6.77,1489
mandelbrot,Java,4.1,796
spectral norm,Java,1.55,756
fannkuch-redux,Java,10.48,1282
fasta,Java,1.2,2543
k-nucleotide,Java,4.83,1812
binary-trees,Java,2.51,835
reverse-complement,Java,1.57,2183
pidigits,Java,0.79,764
regex-redux,Java,5.34,929
n-body,Python,541.34,1196
mandelbrot,Python,177.35,688
spectral norm,Python,112.97,407
fannkuch-redux,Python,341.45,950
fasta,Python,36.9,1947
k-nucleotide,Python,46.31,1967
binary-trees,Python,44.7,660
reverse-complement,Python,6.62,814
pidigits,Python,1.16,567
regex-redux,Python,1.34,1403
n-body,Julia,4.21,1111
mandelbrot,Julia,1.42,619
spectral norm,Julia,1.11,429
fannkuch-redux,Julia,7.83,1067
fasta,Julia,1.13,1082
k-nucleotide,Julia,4.94,951
binary-trees,Julia,7.28,634
reverse-complement,Julia,1.44,522
pidigits,Julia,0.97,506
regex-redux,Julia,1.74,759
"""
using CSV
using DataFrames
using Plots
df = CSV.read(IOBuffer(data), DataFrame)
plot(map([:time, :size],
["execution time (relative to C)",
"code size (relative to C)"]) do col, title
df_plot = unstack(df, :problem, :language, col)
df_plot[!, Not(:problem)] ./= df_plot.c
select!(df_plot, Not(:c))
scatter(df_plot.problem, Matrix(select(df_plot, Not(:problem)));
labels=permutedims(names(df_plot, Not(:problem))),
ylabel=title,
yaxis = col == :time ? :log : :none,
xrotation=20,
markershape=[:rect :diamond :circle],
markersize=[4 5 5],
markercolor=[:lightgray :lightgray :gold],
xtickfontsize=7, ytickfontsize=7,
legendfontsize=7, ylabelfontsize=7)
hline!([1.0]; color="orange", labels="C")
end...)
(the code is not easy, but after reading the whole book you should be able to
confidently read it and create a similar implementation yourself)
The figure produced by this code looks as follows.
If you would like to read a complete interpretation of the plot please check
Chapter 1 in Julia for Data Analysis book. Here let me just summarize
that Julia runs the code fast (left pane) and at the same time is convenient to
use (as measured by the code size; right pane).