Tag Archives: beginner

Using JSON Web APIs from Julia

By: Josh Day

Re-posted from: https://www.juliafordatascience.com/how-to-wrap-a-json-web-api/

Enjoying Julia For Data Science?  Please share us with a friend and follow us on Twitter at @JuliaForDataSci.

Using JSON Web APIs from Julia

A common way to access data is through web APIs that return a JSON response.  In this post we'll look at how the XKCD.jl package implements a user-friendly interface for accessing the xkcd webcomic's JSON interface.

Getting the Data

The xkcd webcomic's JSON interface accepts request of the form:

  1. https://xkcd.com/info.0.json (metadata for most recent comic)
  2. https://xkcd.com/<i>/info.0.json (metadata for i-th comic)

We'll use the HTTP package's get function to retrieve this info:

julia> using HTTP

julia> res = HTTP.get("https://xkcd.com/552/info.0.json");

Interpret Data as JSON

We can then read our result body into JSON using the JSON3 package:

julia> using JSON3

julia> JSON3.read(res.body)
JSON3.Object{Vector{UInt8}, Vector{UInt64}} with 11 entries:
  :month      => "3"
  :num        => 552
  :link       => ""
  :year       => "2009"
  :news       => ""
  :safe_title => "Correlation"
  :transcript => "[[A man is talking to a woman]]\nMan: I used to think correla…
  :alt        => "Correlation doesn't imply causation, but it does waggle its e…
  :img        => "https://imgs.xkcd.com/comics/correlation.png"
  :title      => "Correlation"
  :day        => "6"

Making it User-Friendly

To make using the xkcd JSON interface easier to use, let's create a struct to hold the JSON data and let its constructor do the heavy lifting:

struct Comic 
    json::JSON3.Object
end

Comic() = Comic(JSON3.read(HTTP.get("https://xkcd.com/info.0.json").body))
    
Comic(i::Int) = Comic(JSON3.read(HTTP.get("https://xkcd.com/$i/info.0.json").body))

🎉 Now getting comic metadata is as simple as Comic(552)!

What about Authorization?

The xkcd API does not require authorization.  However, many APIs require that you purchase an API key in order to access them.  In this case, it's best practice to try to ensure API keys won't end up being accidentally saved in git (or other version control).

To do this, we can rely on environmental variables (ENV in Julia).  Suppose the xkcd API requires an apikey parameter.  We would want to change our Comic constructor to something like:

function Comic(i::Int)
    apikey = ENV["MY_API_KEY"]
    url = "https://xkcd.com/$i/info.0.json?apikey=$apikey"
    data = JSON3.read(HTTP.get(url).body)
    Comic(data)
end
Imaginary apikey parameter in xkcd API

Other Neat Stuff

Since we working with images (comics) in the XKCD.jl package, we can add a pretty show method so that the comic image (not just the metadata) will appear in coding environments like Jupyter and Pluto.

function Base.show(io::IO, ::MIME"text/html", c::Comic)
    show(io, MIME"text/html"(), HTML("""
	   <div>
        <h2><code>XKCD.Comic</code> $(c.json.num): $(c.json.title)</h2>
        <img src="$(c.json.img)" alt="$(c.json.alt)" title="$(c.json.alt)">
		<div>
          <a href="https://xkcd.com/$(c.json.num)">
            Link to Original
          </a>
	    </div>
      </div>
    """))
end
Using JSON Web APIs from Julia
XKCD.jl + Pluto.jl = ❤️

🚀 That's It!

Enjoying Julia For Data Science?  Please share us with a friend and follow us on Twitter at @JuliaForDataSci.

Animations with Plots.jl

By: Josh Day

Re-posted from: https://www.juliafordatascience.com/animations-with-plots-jl/

Enjoying Julia For Data Science?  Please share us with a friend and follow us on Twitter at @JuliaForDataSci.


Why Animations are Great

Animations with Plots.jl

The ability to communicate results is an under-appreciated skill in data science.  An important analysis can be unheard or misunderstood if it's not presented well.  Let's borrow the model for data science projects proposed by R for Data Science, in which Communicate is the final step.  

Animations with Plots.jl
R for Data Science: Model for Data Science Projects

Animations tell a story that static images are unable to tell by adding an extra dimension (often time).  They are also more engaging to an audience (here is one of many social marketing blogs on the topic of engagement from video vs. static images).  Getting your audience to pay attention is a part of communicating your results, so animations are a great tool.

Animations with Plots.jl

Plots.jl is a Julia package that provides a unified syntax for multiple plotting backends.  It also provides some super simple and powerful utilities for creating animations in the gif format.  There are several ways to create animations in Plots, with varying levels of complexity.  We recommend using Pluto (see our Pluto introduction here) to make Plots animations because they'll appear in the notebook.

The simplest way is the @gif macro.

  • Place @gif in front of a for loop that generates a plot in each iteration.  Each plot will be saved as a single frame in the animation.
using Plots 

@gif for i in 1:50
    plot(sin, 0, i * 2pi / 10)
end
Animations with Plots.jl
  • You can add "flags" such as every n to only save a frame every n images. or when <condition> to only save certain frames.
@gif for i in 1:50
    plot(sin, 0, i * 2pi / 10)
end when i > 30
Animations with Plots.jl

To control frames-per-second, use @animate.

  • Works just like @gif, but creates a Plots.Animation rather than a gif directly.
anim = @animate for i in 1:50
    Random.seed!(123)
    scatter(cumsum(randn(i)), ms=i, lab="", alpha = 1 - i/50, 
        xlim=(0,50), ylim=(-5, 7))
end
  • You can then use the gif function on your Animation.
gif(anim, fps=50)
Animations with Plots.jl

For the most control, use Plots.Animation directly.

  • Save each frame explicitly with frame.
a = Animation()
	
for i in 1:10
    plt = bar(1:i, ylim=(0,10), xlim=(0,10), lab="")
    frame(a, plt)
end
	
gif(a)
Animations with Plots.jl

🚀 That's It!

You now know how to make some cool animations with Julia and Plots.jl.

Enjoying Julia For Data Science?  Please share us with a friend and follow us on Twitter at @JuliaForDataSci.

Additional Resources

First Steps #2: The REPL

By: Josh Day

Re-posted from: https://www.juliafordatascience.com/first-steps-2-the-repl/

Prerequisite

What is the REPL?

First Steps #2: The REPL

REPL stands for Read-Eval-Print-Loop.  When you open Julia (double-click the icon or type julia in a terminal if you have it on your PATH), you'll enter Julia's REPL.  You'll see something like this:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0 (2021-03-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia>
Julia REPL at Startup

When you type something and press enter, Julia will take your input, evaluate it, and print the result, e.g. typing x = "Hello" and hitting enter will result in:

julia> x = "Hello"
"Hello"

julia>
x = "Hello"

REPL Modes

Julia's REPL has amazing modes that affect how Julia interprets your input.  Here are the modes that are built-in:

? ➡ Help Mode 📖

Help Mode does a few things, best seen through example.  Type ? to enter Help Mode.  Then type println and press enter.

First Steps #2: The REPL
Image of ?println Output
  • The top line displays a fuzzy search of what you searched for.  This will help you find items even if you mistype something.
  • If your search term matches an existing identifier, the documentation for that identifier will print.  In this example our search found the function println.

Press delete to go back to Julia mode and type x = 1.  Now type ?x.  

  • Your search x matches with the x you just created.  However, there is no documentation available, so a summary of x (its type information) is printed out instead.  We'll talk about types in a future post.

; ➡ Shell Mode 💻

This lets you run shell commands (as if you opened a terminal but didn't start Julia).  A benefit of Julia's Shell Mode is that you can interpolate values into a command with $, e.g.

shell> echo $x
1
Interpolating a Julia value into a Shell Mode command.

] ➡ Pkg Mode 📦

Pkg Mode lets you run commands with Julia's package manager (we'll cover this in-depth in a future post) .  In the previous post we added packages with:

using Pkg

Pkg.add("StatsBase")

Alternatively, we could have used Pkg Mode and entered add StatsBase instead 🎉!  On a fresh install of Julia 1.6 you'll see something like:

(@v1.6) pkg> add StatsBase
    Updating registry at `~/.julia/registries/General`
    Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Resolving package versions...
    Updating `~/.julia/environments/v1.6/Project.toml`
  [2913bbd2] + StatsBase v0.33.4
    Updating `~/.julia/environments/v1.6/Manifest.toml`
  [34da2185] + Compat v3.25.0
  [9a962f9c] + DataAPI v1.6.0
  [864edb3b] + DataStructures v0.18.9
	⋮
  [3f19e933] + p7zip_jll
  Progress [========================================>]  10/10
10 dependencies successfully precompiled in 9 seconds
Using Pkg Mode to Install StatsBase

REPL Modes are extendable! 🔌

Julia packages can add their own REPL modes.  Some examples are:

  • $RCall (interpret input as R)
  • <Cxx (interpret intput as C++)

Notable REPL Features

Re-Run a Line

  • Press the up key to display the previously-run command.  Continue pressing up to go through your history.
  • Press ctrl-r to activate a search of your history.  Begin typing and the most recent line that contains your search will appear.  You can continue pressing ctrl-r to move backwards through matching lines in your history and ctrl-s to move forward.  If you've only run the commands seen in this and the previous post, typing x will then find the x = 1 line you entered earlier.  Press enter to choose the matching line.  

The Value of the Previous Line

The Julia REPL stores the value of the previous line in a variable called ans.  

julia> x = 1
1

julia> ans + 1
2
Using ans

Tab-Completion

You can Auto-complete names, file paths, symbols, and emoji.  

  • Try typing prin and pressing tab.  Your input will change to print.  Now press tab again.  Julia will show you that there are more identifiers available that also match with print.
julia> print
print       println      printstyled
print[TAB]
  • Julia can help you type out file/directory paths when you press tab inside of a string.  For example, suppose you have a file at "/Users/<username>/file.csv". Typing "/Users/<username>/f" and then tab will autocomplete the f to file.csv (if it's the only file/directory that begins with f).  If there are multiple matches, press tab again to display them.
  • Try typing \pi and pressing tab.  Your input will change to π ! 🎉
  • Try typing \:smile: and pressing tab.  Your input will change to 😄!  Fun fact: This feature was introduced as an April Fool's Day joke but won traction in the Julia community.  You can even use emoji as variables!  There's no good reason to do this, but it's neat 🙃.
julia> 😄 = 1
1

julia> 🚀 = 2
2

julia> 😄 + 🚀
3
Emoji variables

Skip Printing

Sometime you may not want to display the return value of a line (e.g. the object prints a wall of text).  You can always skip displaying an object by adding ; to the end of a line:

julia> x = "1234"
"1234"

julia> x
"1234"

julia> x;

julia>
Skip Printing with ;

That's It!

This is a high-level overview of the Julia REPL.  It covers the things you'll need to know to get started with the REPL.  If you want to go further into the weeds, check out the official docs here.

Next up is First Steps #3: Plots.

Additional Resources