Author Archives: Simon Danisch

GLVisualize – A modern Graphics Platform for Julia

By: Simon Danisch

Re-posted from: http://randomfantasies.com/2016/11/glvisualize-a-modern-graphics-platform-for-julia/

GLVisualize is a package written in pure Julia, which can create 2D/3D visualizations and GUIs. It’s GPU accelerated and can thus easily handle large datasets and complex 2D/3D scenes. At its core it’s a visualization library heavily using Reactive signals allowing the user to animate and interact with the graphics. I recently wrote a backend for Plots which is a great general purpose plotting library. As a result you can have much of the standard plotting functionality known from libraries such as Matplotlib with GLVisualize + Plots.jl.

Just install it and try it out! I’ve collected some videos, which should give a good overview about what’s possible:

 

Why GLVisualize?

Creating a library like GLVisualize completely from scratch is pretty involved. So I’ve been asking myself why go through all this trouble when there are wonderful alternatives to GLVisualize. Why not concentrate on one existing package?

First of all, some of the established libraries don’t really hold up to what I would call a fast, modern and easy to extend library.

More modern libraries seem to be mostly written for the web. It’s great, but sometimes one really wants a library that works well on the desktop, is not written in JS, and plays well with other desktop APIs (e.g. GPGPU). But I see the beauty of sharing things online, so I do want to break with my `all Julia` approach and write a web backend for GLVisualize as soon as possible!

GUIs are another important aspect to look at. I’ve seen a lot of frameworks using third party GUI libraries which is a very reasonable way to go! But I’ve some use cases in mind which need the graphics rendering to seamlessly integrate with the GUI library. E.g. creating a point cloud where every point is a button. Or draw a line plot which acts as the control for a complex parameter space. I already needed to create quite a few complex 3D UI elements which wouldn’t be possible in most GUI libraries. So at the end of the day putting time into an API for interactive widgets makes a lot of sense.

Another piece of motivation is that in the long run Julia will need some native graphic tools for several reasons. It’s impossible to write one plotting library solving everyone’s needs, so people will start to create their own visualization tools. Having no graphics libraries to build upon will mean throwing these users into the cold water or make them change the language. The livelier Julia’s own (GPU accelerated) Graphics ecosystem, the higher the probability that you can create great, novel visualizations and get insights into your data. GLVisualize can be the starting point, and we as a community have full control over the features and use cases!

Finally, I still haven’t seen another language bringing together speed, scientific computing and easy usage as beautifully as Julia. These are all elements crucial to any visualization library, so GLVisualize is at a very good place with Julia!

State

This is the first version of GLVisualize where I feel comfortable to recommend it to the public! I’ve been working on fixing bugs, backend issues and adding most crucial features for general plotting. As you saw in the video, it now contains a large collection of widgets and many visualization types to chose from. Please note that I haven’t optimized GLVisualize much yet. This means GLVisualize should only get faster from here on! But it also means that there are a couple of bottlenecks. I sketched out a road map to remove them and to improve latency/performance in general. Depending on the use case, you can expect speed improvements by an order of magnitude. One such use case is animations with Plots.jl, which is quite slow right now. For every frame it completely tears down GLVisualize’s state and rebuilds it, which is not what I’ve optimized for yet. So even though that GLVisualize might offer the fastest animation speeds of all backends, animations done with Plots.jl can be slow due to this bad interaction. There are a couple of other things which I collected in one meta issue.

Please report bugs and feedback swiftly 🙂

Best,

Simon

My quest to responsive visualizations with Julia

By: Simon Danisch

Re-posted from: http://randomfantasies.com/2016/09/my-quest-to-responsive-visualizations-with-julia/

Responsive and fun interactivity is notoriously hard! But it’s also the key to less frustration and more patience when working on a project.

There are endless important projects that humanity needs to solve. To become less of a burden to the environment and have a better society we constantly need to advance the state of the art.

Still, people give up on doing this when the initial amount of patience, motivation and curiosity is depleted without getting new incentives.

This actually happened to me with my 3D modeling hobby a few years ago. Not only was the 3D software difficult to learn, it also turned unresponsive fairly quickly, even on a powerful machine.

Just as an example, rendering a 30 second clip in high quality took ~2 weeks with my PC running day and night. This slowness drained my patience and at some point I decided that this is too much hassle for a hobby.

Half a year later, I ran into a similar problem when working on a computer vision pipeline. We simply couldn’t find frameworks which were fast enough for real time processing while maintaining high programming productivity. This time I didn’t give up, but the job could have been much more fun and productive!

At this point I decided that I want to work on software that turns really tough problems like object recognition of 3D models into weekend projects.

I started working on GLVisualize, a library for fast and interactive graphics written in Julia.

Why graphics?

Visualizing something is the first step to understanding and it allows us to explore huge problem spaces that were invisible before.

But current tools seem to be divided into 2D and 3D libraries, high performance libraries, which are kind of a hassle to use, and easily usable libraries which turn into an unresponsive mess quickly.

So with my background, this seemed like a worthy start!

Why Julia?

While there are several reasons for choosing Julia, today I want to show you its impressive speed despite being an easy-to-use dynamic language.

Let me show you the benefit with two toy examples (all graphics including the plots were produced with GLVisualize!).

Consider the following function, the famous Lorenz Attractor:

It takes a point t0 and parameters a to d and returns a new point. Could you imagine what kind of line this will draw, when recursively applying it to the new result and changing parameters?

I guess no one has such a powerful brain, which is why we visualize these kind of things, together with sliders to explore the function:

lorenz

code

These are 100,000 points, and as you can see GLVisualize can deal with that amount easily.

Lets see how long we could have stayed at pleasant interactive speeds in another language.

Lets take for example Python, a language comparable in usability:

 

pyjulo

 

minimal speedup 70.4680453961563
maximal speedup 148.02061279172827
max seconds Julia 0.193422334
max seconds Python 13.630093812942505

As you can see, computation times jump beyond one second at around 10⁵ points for Python, while Julia stays interactive with up to 10⁷ points.

So if you’re interested in a higher resolution in Python you better crank up your patience or call out to C, eliminating all convenience of Python abruptly!

Next example is a 3D mandelbulb visualization:

mandelbulb on the gpu

code

One step through the function takes around 24 seconds with Julia on the CPU, which makes it fairly painful to explore the parameters.

So why is the shown animation still smooth? Well, when choosing Julia, I’ve been betting on the fact that it should be fairly simple

to run Julia code on the GPU. This bet is now starting to become reality.

I’m using CUDAnative and GPUArrays to speed up the calculation of the mandelbulb.

CUDAnative allows you to run Julia code on the GPU while GPUArrays offers a simpler array interface for this task.

Here are the benchmark results:

cujlmandel

minimal speedup 37.05708590791309
maximal speedup 401.59165062897495
max seconds cpu 24.275534426
max seconds cuda 0.128474155

This means that by running Julia on the GPU we can still enjoy a smooth interaction with this function.

But the problem is actually so hard, that I can’t run this interactively at the resolution I would like to.

As you can see, the iso surface visualization still looks very coarse. So even when using state of the art software, you still run into problems that won’t compute under one second.

But with Julia you can at least squeeze out the last bit of performance of your hardware, be it CPU or GPU, while enjoying the comfort of a dynamic high level language!

I’m sure that with Julia and advances in hardware, algorithms and optimizations, we can soon crack even the hardest problem with ease!

 

Benchmarking system:

RAM: 32GB

CPU: Intel® Core™ i7-6700 CPU @ 3.40GHz × 8

GPU: GeForce GTX 950

 

Why I’m betting on Vulkan and Julia

By: Simon Danisch

Re-posted from: http://randomfantasies.com/2016/02/why-im-betting-on-vulkan-and-julia/

Vulkan – The successor of OpenGL and OpenCL
Julia – A fast high-level language for Scientific Computing

There are currently different problem areas when dealing with visualizations and high performance computing, which make it hard to create a pleasant and fast workflow.

  • There is no seamless way from data to high performance visualization libraries. Even if your data processing runs on the GPU, it is common practice to serialize it to the hard drive and load it into an external visualization program, which again uploads the data to the GPU.
  • You need to know different low-level languages and API’s to get full control over the data processing/simulation and visualization pipeline -> if you’re not an expert you won’t get far
  • It is often hard to set up a speedy pipeline with different libraries that all need to work together (The worst setup took me one week before I could get started)
  • There are many problems in scientific computing which need custom solutions. If your technology stack is fragmented and uses a lot of different languages, it is really hard to create a custom solution without starting from scratch.
  • Once your research is done, it might only run on a couple of platforms.

Vulkan and Julia can form the next framework for high performance scientific computing with a new level of native support for interactive 2D/3D visualizations across different platforms.

 

Short introduction to Vulkan:
Vulkan is the newest industry standard for general computing and graphics on the GPU (released by the Khronos Group on the 02/16/16). It can be thought as the successor of Khronos’s OpenGL (Graphics) and OpenCL (GPGPU), but it is designed almost from scratch.

The design offers a new kind of flexibility and performance when programming for the GPU.

Some of the biggest changes are, that OpenGL and OpenCL kernels now share the same intermediate representation (SPIR-V) and execution model. This intermediate binary representation is very similar to that of LLVM-IR and there are bi-directional translators for SPIR-V and LLVM-IR.
SPIR-V is easy to target from different languages finally opening up the world of GPU accelerated graphics and computing to more languages.
Another improvement is the elimination of most driver overhead and better support for multithreading which makes it easier to utilize the CPU while doing work on the GPU.
And last of all, Vulkan is expected to run on different hardware setups, from multi node clusters with thousands of GPU’s down to a smartphone, all while squeezing out the maximum performance.

Supported Platforms

(where is apple?)

All this comes at the cost of a complicated runtime, where you have to manage the memory allocations and schedule programs via command buffers with a fairly complex C API.

 
Short introduction to Julia:

Julia is a young programming language promising to be the greediest of all!
It is easy to use and offers the kind of performance you would expect from C, while offering freedom and usability known from dynamic languages like Python.
The main language features are multiple dispatch and a rather functional style of programming. The two together make it a good fit for mathematical code and parallel programming. As a loosely typed language without much boilerplate, a huge standard library and a lot of scientific libraries, it is very easy to work interactively on powerful scripts. These scripts can go from data analysis to generating HTML for a website.
Still offering high speed in this situation is made possible by a clever runtime type inference and just in time compilation of the then (mostly) fully typed multimethods.
The just in time compilation is done via LLVM, which means Julia already targets LLVM-IR as an intermediate representation. Since Julia’s memory layout and binary format is similar to C’s, Julia has a mostly overhead less C-call API. Finally, Julia runs on most platforms supported by LLVM, and is designed to easily run on large clusters.

 

You might already see where this is going.

We have a language that targets number crunching and scientific computing with a huge demand for processing power, and an API that allows to squeeze out the last bit of performance from heterogeneous hardware.

We have a hard to control Vulkan C-API and a scripting language which excels at calling C code.
We have a language that compiles to LLVM-IR and converters that convert LLVM-IR to SPIR-V, allowing the language to run natively on the GPU.

Everything is there for a great support of high performance computing and visualizations from within one easy to use language.

All these goodies are very close, but there is much work to be done. We still need a wrapper for the Vulkan API and Julia’s LLVM-IR needs some tweaking to conform to the SPIR-V standards, which is a pretty involved task.

To slowly bring Julia and Vulkan together, a two stepped approach is adviced:

  1. Use Julia as a scripting language for Vulkan to have a nice interface to the memory and command buffers, while still relying on C/C++ GLSL kernels to produce SPIR-V executables.
  2. make Julia compile to SPIR-V itself, completely elimating the need for other languages.

 

In the first stage, Julia already offers great advantages.
From my experience with the julian OpenCL and OpenGL wrappers, decorating a low level API like Vulkan with a higher level interface in Julia offers a great improvement in productivity and safety while losing almost no performance.
Then in the final stage, we could even choose freely what to run on the CPU and what to run on the GPU, since the Julia code can run on both.
You could create software that works on the desktop and on tablets and mobile phones without any code duplication.

We could simulate the gravity of large galaxies on a cluster of GPU’s while immediately visualizing the result, all while staying in a nice high level programming language and having absolute control over all libraries involved.
This could be done in an interactive way, refining functions and algorithms while directly getting feedback.

We could finally build the interactive tools that stream big data in parallel, while seamlessly viewing and editing the data.
If you have a powerful GPU, you could even dive into virtual reality to get a good look at the data. If you know a bit about VR, you also know that it is very sensitive to latency, making it a must to rely on tools with the highest possible performance.

I think the motivation is clear, now we only need to implement the missing bits!

Work is done on Vulkan.jl already and I will do my best to add a Vulkan backend to GLVisualize.