Tag Archives: Programming

Capturing output in Julia

By: Terence Copestake

Re-posted from: http://thenewphalls.wordpress.com/2014/03/21/capturing-output-in-julia/

In a previous blog post I pondered whether it may be possible to redirect STDERR to an IOBuffer so that the output can be handled in a controlled way e.g. written to a log file. It turns out that it’s not quite that simple, but capturing output can still be achieved easily with a few more lines of code.

The noteworthy functions here are the redirect_std* family of functions. These functions redirect their respective handles to a new pipe and return a read and write handle for said pipe.

Capturing output

Below is an example of capturing output to STDOUT.

(outRead, outWrite) = redirect_stdout()

print("Test")
print("ing")

close(outWrite)

data = readavailable(outRead)

close(outRead)

At the end of execution, the variable “data” will contain the string ‘Testing’.

(As seen on line 6, it’s advisable to close the write handle before trying to read from the pipe, as this will ensure that any buffered writes are flushed and available for reading.)

If you need to write to the original output stream after redirecting and capturing data, you’ll first need to create a copy of the original handle and restore it later. For example:

originalSTDOUT = STDOUT

(outRead, outWrite) = redirect_stdout()

print("Test")
print("ing")

close(outWrite)

data = readavailable(outRead)

close(outRead)

redirect_stdout(originalSTDOUT)

print(data)

Line 1 is where the original handle is copied; line 14 is where it’s restored. The print on line 16 will therefore write to the original STDOUT (e.g. console window, browser, etc) instead of the outWrite pipe.

Capturing errors

This technique is particularly useful when applied to STDERR, as it can be used to write errors to a log file. The process is the same, but instead using functions applicable to STDERR. Below is a slightly different example:

(errorRead, errorWrite) = redirect_stderr()

atexit(function ()
    close(errorWrite)

    errors = readavailable(errorRead)

    close(errorRead)

    logfile = open("errors.log", "a")
    write(logfile, errors)
    close(logfile)
end)

atexit registers a function to be called when the program execution ends for whatever reason (fatal error, user called the quit function, etc). The code between lines 4 and 8 is similar to the STDOUT example – and code as been added at lines 10 to 12 to log the captured error(s) to a file.

Understanding object-oriented programming in Julia – Inheritance (part 2)

By: Terence Copestake

Re-posted from: http://thenewphalls.wordpress.com/2014/03/06/understanding-object-oriented-programming-in-julia-inheritance-part-2/

In part 1, I explored the concept of objects from the perspective of the Julia language. In this article, I will be looking into Julia’s implementation of object inheritance i.e. the inheritance of behaviour and properties.

Classes / types

Read more at http://julia.readthedocs.org/en/latest/manual/types/

As was covered in part 1, the closest thing to a “class” or “object” in Julia is a type – which may contain fields (properties), but there is no support for including methods.

There are a number of different types, but the two most noteworthy for this article are abstract and concrete types.

Concrete types have a name, a list of fields and a constructor. Concrete types can be instantiated and used to store and manipulate information. These types are closest in behaviour to the traditional “classes” of object-oriented programming.

Abstract types have only a name and no other properties or behaviour. The purpose of abstract types seems to be as a way to indicate relationships between concrete types and can be thought of as being most similar to OOP interfaces than abstract classes.

(There also exist immutable types, which are essentially constants)

Inheritance

In Julia, all types are only able to “inherit” (i.e. be a subtype of) abstract types. This means that you could for example have an abstract type Letter which is inherited by concrete types A and B, but this would serve no other purpose than indicating that A and B are letters.

Having read the documentation and various contributor comments on the help forums, it appears that Julia was never intended to and will never support inheritance of either data or behaviour e.g. concrete types inheriting or extending other concrete types. The consensus appears to be that delegation is the preferred solution.

I began to look for a way to emulate the behaviour of concrete inheritance. As is often the case, the solution is an abomination. Still, for academic purposes…

Forcing inheritance of properties and behaviour

My plan was to find a way to merge two types together. I decided to create a method that, when two “Inheritable” objects are added together (i.e. “a + b”), will produce a new type containing the fields and data from both objects.

As of Julia 0.3, there’s no way to add field definitions to a type after the type has been declared. This meant I needed to create a new type declaration at runtime – which requires the use of an eval.

Sidenote: As if that’s not enough of a performance hit, the code here must first parse a text string containing the code for the type definition, which creates an Expr object that eval can then execute; it would therefore be possible to optimise the below code by directly manipulating an Expr’s args array, rather than relying on the parse function – but this is beyond my current Julia skillZ.

Below is a big chunk of code, which I’ll elaborate on:

abstract Inheritable

+(a::Inheritable, b::Inheritable) = (function (a::Inheritable, b::Inheritable)
    properties = Dict{String, Any}()

    for property in names(a)
        propertyName = string(property)

        if (!haskey(properties, propertyName))
            properties[propertyName] = (propertyName, string(fieldtype(a, property)))
        else
            (fieldName, fieldType) = properties[propertyName]
            properties[propertyName] = (fieldName, "Any")
        end
    end

    for property in names(b)
        propertyName = string(property)

        if (!haskey(properties, propertyName))
            properties[propertyName] = (propertyName, string(fieldtype(b, property)))
        else
            (fieldName, fieldType) = properties[propertyName]
            properties[propertyName] = (fieldName, "Any")
        end
    end

    fieldCode = ""

    for property in values(properties)
        (fieldName, fieldType) = property
        fieldCode = fieldCode * fieldName * "::" * fieldType * "\n"
    end

    randomTypeName = "An" * randstring(16);

    typeCode = "type " * randomTypeName * " <: Inheritable " * fieldCode * " function " * randomTypeName * "() return new () end end"

    eval(parse(typeCode))

    randomTypeName = symbol(randomTypeName)

    c = @eval begin
        $randomTypeName()
    end

    for property in names(a)
        try
            c.(property) = a.(property)
        catch
            c
        end
    end

    for property in names(b)
        try
            c.(property) = b.(property)
        catch
            c
        end
    end

    return c
end)(a, b)

This code contains an abstract type (“Inheritable”) and a method to handle the addition of one Inheritable object to another – the result being both objects merged together to form another Inheritable object.

The first two loops go through each object and create a record of their properties and the properties’ types. You’ll notice that if both objects have a property with the same name, the type is changed to Any to eliminate any conflicts between types e.g. one object accepting an integer and another accepting a float. In practice this is likely to cause a lot of headaches due to Julia’s multiple dispatch, but just roll with it.

Immediately after that, the properties are written into a string of Julia code as field definitions. A random name is generated for the new pseudotype, with care taken to ensure that the first letter is alphabetic (numbers will cause a parse error). The code for the type is then compiled together into a final string, parsed, evaluated and executed.

A second eval calls the pseudotype’s constructor, creating an incomplete instance of the new type.

Two more loops then iterate over the objects being merged together, assigning their current values to the properties on the new type. The resulting object is then returned.

Below is an example of two types making use of this emulated inheritance behaviour:

type A <: Inheritable
    whoAmI::Function
    uniqueFunctionA::Function

    function A()
        instance = new()

        instance.whoAmI = function ()
            println("I am object A")
        end

        instance.uniqueFunctionA = function ()
            println("Function unique to A")
        end

        return instance
    end
end

type B <: Inheritable
    whoAmI::Function
    uniqueFunctionB::Function

    function B()
        instance = new()

        instance.whoAmI = function ()
            println("I am object B")
        end

        instance.uniqueFunctionB = function ()
            println("Function unique to B")
        end

        return A() + instance
    end
end

Type A is a standard type declaration, using the same emulated method bundling from part 1. Type B extends type A in the constructor, by returning an instance of the merged pseudotype instead of an instance of type B. That code is:

return A() + instance

The below code is an example of using these two objects:

a = A()
a.whoAmI()

b = B()
b.whoAmI()

b.uniqueFunctionA()
b.uniqueFunctionB()

Which produces the output:

I am object A
I am object B
Function unique to A
Function unique to B

Issues remain unsolved

Even the above hack-around doesn’t solve the problem of visibility. The code carries an increased performance penalty to run, is less intuitive and not particularly elegant. At this stage, I don’t think Julia is suited to the same approaches and design patterns used in languages like PHP and C#. Is that a good or bad thing? In part 3 I’ll try doing things the “Julia way” and report back with any benefits or limitations I encounter.

Understanding object-oriented programming in Julia – Objects (part 1)

By: Terence Copestake

Re-posted from: http://thenewphalls.wordpress.com/2014/02/19/understanding-object-oriented-programming-in-julia-part-1/

Disclaimer #1: I’m new to Julia – so it’s possible that I may have missed or misunderstood aspects of the language.

Disclaimer #2: Julia is not necessarily intended to be an object-oriented language or even intended for use in professional/commercial software development.

Disclaimer #3: The OOP terminology used here is for familiarity with PHP and such, but may not be appropriate for Julia.

Coming from working with OOP in languages like PHP, Java and C++, it took me a while to adjust to the Julia approach to the concept of objects. Take the following paragraph from the documentation (relevant parts highlighted in bold):

In mainstream object oriented languages, such as C++, Java, Python and Ruby, composite types also have named functions associated with them, and the combination is called an “object”. In purer object-oriented languages, such as Python and Ruby, all values are objects whether they are composites or not. In less pure object oriented languages, including C++ and Java, some values, such as integers and floating-point values, are not objects, while instances of user-defined composite types are true objects with associated methods. In Julia, all values are objects, but functions are not bundled with the objects they operate on. This is necessary since Julia chooses which method of a function to use by multiple dispatch, meaning that the types of all of a function’s arguments are considered when selecting a method, rather than just the first one (see Methods for more information on methods and dispatch). Thus, it would be inappropriate for functions to “belong” to only their first argument. Organizing methods into function objects rather than having named bags of methods “inside” each object ends up being a highly beneficial aspect of the language design.

What this means is that objects have properties, but no methods. However, it’s possible to define a single function within the object’s type definition that will act as the object’s constructor. If you’re familiar with C, objects in Julia can be thought of as C structs with the addition of a constructor.

Having read through the docs and existing Julia source code, it seems that the way to encapsulate functionality within a Julia object is to assign functions manually – ideally (but not necessarily) from within the constructor – to properties within the object. The same applies to defining default values for properties. As an example, below is the current version of the Response class from my experimental web framework.

type Response
    data::String
    headers::Dict

    addContent::Function
    setHeader::Function
    getHeaders::Function
    getContents::Function
    getResponse::Function

    function Response()
        this = new ()

        this.data = ""

        this.headers = Dict{String, String}()

        this.addContent = function (append::String)
            this.data = this.data * append
        end

        this.setHeader = function (header::String, value::String)
            this.headers[header] = value
        end

        this.getHeaders = function ()
            headers = ""

            for (header, value) in this.headers
                headers = headers * header

                if length(value) > 0
                    headers = headers * ": " * value
                end

                headers = headers * "\n"
            end

            return headers
        end

        this.getContents = function ()
            return this.data
        end

        this.getResponse = function ()
            response = ""

            response = this.getHeaders() * "\n" * this.getContents()

            return response
        end

        return this
    end
end

To expand on the noteworthy parts:

data::String
headers::Dict

addContent::Function
setHeader::Function
getHeaders::Function
getContents::Function
getResponse::Function

These are definitions for the properties of the object. The top two are intended for use as typical data properties, whereas the Function properties are intended to contain the methods for the object, which will be assigned in the constructor. This distinction is arbitrary; Julia itself makes no distinction between these two types.

function Response()
    ...
end

This is the constructor definition. There are a number of different options for this within Julia’s syntax, but the style used here is the most familiar to me.

Constructors in Julia differ from the PHP-esque style of constructors, which exist within an instance of an object and are executed immediately after the instance has been created. In Julia, constructors are responsible for the actual creation of an instance, as you’ll see below.

this = new ()

This code is where the instance of the object is created and assigned to a “this” variable. My use of “this” here is again only for the sake of familiarity; any valid variable name would work the same.

this.data = ""

this.headers = Dict{String, String}()

Here, default values are assigned to the new instance’s properties.

this.addContent = function (append::String)
    ...
end

This code is an example of assigning a module to the object. Essentially, it’s the same as assigning a value to a property; it’s just that in this case, the value is a callable function.

return this

When the instance has been constructed, it’s returned to the caller.

Instances of the object are then created in user code as below:

response = Response()

Methods can be called as:

response.addContent("Something to print")

Bells and whistles

As mentioned before, access modifiers appear to be missing from Julia. Therefore, everything is public – and even methods can be overwritten if new functions are assigned to their respective properties. Also, as methods don’t “belong” to the object, it may not be possible to reproduce the behaviour of inheritance as in other languages (getting my head around Julia’s inheritance will be part 2).

It may be too early to say, but at this stage it appears that Julia may be lacking the necessary features for effective use of traditional OOP development patterns – though that doesn’t necessarily mean that the language itself or its implementation of objects is broken.

Update

Part 2 is now online here.