Getting started

We show for Python two separate "Julia from Python" interfaces, PyJulia and JuliaCall with the second one being the most recent one.

With the classical pyjulia package

PyJulia is a relativelly old method to use Julia code and libraries in Python. It works great but it requires that you already have a Julia working installation on your PC, so we need first to download and install the Julia binaries for our operating system from JuliaLang.org. Be sure that Julia is working by opening the Julia terminal and e.g. typing println("hello world")

Install PyJulia with:

$ python3 -m pip install --user julia   # the name of the package in `pip` is `julia`, not `PyJulia`

For the sake of this tutorial, let's also install in Python a package that contains the dataset that we will use:

$ python3 -m pip install --user sklearn # only for retrieving the dataset in the python way

We can now open a Python terminal and, to obtain an interface to Julia, just run:

>>> import julia
>>> julia.install() # Only once to set-up in julia the julia packages required by PyJulia
>>> jl = julia.Julia(compiled_modules=False)

If we have multiple Julia versions, we can specify the one to use in Python passing julia="/path/to/julia/binary/executable" (e.g. julia = "/home/myUser/lib/julia-1.8.0/bin/julia") to the install() function.

The compiled_module=False in the Julia constructor is a workaround to the common situation when the Python interpreter is statically linked to libpython, but it will slow down the interactive experience, as it will disable Julia packages pre-compilation, and every time we will use a module for the first time, this will need to be compiled first. Other, more efficient but also more complicate, workarounds are given in the package documentation, under the https://pyjulia.readthedocs.io/en/stable/troubleshooting.html[Troubleshooting section].

Let's now add to Julia the BetaML package. We can surely do it from within Julia, but we can also do it while remaining in Python:

>>> jl.eval('using Pkg; Pkg.add("BetaML")') # Only once to install BetaML

While jl.eval('some Julia code') evaluates any arbitrary Julia code (see below), most of the time we can use Julia in a more direct way. Let's start by importing the BetaML Julia package as a submodule of the Python Julia module:

>>> from julia import BetaML
>>> jl.eval('using BetaML')

As you can see, it is no different than importing any other Python module.

For the data, let's load it "Python side":

>>> from sklearn import datasets
>>> iris = datasets.load_iris()
>>> X    = iris.data[:, :4]
>>> y    = iris.target + 1 # Julia arrays start from 1 not 0

Note that X and y are Numpy arrays.

We can now call BetaML functions as we would do for any other Python library functions. In particular, we can pass to the functions (and retrieve) complex data types without worrying too much about the conversion between Python and Julia types, as these are converted automatically:

>>> (Xs,ys) = BetaML.consistent_shuffle([X,y]) # X and y are first converted to julia arrays and then the returned julia arrays are converted back to python Numpy arrays
>>> m       = BetaML.KMeansClusterer(n_classes=3)
>>> yhat    = BetaML.fit_ex(m,Xs) # Python doesn't allow exclamation marks in function names, so we use `fit_ex(⋅)` instead of `fit!(⋅)`
>>> acc     = BetaML.accuracy(ys,yhat,ignorelabels=True)
>>> acc
 0.8933333333333333

Note: If we are using the jl.eval() interface, the objects we use must be already known to julia. To pass objects from Python to Julia, import the julia Main module (the root module in julia) and assign the needed variables, e.g.

>>> X_python = [1,2,3,2,4]
>>> from julia import Main
>>> Main.X_julia = X_python
>>> jl.eval('BetaML.gini(X_julia)')
0.7199999999999999

Another alternative is to "eval" only the function name and pass the (python) objects in the function call:

>>> jl.eval('BetaML.gini')(X_python)
0.7199999999999999

With the newer JuliaCall python package

JuliaCall is a newer way to use Julia in Python that doesn't require separate installation of Julia.

Istall it in Python using pip as well:

$ python3 -m pip install --user juliacall

We can now open a Python terminal and, to obtain an interface to Julia, just run:

>>> from juliacall import Main as jl

If you have julia on PATH, it will use that version, otherwise it will automatically download and install a private version for JuliaCall

If we have multiple Julia versions, we can specify the one to use in Python passing julia="/path/to/julia/binary/executable" (e.g. julia = "/home/myUser/lib/julia-1.8.0/bin/julia") to the install() function.

To add BetaML to the JuliaCall private version we evaluate the julia package manager add function:

>>> jl.seval('using Pkg; Pkg.add("BetaML")')# Only once to install BetaML

As with PyJulia we can evaluate arbitrary Julia code either using jl.seval('some Julia code') and by direct call, but let's first import BetaML:

>>> jl.seval("using BetaML")
>>> bml = jl.BetaML

For the data, we reuse the X and y Numpy arrays we loaded earlier.

We can now call BetaML functions as we would do for any other Python library functions. In particular, we can pass to the functions (and retrieve) complex data types without worrying too much about the conversion between Python and Julia types, as these are converted automatically:

>>> (Xs,ys) = bml.consistent_shuffle([X,y])
>>> m       = bml.KMeansClusterer(n_classes=3)
>>> yhat    = bml.fit_ex(m,Xs)
>>> m._jl_display() # force a "Julian" way of displaying of Julia objects
>>> acc     = bml.accuracy(ys,yhat,ignorelabels=True)
>>> acc
 0.8933333333333333

Note: If we are using the jl.eval() interface, the objects we use must be already known to julia. To pass objects from Python to Julia, we can write a small Julia macro:

>>> X_python = [1,2,3,2,4]
>>> jlstore = jl.seval("(k, v) -> (@eval $(Symbol(k)) = $v; return)")
>>> jlstore("X_julia",X_python)
>>> jl.seval("BetaML.gini(X_julia)")
0.7199999999999999

Another alternative is to "eval" only the function name and pass the (python) objects in the function call:

>>> X_python = [1,2,3,2,4]
>>> jl.seval('BetaML.gini')(X_python)
0.7199999999999999

Conclusions about using BetaML in Python

Using either the direct call or the eval function, wheter in Pyjulia or JuliaCall, we should be able to use all the BetaML functionalities directly from Python. If you run into problems using BetaML from Python, open an issue specifying your set-up.

For R, we show how to access BetaML functionalities using the JuliaCall R package (no relations with the homonymous Python package).

Let's start by installing JuliaCall in R:

> install.packages("JuliaCall")
> library(JuliaCall)
> julia_setup(installJulia = TRUE) # use installJulia = TRUE to let R download and install a private copy of julia, FALSE to use an existing Julia local installation

Note that, differently than PyJulia, the "setup" function needs to be called every time we start a new R section, not just when we install the JuliaCall package. If we don't have julia in the path of our system, or if we have multiple versions and we want to specify the one to work with, we can pass the JULIA_HOME = "/path/to/julia/binary/executable/directory" (e.g. JULIA_HOME = "/home/myUser/lib/julia-1.1.0/bin") parameter to the julia_setup call. Or just let JuliaCall automatically download and install a private copy of julia.

JuliaCall depends for some things (like object conversion between Julia and R) from the Julia RCall package. If we don't already have it installed in Julia, it will try to install it automatically.

As in Python, let's start from the data loaded from R and do some work with them in Julia:

> library(datasets)
> X     <- as.matrix(sapply(iris[,1:4], as.numeric))
> y     <- sapply(iris[,5], as.integer)
> xsize <- dim(X)

Let's install BetaML. As we did in Python, we can install a Julia package from Julia itself or from within R:

> julia_eval('using Pkg; Pkg.add("BetaML")')

We can now "import" the BetaML julia package (in julia a "Package" is basically a module plus some metadata that facilitate its discovery and integration with other packages, like the reuired set) and call its functions with the julia_call("juliaFunction",args) R function:

> julia_eval("using BetaML")
> shuffled <- julia_call("consistent_shuffle",list(X,y))
> Xs       <- matrix(sapply(shuffled[1],as.numeric), nrow=xsize[1])
> ys       <- as.vector(sapply(shuffled[2], as.integer))
> m        <- julia_eval('KMeansClusterer(n_classes=3)')
> yhat     <- julia_call("fit_ex",m,Xs)
> acc      <- julia_call("accuracy",yhat,ys,ignorelabels=TRUE)
> acc
[1] 0.8933333

As alternative, we can embed Julia code directly in R using the julia_eval() function:

kMeansR  <- julia_eval('
  function accFromKmeans(x,k,y)
    m    = KMeansClusterer(n_classes=Int(k))
    yhat = fit!(m,x)
    acc  = accuracy(yhat,y,ignorelabels=true)
    return acc
  end
')

We can then call the above function in R in one of the following three ways:

1. kMeansR(Xs,3,ys)
2. julia_assign("Xs_julia", Xs); julia_assign("ys_julia", ys); julia_eval("accFromKmeans(Xs_julia,3,ys_julia)")
3. julia_call("accFromKmeans",Xs,3,ys)

While other "convenience" functions are provided by the package, using julia_call, or julia_assign followed by julia_eval, should suffix to use BetaML from R. If you run into problems using BetaML from R, open an issue specifying your set-up.