This lecture introduces the GraalVM ecosystem, a runtime able to develop applications not only with JVM-based languages such as Java, Scala, and Kotlin, but also with other programming languages such as JavaScript, Ruby, Python, and R. Additionally, it enables the execution of native code via an LLVM front-end, and WebAssembly programs on the JVM. Recall that the LLVM project is a compiler infrastructure and here an intermediate representation of programs. C and C++ programs can be translated into LLVM, then they can be used by the GraalVM virtual machine. WebAssembly is a technology for cross-browser collaboration on a new, portable, size- and load-time-efficient format suitable for compilation. Again, this is an intermediate representation for "programs" that can be executed by the browser without the need of a JavaScript interpreter, more exactly in cooperation with JavaScript.
Summarizing, GraalVM aims at building applications composed according to multiple programming languages. The following diagram illustrates the architecture of GraalVM as an open ecosystem.
GraalVM consists of core and optional components, and is distributed as an archive. You can download the archives from the project home page. After unarchiving, I get the following tree structure:
cerin@ordinateur-cerin:~/Bureau$ ls Documents Dropbox graalvm-ce-java11-20.0.0 'Old Firefox Data' cerin@ordinateur-cerin:~/Bureau$ cd graalvm-ce-java11-20.0.0/ cerin@ordinateur-cerin:~/Bureau/graalvm-ce-java11-20.0.0$ ls 3rd_party_license_llvm-toolchain.txt include lib tools bin jmods LICENSE.txt conf languages release GRAALVM-README.md legal THIRD_PARTY_LICENSE.txt cerin@ordinateur-cerin:~/Bureau/graalvm-ce-java11-20.0.0$ ls bin gu jcmd jimage jrunscript keytool polyglot jar jconsole jinfo js lli rmic jarsigner jdb jjs jshell node rmid java jdeprscan jlink jstack node_modules rmiregistry javac jdeps jmap jstat npm serialver javadoc jfr jmod jstatd pack200 unpack200 javap jhsdb jps jvisualvm package-lock.json cerin@ordinateur-cerin:~/Bureau/graalvm-ce-java11-20.0.0$
The bin directory contains, among others,
the gu tool which is a package installer that can be used
to install language packs for Python, R, and Ruby, js
that runs a JavaScript console with GraalVM, node which
is a drop-in replacement for Node.js, using GraalVM’s JavaScript
engine and lli which is a high-performance LLVM bitcode
interpreter integrated with GraalVM.
Before starting with examples, check the versions of some installed
tools, and if necessary, modify the PATH environment variable to set
it to the highest priority with the GraalVM bin
directory. Otherwise, "normal" Java, NodeJS... binaries will
disturbing the compiling and running steps.
cerin@ordinateur-cerin:~/Bureau/graalvm-ce-java11-20.0.0/bin$ ./java -version openjdk version "11.0.6" 2020-01-14 OpenJDK Runtime Environment GraalVM CE 20.0.0 (build 11.0.6+9-jvmci-20.0-b02) OpenJDK 64-Bit Server VM GraalVM CE 20.0.0 (build 11.0.6+9-jvmci-20.0-b02, mixed mode, sharing) cerin@ordinateur-cerin:~/Bureau/graalvm-ce-java11-20.0.0/bin$ ./node -v v12.15.0 cerin@ordinateur-cerin:~/Bureau/graalvm-ce-java11-20.0.0/bin$ ./lli --version LLVM (GraalVM CE Native 20.0.0) cerin@ordinateur-cerin:~/Bureau/graalvm-ce-java11-20.0.0/bin$
You are now invited to compile and run many small examples from this page. Please, go ahead! More elaborated examples follow, in order to use the polyglot facilities of GraalVM. Stop before the Docker containers section.
If you want to learn what GraalVM offers to different types of teams, read the Why GraalVM page. Some of the diverse features of GraalVM are disclosed and supported with examples in Top 10 Things To Do With GraalVM article. Or, you can examine different supported languages in action by looking at example applications. If you want to learn about the common tools GraalVM enables for the supported languages, proceed to the tools section of the reference manual. And if you are mostly interested in a specific language, more extensive documentation is available in the reference manual as well.
The incompatibilities between GraalVM and some libraries are the
main issue for developing programs. In the following fragment of code,
I tried to install the treemap package of R, but I
received some message of failures. It is difficult to say if it is a
problem due to an incompatibility of the package with my system or
with any system or a problem related to the integration of the package
with the R/GraalVM software itself. I got:
:~/Bureau/graalvm-ce-java11-20.0.0/graalvm-demos/polyglot-javascript-java-r$ R
R version 3.6.1 (FastR)
Copyright (c) 2013-19, Oracle and/or its affiliates
Copyright (c) 1995-2018, The R Core Team
Copyright (c) 2018 The R Foundation for Statistical Computing
Copyright (c) 2012-4 Purdue University
Copyright (c) 1997-2002, Makoto Matsumoto and Takuji Nishimura
All rights reserved.
FastR is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information.
Type 'q()' to quit R.
> install.packages("treemap")
Warning: dependency ‘data.table’ is not available
also installing the dependencies ‘glue’, ‘stringi’, ‘assertthat’, ‘utf8’, ‘Rcpp’, ‘stringr’, ‘labeling’, ‘munsell’, ‘cli’, ‘fansi’, ‘pillar’, ‘BH’, ‘digest’, ‘gtable’, ‘lazyeval’, ‘mgcv’, ‘plyr’, ‘reshape2’, ‘rlang’, ‘scales’, ‘tibble’, ‘viridisLite’, ‘withr’, ‘magrittr’, ‘pkgconfig’, ‘httpuv’, ‘mime’, ‘jsonlite’, ‘xtable’, ‘htmltools’, ‘R6’, ‘sourcetools’, ‘later’, ‘promises’, ‘crayon’, ‘colorspace’, ‘ggplot2’, ‘gridBase’, ‘igraph’, ‘RColorBrewer’, ‘shiny’
Content type 'application/octet-stream' length 56368 bytes (55 KB)
Content type 'application/octet-stream' length 17916239 bytes (17,1 MB)
Content type 'application/octet-stream' length 11612 bytes (11 KB)
Content type 'application/octet-stream' length 218882 bytes (213 KB)
Content type 'application/octet-stream' length 3635277 bytes (3,5 MB)
Content type 'application/octet-stream' length 135777 bytes (132 KB)
Content type 'application/octet-stream' length 10722 bytes (10 KB)
Content type 'application/octet-stream' length 182653 bytes (178 KB)
Content type 'application/octet-stream' length 2186755 bytes (2,1 MB)
Content type 'application/octet-stream' length 266123 bytes (259 KB)
Content type 'application/octet-stream' length 103972 bytes (101 KB)
Content type 'application/octet-stream' length 12378154 bytes (11,8 MB)
Content type 'application/octet-stream' length 128553 bytes (125 KB)
Content type 'application/octet-stream' length 24157 bytes (23 KB)
Content type 'application/octet-stream' length 80150 bytes (78 KB)
Content type 'application/octet-stream' length 915013 bytes (893 KB)
Content type 'application/octet-stream' length 392451 bytes (383 KB)
Content type 'application/octet-stream' length 36405 bytes (35 KB)
Content type 'application/octet-stream' length 857682 bytes (837 KB)
Content type 'application/octet-stream' length 299262 bytes (292 KB)
Content type 'application/octet-stream' length 729338 bytes (712 KB)
Content type 'application/octet-stream' length 44019 bytes (42 KB)
Content type 'application/octet-stream' length 53578 bytes (52 KB)
Content type 'application/octet-stream' length 200504 bytes (195 KB)
Content type 'application/octet-stream' length 6024 bytes
Content type 'application/octet-stream' length 1675978 bytes (1,6 MB)
Content type 'application/octet-stream' length 12960 bytes (12 KB)
Content type 'application/octet-stream' length 1052728 bytes (1,0 MB)
Content type 'application/octet-stream' length 610140 bytes (595 KB)
Content type 'application/octet-stream' length 45408 bytes (44 KB)
Content type 'application/octet-stream' length 30968 bytes (30 KB)
Content type 'application/octet-stream' length 24155 bytes (23 KB)
Content type 'application/octet-stream' length 40293 bytes (39 KB)
Content type 'application/octet-stream' length 106866 bytes (104 KB)
Content type 'application/octet-stream' length 658694 bytes (643 KB)
Content type 'application/octet-stream' length 2146163 bytes (2,0 MB)
Content type 'application/octet-stream' length 2863109 bytes (2,7 MB)
Content type 'application/octet-stream' length 153373 bytes (149 KB)
Content type 'application/octet-stream' length 2703891 bytes (2,6 MB)
Content type 'application/octet-stream' length 11532 bytes (11 KB)
Content type 'application/octet-stream' length 2987723 bytes (2,8 MB)
Content type 'application/octet-stream' length 121814 bytes (118 KB)
* installing *source* package ‘glue’ ...
** package ‘glue’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
.....
.....
.....
.....
** R
** demo
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (httpuv)
ERROR: dependency ‘mgcv’ is not available for package ‘ggplot2’
* removing ‘/home/agathe/Bureau/graalvm-ce-java11-20.0.0/languages/R/library/ggplot2’
* installing *source* package ‘shiny’ ...
** package ‘shiny’ successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (shiny)
ERROR: dependencies ‘data.table’, ‘ggplot2’, ‘igraph’ are not available for package ‘treemap’
* removing ‘/home/agathe/Bureau/graalvm-ce-java11-20.0.0/languages/R/library/treemap’
The downloaded source packages are in
‘/tmp/RtmpJxnzkm/downloaded_packages’
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
Warning messages:
1: In install.packages("treemap") :
installation of package ‘mgcv’ had non-zero exit status
2: In install.packages("treemap") :
installation of package ‘igraph’ had non-zero exit status
3: In install.packages("treemap") :
installation of package ‘ggplot2’ had non-zero exit status
4: In install.packages("treemap") :
installation of package ‘treemap’ had non-zero exit status
>
Another issue is to guess, for a language, the polyglot interface to make a dialog with yet another language. For R, the manual/documentation says that the interface is as follows:
GraalVM execution of R provides the following interoperability primitives:
eval.polyglot('languageId', 'code') evaluates code in some other
language, the languageId can be, e.g., js.
eval.polyglot(path = '/path/to/file.extension') evaluates code
loaded from a file. The language is recognized from the extension.
export('polyglot-value-name', rObject) exports an R object so that
it can be imported by other languages.
import('exported-polyglot-value-name') imports a polyglot value
exported by some other language.
Then the documentation gives the following example of interoperability:
# use R vector in JavaScript
export('robj', c(1,2,3))
eval.polyglot('js', paste0(
'rvalue = Polyglot.import("robj"); ',
'console.log("JavaScript: " + rvalue.length);'))
# JavaScript: 3
# NULL -- the return value of eval.polyglot
You are invited to play with this source code and restart the application to see what else you can do with the mix of JavaScript, Java, and R. From the server side, you will get:
$ node --polyglot --jvm server.js
Example app listening on port 3000!
Loading required package: lattice
We now put in place a Node (JavaScript) program that fetch an
online weather station to get some information about the weather in
Paris, then illustrates the interoperability of JavaScript with R. For
that purpose, the JavaScript program exports to the R program an
integer, a vector of integer and a vector of strings. Then the R
program do some computation on the objects and export the results to
the JavaScript program. The JavaScript program explicitly imports the
results and aggregate them to the Web page. The JavaScript program also
explores the JSON object returned by the POST method, in search of the
name of the city. This name is also aggregated to the Web page. Note
also that we communicate between the two languages through
the foo.r program which is a R program.
The JavaScript program is:
const express = require('express')
const app = express()
const BigInteger = Java.type('java.math.BigInteger')
app.get('/', function (req, res1) {
var text = '<h1>Weather Station</h1> '
// Using Java standard library classes
text += '<p>Version: ' + BigInteger.valueOf(2).pow(12).toString() + '</p>'
// Using R methods to return arrays
//text += Polyglot.eval('R',
// 'ifelse(1 > 2, "no", paste(1:42, c="|"))') + '<br>'
text += '<p>Arithmetic with R: ' +Polyglot.eval('R',
'3+4') + '</p>'
// use JavaScript objects in R
var fruits = [1, 2, 3, 4, 5, 6, 7];
Polyglot.export('robj', fruits);
var mystr = ["Bonjour ","la ","ville ","de ","Paris"];
Polyglot.export('mystr', mystr);
var myint = 1;
Polyglot.export('intobj', myint);
Polyglot.evalFile("R", "./foo.r");
var lvalue = Polyglot.import('lvalue');
text += '<p>Arithmetic with R (calling the foo.r file): '+lvalue+'</p>'
var myint = Polyglot.import('myint');
text += '<p>Sum of Vector fruits (calling the foo.r file): '+myint+'</p>'
var mystr = Polyglot.import('mystrres');
text += '<p>Concatenate strings (calling the foo.r file): '+mystr+'</p>'
const https = require('https')
const options = {
hostname: 'worldweather.wmo.int',
port: 443,
path: '/fr/json/194_fr.xml',
method: 'GET'
}
const req1 = https.request(options, res => {
console.log(`statusCode: ${res.statusCode}`)
res.on('data', d => {
process.stdout.write(d)
var obj=JSON.parse(d)
res1.send(text+'<h2>City name: '+obj.city.cityName+'</h2>')
//console.log(Polyglot.eval('R', 'library(treemap)')[10]);
//text += Polyglot.eval('python',"3+4");
//import org.graalvm.polyglot.Context;
//try (Context context = Context.create()) {
// Value function = context.eval("python", "lambda x: x + 1");
// assert function.canExecute();
// int x = function.execute(41).asInt();
// assert x == 42;
// text += '<p>'+x.toTring()+'</p>';
// res1.send(text)
//}
})
})
req1.on('error', error => {
console.error(error)
})
req1.end()
})
app.listen(3000, function () {
console.log('Example app listening on port 3000!')
})
The R program is:
print("Start script R");
intvalue <- import("intobj");
print(typeof(intvalue));
lvalue=intvalue+3;
export('lvalue',lvalue);
rvalue <- import("robj");
print(typeof(rvalue));
print(rvalue)
print(rvalue[1:1])
myint <- 0;
for(i in 1:(length(rvalue))) {
myint=myint+rvalue[i:1][1];
print(rvalue[i:1][1]);
}
export('myint',myint);
rsrt <- import("mystr");
#paste(rsrt, collapse = '')
# It works with an explicit iteration
mystrres <- "";
for(i in 1:(length(rsrt))) {
mystrres=paste(mystrres,rsrt[i:1][1]);
print(rsrt[i:1][1]);
}
print(mystrres)
#print(rsrt)
export('mystrres',mystrres);
#export('mystrres',rsrt);
print("End script R")
The JavaScript program is started
with the node --experimental-options --js.nashorn-compat=true
--jvm --polyglot server_weather.js command, and we get the
following result, with on the left part of the Figure the message sent
to the console by the JavaScript and the R
programs, and on the right part of the Figure the browser's
layout.
We are now ready to improve the weather station program written in JavaScript and R. The key idea is to ask the user to enter a city code, then to draw a barchart for the temperatures. The drawing function is implemented with a R program. The JavaScript program explores the JSON object returned by the POST request, parametrized by the city code.
The code architecture is through a POST method to send the HTML
code to the page when the button is pressed. Inside the POST method we
also have a GET method to download the weather information from the
Web site. This GET method is done with the
Javacript fetch() method. This analysis of the obtained
JSON file is done in the makeCsvFile(dico) function. This
function do a synchronous write to the cerin.csv. Then we
call the plotFunction(json.city.cityName) function for
the drawing. The reader should also notice the way we realize the
interoperability between Javacript and R.
The server.js is:
var express = require('express');
var bodyParser = require("body-parser");
var app = express();
var APP_PORT = 8088;
const fetch = require('node-fetch');
const fs = require('fs');
Polyglot.evalFile("ruby", "validator.rb");
var Validator = Polyglot.import('Validator');
Polyglot.evalFile("R", "functionGraph.r");
var plotFunction = Polyglot.import('plotFunction');
app.use(bodyParser.urlencoded({
extended: false
}));
app.use(bodyParser.json());
function makeCsvFile(dico) {
var res = 'Id,Date,Temp' + '\n'
var i = 0;
dico.forEach((value, index, self) => {
//console.log(value, index, self)
res += (1 + i).toString() + ',' + '(min) ' + value.forecastDate + ',' + value.minTemp.toString() + '\n';
res += (2 + i).toString() + ',' + '(max) ' + value.forecastDate + ',' + value.maxTemp.toString() + '\n';
i += 2;
})
console.log(res);
// Asynchronous write
//fs.writeFile('cerin.csv', res, (err) => {
// if(err) {
// return console.log(err);
//}
//console.log("The file was saved!");
//});
// Synchronous write
// Write a string to another file and set the file mode to 0755
try {
fs.writeFileSync('cerin.csv', res, {
mode: 0o755
});
} catch (err) {
// An error occurred
console.error(err);
}
}
app.post('/graph', function(req, res) {
let expr = req.body.expr;
let errorMsg = Validator.validate(expr);
if (errorMsg.length() == 0) {
fetch('https://worldweather.wmo.int/fr/json/' + expr + '_fr.xml')
.then(res => res.json())
.then(json => {
console.log(json);
//console.log(json.city.forecast.forecastDay);
makeCsvFile(json.city.forecast.forecastDay);
res.send(plotFunction(json.city.cityName));
});
} else {
res.send(errorMsg);
}
});
app.use(express.static(__dirname + "/public"));
app.listen(APP_PORT, function() {
console.log("Server listening on port " + APP_PORT);
});
The functionGraph.r is:
library(stats)
library(lattice)
plotFunction <- function(expr) {
svg()
#x <- seq(from = x1, to = x2, by = 0.01)
#y <- eval(parse(text=expr))
#print(xyplot(y ~ x, type="l"))
#data <- read.csv("./barley.csv", header = TRUE, sep = ',')
#print(barchart(gen ~ yield, data=data,
data <- read.csv("./cerin.csv", header = TRUE, sep = ',')
print(barchart(Date ~ Temp, data=data,
main = expr,
xlab = "Temperature",
ylab = "Date",
col = c("chocolate", "green", "grey", "blue", "yellow")))
grDevices:::svg.off()
}
export('plotFunction', plotFunction)
The full package is available by clicking on
this TAR archive. Check also
th public directory for the index.html file.
An example of the execution of the code is given in the Figure below. On the left side you get the messages on the console, on the right side of the Figure you get the bar chart.
Christophe Cérin
christophe.cerin@univ-paris13.fr
May 10, 2020