This manual is mostly examples-based. It starts with two simple step-by-step examples (on dealing with static and dynamic libraries), which should give enough information for basic JEL usage (but don't forget to read the rest of this manual to learn how to get the top performance from JEL). The javadoc generated documentation contains the additional information.
There are following sections in the manual:
This manual is by no means comprehensive, but I feel it to be enough to start using JEL. If I'm wrong, please send me suggestions on what you would like to see in the next version of this manual.
The main design goal was to create light weight expression compiler generating extremely fast code. The main emphasis is the code execution time and not the compilation time (it is nevertheless small). The other goal was to make JEL language to be very close to Java language with direct access to all built-in Java data types and functions.
In this section a simple example of a program using JEL is given, and explained with references to more detailed sections of this manual. The example program evaluates the expression given on its command line (similar program exists in the distribution under the name ./samples/Calculator.java), let's follow it step by step.
public static void main(String[] args) {
// Assemble the expression
StringBuffer expr_sb=new StringBuffer();
for(int i=0;i<args.length;i++) {
expr_sb.append(args[i]);
expr_sb.append(' ');
};
String expr=expr_sb.toString();
This first part of the program is not related to JEL. It's purpose is to assemble the expression, possibly, containing spaces into the single line. This has to be done, because shells tend to tokenize parameters but we don't need it here.
// Set up the library
Class[] staticLib=new Class[1];
try {
staticLib[0]=Class.forName("java.lang.Math");
} catch(ClassNotFoundException e) {
// Can't be ;)) ...... in java ... ;)
};
Library lib=new Library(staticLib,null,null,null,null);
try {
lib.markStateDependent("random",null);
} catch (NoSuchMethodException e) {
// Can't be also
};
This piece of code establishes the namespace for use in JEL compiled expressions. The gnu.jel.Library object maintains this namespace.
There can be two types of names in the Library : static(constant) and virtual (dynamic).
Methods and variables of the first class are assumed (by default) to be dependent only on their arguments i.e. not to save any information from call to call (they are "stateless")... Examples are mathematical functions like sin, cos, log, constants E, PI in java.lang.Math. For such methods (fields) it does not matter how many times (when) they will be called (their value will be taken) the result will always be the same provided arguments (if they are present) are the same. Stateless methods will be evaluated by JEL at compile time if their arguments are constants (known at compile time). To define set of static functions(fields) it is needed to pass the array of Class objects, defining those functions, as the first parameter of the library constructor (see example above). Note ONLY STATIC functions of the Classes, passed in the first argument of the gnu.jel.Library constructor will be defined in the namespace. By default all static functions are considered "stateless" by JEL.
However, some static functions still save their state (in static variables) in between calls. Thus they return different results, depending on when (how many times) they are is called even if their arguments are the same. If such function is evaluated at compile time, we have troubles, because it will be evaluated only once during expression lifetime and it's state dependence will be lost. Typical example of the static function, having a state is java.lang.Math.random(). JEL has special mechanism, provided by gnu.jel.Library class to mark static functions as state dependent. (see the above example to find out how it was done for the java.lang.Math.random())
The virtual functions, which are explicitly state dependent, will be discussed later in this document. The example we currently consider does not use them. However, virtual functions are, actually, most important to JEL because expression, containing all stateless functions, is a constant, it will be completely evaluated at compile time, there is absolutely no sense to evaluate such expression repeatedly (this is what JEL was designed for). Still we shall continue with this simple example as the following code is mostly independent of whether we use virtual functions or not...
// Compile
CompiledExpression expr_c=null;
try {
expr_c=Evaluator.compile(expr,lib);
} catch (CompilationException ce) {
System.err.print("–––COMPILATION ERROR :");
System.err.println(ce.getMessage());
System.err.print(" ");
System.err.println(expr);
int column=ce.getColumn(); // Column, where error was found
for(int i=0;i<column+23-1;i++) System.err.print(' ');
System.err.println('^');
};
This chunk of code is for the expression compilation. The crucial line is the call to Evaluator.compile(...), it is the point, where expression gets transformed into Java bytecode, loaded into the Java Virtual Machine using JEL ClassLoader and returned to caller as an instance of the subclass of gnu.jel.CompiledExpression. Typical user of JEL is not required to know what magic is going on inside of Evaluator.compile(...). Other code in this chunk is for the error reporting and will be discussed in the specialized section Error detection and reporting below.
if (expr_c !=null) {
// Evaluate (Can do it now any number of times FAST !!!)
Number result=null;
try {
result=(Number)expr_c.evaluate(null);
} catch (Throwable e) {
System.err.println("Exception emerged from JEL compiled"+
" code (IT'S OK) :");
System.err.print(e);
};
This code does the evaluation of the expression. It is done by calling the evaluate method of the JEL compiled class, it is defined abstract in gnu.jel.CompiledExpression but is redefined in the class compiled by JEL. The argument of this method is discussed in the section on virtual functions below. If only static functions are present in the library it is safe to pass the null pointer as the argument to evaluate.
Result of the evaluate method is always an object. JEL converts primitive numeric types into instances of corresponding Java reflection classes (read the section Making things faster to find out how to avoid this conversion). For example, a value of primitive type long will be returned as an instance of java.lang.Long class (int maps to java.lang.Integer, float to java.lang.Float, etc.). If result is an arbitrary Java object it is returned as the reference to that object.
The try ... catch clause around the call to evaluate() will be enforced by the Java compiler. It is required as errors can appear during evaluation. The general rule is : syntax, types incompatibility and function resolution errors will be reported at compile time ( as thrown instance of gnu.jel.CompilationException), while the errors in the values of numbers will be reported at the execution time. For example expression "1/0" will generate no error at compile time (nevertheless it is the constant expression and its evaluation is attempted), but at the time of calling execute you will get a java.lang.ArithmeticError (division by zero) as it should be.
// Print result
if (result==null)
System.out.println("void");
else
System.out.println(result.toString());
};
};
This last piece of code will print the result. And is concluding our brief tour of the JEL usage.
The namespace of JEL expressions is represented by
gnu.jel.Library class. Its constructor
Library(Class[] staticLib, Class[] dynamicLib,
Class[] dotClasses, DVMap resolver,
Hashtable cnmap)
has five arguments. Their purposes are
following:
The details on usage of each of these arguments are given in a separate sections below.
The working example using all current functionality of JEL namespace is given in the examples/YourTestBed/ directory in the distribution. You'll want to check it after reading this section.
The array of references to classes (java.lang.Class) whose public static methods and fields are to be exported should be passed as the first argument of the library constructor (staticLib). The public static fields and methods of all these classes are merged together into the JEL namespace. The non-public or non-static members of staticLib classes are ignored.
Methods overloading is supported and works also across classes (because the JEL namespace works similarly to the namespace defined in a single Java class). For example, if a class C1 contains the method public static C1.func(int) and a class C2 contains the method public static C2.func(double) and both these classes are passed as elements of the staticLib array. Then, the JEL expression "func(1)" calls C1.func(int) and the expression "func(1.0)" calls C2.func(double). It also means that methods and fields of all classes supplied to the Library are subject to the same constraints as members of a single Java class.
Moreover, because JEL allows to call methods with no arguments omitting the empty brackets (that is "func()" and "func" are equivalent) there should be no fields and methods with no arguments having the same names in all classes presented to the Library constructor.
To check whether the set of classes you gave to the library constructor satisfies all required constraints run your program against the debug version of JEL library (jel_g.jar). Then, potential problems will be reported to you on the standard output.
The second argument of the library constructor (dynamicLib) works similarly to the first one. Except that only public virtual members are taken from the listed classes. These members are merged into the namespace created from classes from the staticLib. The rules for methods overloading are the same as for classes listed in the first argument of library constructor. Also, the overloading is working across the classes listed in both first and second arguments of the Library constructor.
The crucial difference in the handling of classes listed in the dynamicLib and the staticLib comes from the fact that virtual members of dynamicLib require this reference to the instance of the object of their defining class be supplied at run-time. Thus, if C1 contains the virtual method public func(double x) its invocation actually requires two arguments, one is x and the other is the reference to the instance of class C1.
The references to the instances of classes of the dynamicLib array are supplied at the execution time to the argument of the evaluate(Object[] context) method of gnu.jel.CompiledExpression. The elements of the context array should be instances of classes listed in dynamicLib array at compile time and there should be one-to-one correspondence between them. For example, if dynamicLib[0]=com.mycompany.MyClass.class), the corresponding entry in the context array,context[0], must be a reference to the instance of com.mycompany.MyClass.
Formally, each object, in the context[i] should be possible to cast to the class in the dynamicLib[i] array for any i, otherwise ClassCastException will be thrown from evaluate.
Let's walk through the example, which calculates function of the single variable many times and uses virtual method calls. This example will consist of two classes : a user written class (providing access to the variable) and the main class compiling and evaluating expressions. First start with the variable provider :
public class VariableProvider {
public double xvar;
public double x() {return xvar;};
};
This class is trivial, it just defines the function, returning the value of the variable x.
In the main class (see the first JEL example for headers) the code, constructing the library will be replaced with :
// Set up library
Class[] staticLib=new Class[1];
try {
staticLib[0]=Class.forName("java.lang.Math");
} catch(ClassNotFoundException e) {
// Can't be ;)) ...... in java ... ;)
};
Class[] dynamicLib=new Class[1];
VariableProvider variables=new VariableProvider();
Object[] context=new Object[1];
context[0]=variables;
dynamicLib[0]=variables.getClass();
Library lib=new Library(staticLib,dynamicLib,null,null,null);
try {
lib.markStateDependent("random",null);
} catch (NoSuchMethodException e) {
// Can't be also
};
Compared to the static example the additional code creates the VariableProvider and assigns its reference to an element of context array (to be passed to the evaluate method of the compiled expression). Also, now the dynamicLib array as not null and contains the reference to the VariableProvider class.
The code for compilation is exactly the same as in the example for static functions, except we have additional function x and the variable xvar defined for use inside the compiled expressions. JEL has the special notation for the functions, having no arguments, namely, brackets in "x()" can be omitted to be "x". This allows to compile now ( with the above defined library) the expressions like "sin(x)", "exp(x*x)", "pow(sin(x),2)+pow(cos(x),2)"...
The code for evaluation of an expression having virtual functions is replaced with :
if (expr_c !=null) {
try {
for(int i=0;i<100;i++) {
variables.xvar=i; // <- Value of the variable
System.out.println(expr_c.evaluate(context));
//^^^^^^^^^^^^^^^ evaluating 100 times
};
} catch (Throwable e) {
System.err.println("Exception emerged from JEL compiled"+
" code (IT'S OK) :");
System.err.print(e);
};
};
Note the two major differences: 1. we have explicitly assigned the value to the variable; 2. the array of object references ( consisting of one element in this example) is passed to the evaluate method. This piece of code will evaluate expressions for x=0..99 with step 1.
This concludes our dynamic library example. Try to modify the ./Calculator.java sample yourself to allow compilation of virtual functions as described above.
The third argument of gnu.jel.Library constructor enumerates classes which are available for dot operator within the expression. If this parameter is null JEL would not allow to use the dot operator at all. If it is an array of the length zero (e.g. new Class[0]) JEL will open access to public methods of ALL objects encountered in the expression. From the security point of view allowing access to all objects can be dangerous, that is why there is a third case of non-zero length array explicitly enumerating classes allowing the dot operator on them.
Once the dot operator is allowed on a class,it is possible to call all its public methods using the syntax ".method(arg1,arg2,...)" in any context that class appears in an expression.
All methods of exporting names into JEL namespace described up to this point relied on the Java class files for actual description of methods names and parameters. However, sometimes it is required to add a new variable to JEL namespace at run-time.
One of the solutions would be to generate a new class file (e.g. using JEL) and supply it as a first or second argument of the library constructor. Unfortunately this can be quite cumbersome and time consuming.
The other solution can be to define a family of methods in JEL namespace
YYY getXXXProperty(String name)
for each possible variable types, where YYY is the class representing the property type and XXX is the name of the type. Then, supposing we have methods
double getDoubleProperty(String name); // YYY=double XXX=Double String getStringProperty(String name); // YYY=java.lang.String XXX=String
in the JEL namespace (either static or dynamic),
the variables with arbitrary names can be entered into expression
using the syntax
getStringProperty("x") +
(getDoubleProperty("y")+1.0)
This way has two drawbacks: 1) user has to remember the type of the variable (to call the appropriate getXXX() method); 2) a lot to type.
Since the version 0.9.3 JEL provides the way to solve both these problems. To do that the fourth argument (resolver) of the library constructor is used. This argument supplies the reference to the subclass of gnu.jel.DVMap, and is used by JEL to resolve the dynamic variable names. The gnu.jel.DVMap has an abstract method
public String getTypeName(String name)
which returns XXX (see above) for a given variable name, or null if no such variable is defined. Note that for resolver to work the family of methods
YYY getXXXProperty(String name)
must still be present in JEL namespace (e.g. as a members of one of dynamicLib[] classes).
Then, supposing
resolver.getTypeName("x")=="String" &&
resolver.getTypeName("y")=="Double"
the expression "x+(y+1.0)" will be automatically converted
by JEL into
getStringProperty("x")+(getDoubleProperty("y")+1.0)
and compiled. Thus, user does not have to remember the variable types, typing is reduced and the existence of variables can be checked at the compile time.
JEL also supports a hierarchical structure of variables. This means the dot (".") symbol can be present in the dynamic variable names. For example if
resolver.getTypeName("x")!=null &&
resolver.getTypeName("x.f1")=="String" &&
resolver.getTypeName("x.f2")=="Double"
the expression "x.f1+(x.f2+1.0)" will be compiled by JEL as
getStringProperty("x.f1")+(getDoubleProperty("x.f2")+1.0)
and (combined with dot operator) the expression "x.f1.length()" will result in the length of the string getString("x1.f1").
Notice in the last example that if one wants to have defined the dynamic variable "x.y" the variable "x" must also be the dynamic variable (resolver.getTypeName("x")!=null).
If there is conflict between the dynamic variable name and other name in JEL namespace the dynamic variable has a priority.
Since JEL 0.9.9 it is possible to translate the names of dynamic variables from strings into the constants of Java primitive types. This is done using non-identity DVMap.translate method. The translation helps to improve performance in some cases.
Consider the following example. Suppose the underlying storage for dynamic variables is an array (or Vector), so that the value of the variable can be obtained by an integer index into that array (like numbered columns in a spreadsheet). Next, assume you still want to refer to the variables by names (e.g. you allowed user to assign names to the columns). Now, if the first column is named "x" and is of Double type, an expression "x", using dynamic variables interface with identity translation will be compiled into getDoubleProperty("x"). It means the translation of the string "x" into the column number 1 will have to be performed at run-time each time the expression is evaluated. Considering that Java strings are immutable, this may incur a substantial performance penalty.
The performance can be improved if the translate method of DVMap is overridden by the following:
public Object translate(String name) {
if (name.equals("x")) return new Integer(1);
return name;
};
This is already a non-identity translation. With such DVMap the expression "x" will be compiled by JEL into getDoubleProperty(1), note that it is getDoubleProperty(int) method, which is called. This way the mapping of the variable name into the variable index is performed at compile-time, while at run-time the index is readily available. By defining the appropriate translations the dynamic variable lookup can be split in a user-controlled way between the expression compilation and execution stages to achieve the best performance. The translate method is allowed to return only instances of Java reflection classes wrapping the primitive types (java.lang.Integer, java.lang.Double, etc), or strings (otherwise an exception will emerge at compile-time). This is because only these types of objects can be stored in the Java class files directly. Also, it is responsibility of the caller to ensure that JEL namespace contains getXXXProperty methods with all the necessary argument types, corresponding to the translations defined in DVMap. For identity translations only getXXXProperty methods accepting strings are necessary.
The cnmap argument of gnu.jel.Library constructor, allows to enable the non-primitive type casts in JEL compiled expressions. If cnmap!=null it must be java.util.Hashtable with java.lang.Class objects as elements and java.lang.String objects as keys. When the object cast "(non_primitive_type_name) var" is encountered in the expression, "the non_primitive_type_name" string is looked in the cnmap hashtable and the cast to the corresponding class is generated by JEL. The absence of the name in the hashtable produces the compile-time error. It is possible for keys in cnmap to contain "." (dot) symbols in them.
This problem appears mostly when one uses dynamic variables, but may also arise in other cases. Suppose a reference to the object of the class Weight (representing a weight of a certain item) appeared in the expression. It is clear that Weight is always represented by a floating point number (although it may have other properties, like units). If the class Weight has the method
public double getValue()
the value of weight can be accessed in expressions using syntax w.getValue(), supposing the variable w has type Weight.
To save typing (since version 0.9.3 of JEL) one may have the class "Weight" implement gnu.jel.reflect.Double interface. Then, the aforementioned getValue method will be called automatically by JEL (or object w will be "unwrapped" to primitive type). This unwrapping will be performed automatically when needed: one can have expressions "w+1.0" meaning "w.getValue()+1" and "w.getUnits()" both valid (in the second case w is not "unwrapped").
There are gnu.jel.reflect.* interfaces for all Java primitive types. To use the automatic unwrapping one just needs to make his classes to implement one of these interfaces.
There is a similar mechanism for strings (since version 0.9.6) and a corresponding empty interface gnu.jel.reflect.String to denote objects automatically convertible to java.lang.String by means of their .toString() method. For example, if x is of a class implementing gnu.jel.reflect.String interface the expression x+"a" will be compiled into x.toString()+"a" (otherwise this expression produces a error message). The objects automatically convertible to strings can also be supplied as arguments of methods requiring java.lang.String (usual method overloading rules apply). Still, in the current version of JEL it is impossible to cast methods of java.lang.String on such objects. That is x.substring(1) is a syntax error (unless x itself has the .substring(int) method). This deficiency can be addressed in future.
Expressions are made by human, and making errors is the natural property of humans, consequently, JEL has to be aware of that.
There are two places, where errors can appear. First are the compilation errors, which are thrown in the form of gnu.jel.CompilationException by the gnu.jel.Evaluator.compile(...). These errors signal about syntax problems in the entered expressions, wrong function names, illegal types combinations, but NOT about illegal values of arguments of functions. The second source of errors is the compiled code itself, Throwables, thrown out of gnu.jel.CompiledExpression.evaluate() are primarily due to the invalid values of function arguments.
Compilation errors are easy to process. Normally, you should surround compilation by the try ... catch (CompilationException )... block. Caught CompilationException can be interrogated, then, on the subject of WHERE error has occurred (gnu.jel.CompilationException.getCol()) and WHAT was the error gnu.jel.CompilationException.getMessage(). This information should then be presented to user. It is wise to use information about error column to position the cursor automatically to the erroneous place in the expression.
Errors of the second type are appearing during the function evaluation and can not be so nicely dealt with by JEL. They depend on the actual library, supplied to the compiler. For example methods of java.lang.Math do not generate any checked exceptions at all (still, Errors are possible), but you may connect library, of functions throwing exceptions. As a general rule : exceptions thrown by functions from the library are thrown from evaluate method.
In the above text the result of the computation, returned by gnu.jel.CompiledExpression.evaluate() was always an object. While this is very flexible it is not very fast. Objects have to be allocated on heap and garbage collected. When the result of computation is the Java primitive type it can be desirable to retrieve it without creation of the object. This can be done (since the version 0.2 of JEL) with evaluateXX() family of calls (see gnu.jel.CompiledExpression. There is an evaluateXX() method for each Java primitive type, if you know what type expression has you can just call the corresponding method.
If you do not know the type you can query it using gnu.jel.CompiledExpression.getType(). Be warned, that the call to wrong evaluateXX() method will result in exception. Another tricky point is that JEL always selects smallest data type for constant representation. Namely, expression "1" has type byte and not int, thus in most cases you will have to query the type, and only then, call the proper evaluateXX() method.
It is anyway possible to eliminate type checks at evaluation time completely. There is a version of gnu.jel.Evaluator.compile(...) method, which allows to fix the type of the result. It directs the compiler to perform the widening conversion to the given type, before returning the result. For example : if you fix the type to be int (passing java.lang.Integer.TYPE as an argument to compile) all expressions such as "1", "2+5", "2*2" will be evaluated by evaluate_int method of the compiled expression. Also, the attempt to evaluate "1+2L" will be rejected by compiler, asking to insert the explicit narrowing conversion (such as "(int)(1+2L)").
There used to be a specialized serialization interface in JEL up to version 0.8.3. The need for such interface was dictated by the fact that JEL allowed to use constants of arbitrary reference types in expressions, which is not supported directly by the Java class file format. Starting with version 0.9 this feature was removed and now JEL generates ordinary Java class files. To store compiled expressions into a file just grab their code with gnu.jel.Evaluator.compileBits(...). The code is returned as a byte array which is easy to save/restore. Then, the expression can be instantiated using gnu.jel.ImageLoader with the code
byte[] image; // ... code to read the JEL-generated class file into the "image" ... CompiledExpression expression=(CompiledExpression)(ImageLoader.load(image)).newInstance();
or, alternatively, by compiling your source against generated class file. Note that in this version of JEL all generated classes have the name "dump" and are in the root package. If there will be such need in future the Evaluator interface can be extended to assign user-supplied names for new expressions.
There is one serious limitation, which should be mentioned. Actually it is not a JEL limitation but rather a limitation of the typical Java run-time.
To load compiled expressions into the Java virtual machine memory JEL uses a custom java.lang.ClassLoader. While there is nothing wrong with that, setting up a ClassLoader is a privileged operation in Java. This means either JEL should run in a Java application (there are no security restrictions on Java applications), or , if JEL is distributed in some custom applet the applet should be signed.
I hope you found JEL useful. Don't hesitate to contact me if there are any problems with JEL, please, report BUGS, suggest tests, send me your patches,... There are still many improvements to be done.
Most current information about JEL should be available at http://galaxy.fzu.cz/JEL/.
JEL is the "free software" and is distributed to you under terms of GNU General Public License. Find the precise terms of the license in the file ./COPYING in the root of this distribution.
Please, contact the author directly if you'd like JEL to be commercially licensed to you on a different terms.
JEL © 1998 — 2001 by Konstantin Metlov <metlov@fzu.cz>