In order to checkout RacerPro's Performance with LUBM.
Download RacerPro 1.9 from Racer Systems (available Friday Nov. 18th). Note that RacerPro 1.9 contains significant performance improvements for handling large ABoxes. So, please do not use older versions of Racer or RacerPro.
The file http://www.sts.tu-harburg.de/~r.f.moeller/racer/lubm/lubm.lisp contains a description of how we run the LUBM benchmarks with RacerPro 1.9. Download this file by clicking the link above.
If you would like to run the LUBM benchmark with RacerPro 1.9 on your machine and reproduce our results, also download the following files:
We assume you copy the files into a folder c:\Benchmark\ and unzipped the files into according directories (make sure the extraction process does not generate unnecessarily nested directories). Thus you have
Clisp and lracer are needed to send the benchmark commands to RacerPro 1.9. Start clisp by typing
c:\Benchmark\clisp-2.33\base\lisp.exe -M "c:\Benchmark\clisp-2.33\lispinit.mem"
In Clisp you type the following commands
(load "c:/Benchmark/lracer/lracer-sysdcl.lisp") (compile-load-lracer "c:/Benchmark/lracer/") (load "c:/Benchmark/LUBM/lubm.lisp") (open-server-connection)
Then, you can start the following benchmarks:
(test1This is the fastest mode which is complete for LUBM)
(test1Additionallly enforce an ABox consistency test and measure the runtime1 t)
(test1Mode 3 is complete for SHIQ(Dn)-. This mode is not required for LUBM. An ABox consistency test is always included.3)
The complete set of LUBM queries can be evaluated as follows:
(test2This is the fastest mode which is complete for LUBM)
(test2Enforce an ABox consistency test and measure the runtime1 t)
(test2Mode 3 is complete for SHIQ(Dn)-. An ABox consistency test is always included.3)
Using Clisp you can for instance type
(test2 1)
and you should see the following output (Windows XP, 2GHz, 1GB):
[6]> (test2 1) Initializing... Reading C:/Ralf/LUBM/university/univ-bench.owl... done. Reading C:/Ralf/LUBM/university/university0-0.owl... done. Reading C:/Ralf/LUBM/university/university0-1.owl... done. Reading C:/Ralf/LUBM/university/university0-2.owl... done. Reading C:/Ralf/LUBM/university/university0-3.owl... done. Reading C:/Ralf/LUBM/university/university0-4.owl... done. Reading C:/Ralf/LUBM/university/university0-5.owl... done. Reading C:/Ralf/LUBM/university/university0-6.owl... done. Reading C:/Ralf/LUBM/university/university0-7.owl... done. Reading C:/Ralf/LUBM/university/university0-8.owl... done. Reading C:/Ralf/LUBM/university/university0-9.owl... done. Reading C:/Ralf/LUBM/university/university0-10.owl... done. Reading C:/Ralf/LUBM/university/university0-11.owl... done. Reading C:/Ralf/LUBM/university/university0-12.owl... done. Reading C:/Ralf/LUBM/university/university0-13.owl... done. Reading C:/Ralf/LUBM/university/university0-14.owl... done. Real time: 13.85993 sec. Run time: 0.2703888 sec. Space: 410296 Bytes GC: 1, GC time: 0.0 sec. ABox preparation Concept (http://www.lehigh.edu/%7Ezhp2/2004/0401/univ-bench.owl#Faculty) causes a cycle in TBox LUBM. Concept (http://www.lehigh.edu/%7Ezhp2/2004/0401/univ-bench.owl#Professor) cause s a cycle in TBox LUBM. Concept (http://www.lehigh.edu/%7Ezhp2/2004/0401/univ-bench.owl#TeachingAssistan t) causes a cycle in TBox LUBM. Concept (http://www.lehigh.edu/%7Ezhp2/2004/0401/univ-bench.owl#Person) causes a cycle in TBox LUBM. Concept (http://www.lehigh.edu/%7Ezhp2/2004/0401/univ-bench.owl#Chair) causes a cycle in TBox LUBM. Concept (http://www.lehigh.edu/%7Ezhp2/2004/0401/univ-bench.owl#Dean) causes a c ycle in TBox LUBM. Concept (http://www.lehigh.edu/%7Ezhp2/2004/0401/univ-bench.owl#Employee) causes a cycle in TBox LUBM. Real time: 1.3719728 sec. Run time: 0.1201728 sec. Space: 26968 Bytesdone. Compute index structures... Real time: 7.8713183 sec. Run time: 0.0 sec. Space: 6648 Bytesdone. Load: 13.8699 Preparation: 1.3820 Consistency: 0.0000 Index: 7.8713 Querying... *** Universities: 1 Max. Deps: ALL Query: 1 Answers: 4 Time: 0.0400576 *** Universities: 1 Max. Deps: ALL Query: 2 Answers: 0 Time: 0.550792 *** Universities: 1 Max. Deps: ALL Query: 3 Answers: 6 Time: 0.0300432 *** Universities: 1 Max. Deps: ALL Query: 4 Answers: 34 Time: 0.0600864 *** Universities: 1 Max. Deps: ALL Query: 5 Answers: 719 Time: 1.6724048 *** Universities: 1 Max. Deps: ALL Query: 6 Answers: 7790 Time: 1.2317712 *** Universities: 1 Max. Deps: ALL Query: 7 Answers: 67 Time: 0.1301872 *** Universities: 1 Max. Deps: ALL Query: 8 Answers: 7790 Time: 2.3934417 *** Universities: 1 Max. Deps: ALL Query: 9 Answers: 208 Time: 0.8912816 *** Universities: 1 Max. Deps: ALL Query: 10 Answers: 4 Time: 0.0400576 *** Universities: 1 Max. Deps: ALL Query: 11 Answers: 224 Time: 0.050072 *** Universities: 1 Max. Deps: ALL Query: 12 Answers: 15 Time: 0.650936 *** Universities: 1 Max. Deps: ALL Query: 13 Answers: 1 Time: 2.0329232 *** Universities: 1 Max. Deps: ALL Query: 14 Answers: 5916 Time: 0.2203168 [7]>
Some comments:
Loading time includes the time for reading the files (i.e. produce abstract syntax trees). ABox preparation time includes the time for encoding the ABox contents - no reasoning is applied. Indexing included the time for building index structures for retrieval - no reasoning is applied.
From Java you can access RacerPro with JRacer:
For LUBM, make sure the following commands are sent to Racer before running the LUBM benchmark.
(full-reset) (set-nrql-mode 1) (set-unique-name-assumption t) (enable-optimized-query-processing t)
Note that these settings apply to inference problems of same class as LUBM. They have been developed for accessing large-scale data repositories with DL reasoners.
You can study lubm.lisp for avoiding the transfer of tuples from the server in case of benchmarking. If tuples are transferred, you measure network latency.
Use the following pattern:
(racer-prepare-query:id ) (setf number-of-answers (get-answer-size t)))
If you have any question, please let us know.