Contact People Projects Teaching Papers Intranet






l
Prototype: ScalaQL (beta)
Last updated: 2009-10-08
Authors: Miguel Garcia, http://www.sts.tu-harburg.de/people/mi.garcia/

Anastasia Izmaylova,
Important Notice: This prototype is a proof of concept, and a partial implementation of the design discussed in the paper Extending Scala with database query capability (for details see Publications at the end of this page). The intended audience for the prototype are compiler geeks, interested in the field of language integrated query. The prototype is not supported in any way.
Introduction:

The integration of database and programming languages is difficult due to the different data models and type systems prevalent in each field. We present a solution where the developer may express queries encompassing program and database data. The notation used for queries is based on comprehensions, a declarative style that does not impose any specific execution strategy. In our approach, the type safety of language-integrated queries is analyzed at compile-time, followed by a translation that optimizes for database evaluation. We show the translation total and semantics preserving, and introduce a language-independent classification. According to this classification, our approach compares favorably with Microsoft's LINQ, today's best known representative. We provide an implementation in terms of a Scala compiler plugin, accepting two notations for queries: LINQ and the native Scala syntax for comprehensions. The prototype relies on Ferry, a query language that already supports comprehensions yet targets SQL:1999. The reported techniques pave the way for further progress in bridging the programming and the database worlds.

From LINQ to Scala: From a user perspective, given an input LINQ query in a Scala source file as depicted in Figure 1, running ScalaQL as part of a build results in a Scala-based formulation of the same query.
Figure 1: Before LINQ query translation

Scala to Ferry: Figure 2 illustrates the second main use case, where the starting point is a query in Scala that is translated into the database query language (Ferry).
Figure 2: Translation from Scala into Ferry

Architecture of the Prototype: The ScalaQL prototype follows the Scala compiler architecture, which is based on a compiler operating in phases and an interaction protocol defined for compiler plugins. These plugins can update ASTs on their way from one phase to the next one, as well as generate errors and warnings.
Figure 3: Integration of the ScalaQL compiler plugins and other compilation phases

The organization of the source code mirrors the architecture of ScalaQL. The delivered Eclipse project contains four packages:
  • sts.ferry: contains the source for our Scala-based type and well-formedness checker of Ferry queries (Full Ferry including Ferry Core);
  • sts.compilerPlugin: contains the source of the compiler plugin proper, which extends scalac with two custom phases, LINQToScala and ScalaToFerry. These phases are executed after the scalac 'parser' phase.
  • sts.linq: contains the source of LINQToScala, a parser of LINQ queries that translates them to Scala comprehensions. In more detail, the parser takes as input a string representing a LINQ query and returns another string containing the corresponding Scala comprehension.
  • sts.compilerPlugin.annotations: contains the annotation types (@LINQAnn and @Persistent) used to tag the inputs to the LINQToScala and ScalaToFerry phases.

Summing up, the LINQToScala phase invokes the LINQToScala Converter, parses the resulting string into Abstract Syntax Tree nodes (using scalac's parser), and replaces the AST node of the original embedded LINQ query with the obtained AST nodes (LINQToScala Transformer).

The ScalaToFerry phase translates Scala comprehensions into Ferry queries (ScalaToFerry Translator) and typechecks the obtained Ferry queries (ScalaToFerry Typer).

How to Build the Source Code:

The ScalaQLproject can be built following these steps:

  1. Download the zip file containg sources (see below)
  2. Import the existing ScalaQL project from the downloaded archive to your Eclipse workspace (the Scala IDE for Eclipse has to be pre-installed, detailed instructions can be found here)
  3. Add the following jars to the 'lib' folder, make them part of the buildpath:
    • scala-library.jar (scala distribution)
    • scala-compiler.jar (scala distribution)
    • kiama-0.5dev.jar (kiama project)
  4. Run the provided build.xml. This requires one slight change: the property scala.home should be set to reflect the location of your Scala installation (i.e., where scala-library.jar and scala-compiler.jar are located, and by the way, we have used Scala 2.7.4-final). That distribution is available at http://www.scala-lang.org/downloads). Running build.xml will delete the 'bin' directory (the output folder for class files), compile the source code from the 'src' folder to a new 'bin' folder, and prepare two jar files, ScalaQL-1.0(beta)dev.jar and ScalaQL-1.0(beta)dev-src.jar, placed to 'dist' child directory. The first file, ScalaQL-1.0(beta)dev.jar, includes all the required class files along with plugin.properties and the all-important scalac-plugin.xml.
  5. Development has relied on Kiama 0.5 for parsing, which can be found at http://code.google.com/p/kiama/
  6. Create a new run/debug configuration (as shown in Figure 4 below) with the main class scala.tools.nsc.Main. The file ScalaQL-1.0(beta)dev.jar should be passed to scalac as -Xplugin option argument (i.e. ${workspace_loc}\ScalaQL\dist\ScalaQL-1.0(beta)dev.jar) along with the target .scala source file to be compiled (for example, ${workspace_loc}\ScalaQLTest\test\MyQLAppl.scala).
    For additional details see [scala-tools] running/debuging compiler plugin from Eclipse
Figure 4: Run configuration to debug the compiler plugin

The ScalaQLTest project is also available as a zip archive and can be downloaded and imported as an Eclipse Scala project. It includes two test source files, MyQLAppl.scala and SomeTestCases.scala, provided to demonstrate the ScalaQL prototype in action. The AST tree before and after LINQ2Scala transformation can be seen from the console output as well as ScalaToFerry translation results performed by the second ScalaToFerry phase.
The Scala Compiler Corner A collection of resources contributed by the Scala user community.

Additional links:

License: /*******************************************************************************
* This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v1.0
* which accompanies this distribution, and is available at
* http://www.eclipse.org/legal/epl-v10.html
*
* Contributors:
* Anastasia Izmaylova,
* Miguel Garcia, http://www.sts.tu-harburg.de/people/mi.garcia
*******************************************************************************/
Download (binary): ScalaQL-1.0(beta)dev.jar
Download (sources): ScalaQL20091008.zip
ScalaQLTest20091008.zip
Publications: Miguel Garcia, Anastasia Izmaylova, Sibylle Schupp.
Extending Scala with database query capability
Abstract   Preprint   BibTeX
in Journal of Object Technology, July-August 2010, (To appear).
http://www.jot.fm.

Miguel Garcia, Anastasia Izmaylova.
Compiling LINQ and a Scala subset into SQL:1999
PDF   BibTeX
Technical report, Software Systems Institute (STS), Hamburg University of Technology (TUHH), Germany, September 2009.

Anastasia Izmaylova.
Program transformation in Scala
PDF   BibTeX
Master thesis, Software Systems Institute (STS), Hamburg University of Technology (TUHH), Germany, October 2009.