When Teradata bought Aster Data in 2011, it set out on a quest to bring (then) esoteric Big Data technology to the relational database faithful.  Specifically, Aster’s SQL-MapReduce facility allowed specialists to write MapReduce code that would be executed not by Hadoop over data in HDFS (the Hadoop Distributed File System), but rather in data stored in the relational database engine.  The MapReduce code can then be called form SQL queries.

Subsequent to that, Aster added a technology it calls SQL-H, which allows the Teradata Aster relational engine to operate directly over data stored in HDFS.  SQL-H has now been added to the core Terdata data warehouse product as well.

Remember 6

Today, Teradata is releasing version 6 of its Teradata Aster Discovery Platform, and extending the SQL-MapReduce concept of bringing Big Data computation patterns to a familiar SQL environment.  Specifically, the product is adding support for graph database technology via its new SQL-GR technology, and support for HDFS-compatible file storage, with the addition of the Teradata Aster File Store (AFS).  Together, these two technologies add important NoSQL and Hadoop data processing capabilities as native features of the Teradata Aster Discovery Platform.

Graph Databases for all

SQL-GR will allow for data to be stored in graph form, as a series of “nodes” and “edges.”  Essentially, this permits a database to model not only entities but the relationships between them.  Graph databases form one fo the four major subcategories of NoSQL database and, arguably, the one with the fewest practitioners.  Teradatais the best known product in the Graph database space, and the skill set required to work with it is uncommon at best.

Teradata Aster’s take on this is that embedding a graph database store inside the product and allowing functions that perform graph data manipulation to be callable from SQL queries (much as SQL-MapReduce functions are) will make this technology more widely deployed, and more widely used.  To further this goal, Teradata has designed SQL-GR in such a way that its graph databases need not fit into available system RAM on the server, thus allowing for greater scale.

HDFS home to roost

Likewise, the Aster File Store, which joins the Teradata Aster product as a peer to its column store and row store, will make it possible for any file stored in HDFS to be copied over to the Aster database and examined.  It should also make it possible for Hadoop MapReduce jobs to read from, and write data to, the Aster database.

Even Hadoop components like Sqoop and HBase should, in theory, be able to use AFS as if it were a native HDFS file store.

By Andrew Brust Read more