First Look – i-luminate

Neil and I caught up with Joe Foley, Chief Architect and Kirsten Chapman, VP Global Marketing and Alliances from i-lluminate this week. illuminate (www.i-lluminate.com) is a Spanish company that has recently begun to expand in the US.

illuminate is trying to tackle what it sees as the unsolved performance problem of ad-hoc analysis in companies who have made investments in data warehousing and business intelligence. Despite the dramatic price/performance improvements, the illuminate folks think that the limitations of SQL and fixed data structures can cause complex queries to become large IT projects or even complete nightmares. In particular the fact that traditional data analytic infrastructure develops layers of de-normalized data, star schemas and massive parallelism to meet performance needs means that flexibility is lost.

illuminate’s approach is to store everything as values and to store each value only once. This means that all relationships are kept, not just those designed in, and that the database schema is built on the fly as actual data is loaded. While this takes longer that loading a traditional RDBMS, it is comparable to the time it takes to load and index a database while making the generation of extended metadata and incremental addition of new data sources very easy. One consequence of this is that after about 20-50Gb of data loaded the illuminate value pool tends to stabilize at around 6Gb. Adding more data does not increase the size of the value pool. While disk space is cheap, this approach makes it much easier to keep all the critical information in memory.

illuminate likes to talk about the time to understand – the time from the definition of a problem to the understanding of an answer. This time is improved, they say, both by matching the load and index time of a traditional database and by improving dramatically the time to get results, especially results from ad-hoc analysis. The value-based approach also allows any kind of query to be developed and supports navigation by value (regardless of logical tables). Because Illuminate stores values (“London”,” Manager”, etc.) instead of values in records, it is able to provide some interesting discovery capabilities through associative queries. For example, find everything that is associated with the date December 7, 1952. Relationships that weren’t understood before can be mined. A relational database would need to scan every table to get this information. Another associative query could start with a product and find everything that is related to it.

The iLuminate product range includes iCorrelate, a tool for ad-hoc querying that takes advantage of their approach, as well as iAnalyze, a simple dashboard tool (OEM’d from QlikTech). Support for ODBC/JDBC means that analytics tools from SAS to SPSS and Cognos to Business Objects can also use the database.

They made their point with a couple of nice case studies and then talked about their North American FREE product introduction Proof of Concept program. This is designed to prove that their database can do what nothing else can and do it fast. Companies interested in this program can define a high-value information need that users can’t get-or that takes too long to get-that uses up to 25GB of raw data from up to 6 data sources. No schemas are required, just data and there is no charge with full evaluation team participation. You can check out the program at i-lluminate.com/pilot_request.asp or the company at i-lluminate.com.