I got an update from the folks at Ingres the other data to talk about Ingres VectorWise. Like many of the companies I have spoken to recently, Ingres is focused on how to provide an analytic infrastructure that handles the increasing volume and complexity of data so that analysis can be done on up to the second data in a variety of ways to support strategic, management and operational /transactional decision making. As someone focused on operational / transactional decision making in particular I was interested in the ability of Ingres VectorWise to support and allow detailed analysis of very granular data.
For instance one hedge fund they work with needs to do risk analytics against lots of data. Historically these analyses took too long and were restricted to summary data. VectorWise allows them to load all the raw data in a single big table without having to apply a star schema or pre-defined dimensions. The group was able to use fundamentally the same applications and SQL and got their queries down from 3 hours to 15-30 seconds. As a result the users can now interact with the operational data at a granular level. They can also add a new data set (of any particular kind of data that might be useful) and immediately analyze it. All, say Ingres, with no change to SQL
The key element of delivering faster access and fast loads without pre-defining how it will be accessed is to “light up” the chips in the servers to make the most of processing power. VectorWise is fundamentally a column store that takes full advantage of the on-chip cache memory (much faster than regular memory) and uses vector processing to maximize the usage of the chips. The use of a column store means they avoid the delay of having to pre-define dimensions while the memory/chip enhancements deliver additional performance gains. And this speed comes from commodity servers which, when combined with a low software price point, makes it very cost effective.
They had another example where a company provides predictive analytics for their clients and had always used in-memory processing with specialty C algorithms to deliver these models. Using this approach caused problems however as all the routines were custom and the use of only in-memory data meant it was hard to re-run models. With Vectorwise this company can load more data (no memory limit) and run the code against the database with slower but still very acceptable performance. Ingres is also partnering with Revolution Analytics to deliver advanced analytics that work against this store.
At the end of the day they say their customers find themselves able to do things that were not possible before – like giving professionals access to raw data because the queries/analysis won’t grind to a halt and don’t require pre-defined dimensions or being able to combine reporting/reference data with real-time data.
There’s more information on the Rohatyn Group here and on VectorWise here.
