First Look: Precog

Precog was founded in late 2010 to solve the challenges identified by the founder when he was working on developing an online advertising platform. An online ad platform generates a tremendous amount of data, data that can be used to target ads or select specific creative for an ad, but managing and using this data is a challenge when you are trying to develop this as an operational system. You need to handle a lot of data that is generated automatically by the platform, much of it only semi-structured (log data for instance), integrate it with other data sources, and then automate how this data is analyzed and used so that you can scale to the number of analytic models that you need. He found that existing tools required a set of data scientists or analytic professionals to do the modeling and another set of engineers to build the resulting models into a production environment.

Precog was founded to address the challenges found in building such a platform. Precog is designed to support the exploratory analysis element of analytics, the construction of predictive analytic models and the operationalizing of the resulting analytics. Precog is now about 15 people (mostly engineers) and is based in Boulder, CO.

Precog is a cloud-based or on-premise “next-generation analytics platform” that has a number of elements including:

• The Precog Platform itself
  The core platform supports the analytic process from data capture to deployment. Precog stores all the various data types involved for analysis. This could be structured, unstructured or semi-structured data. Data can be captured from applications, uploaded or synchronized with existing databases. Data that has a format is stored into its structure as it is ingested.
  The Precog Marketplace provides a collection of applications for transforming, integrating or cleaning data – from Precog and from third parties. The applications are strung together into a workflow to process data into a form that works. Each app might add value or detail to the data. For instance an app might transform it from a raw log to something more useful by applying semantic identification to identify IP addresses or dates and then storing this as meta data associated with the log. Another app might use this meta data to do geolocation while another applies a place identification application that adds meta data about the real-world place represented by the location. Many of these applications rely heavily on machine learning.
• Labcoat
  Labcoat is an interactive query builder and explorer that works on the resulting data. This environment lets you do all the usual data manipulation required in analytics, whether the data is structured or unstructured. Everything, from data queries to data clean up to predictive analytics, is done through a single interface. Tutorials and libraries of reusable elements are integrated into the interface which has deep support for Quirrel, a declarative data query language designed to have a very small set of core elements (making it easy to lean like SQL) while also being completely multi-threaded/distributable to clusters and suitable for handling large amounts of unstructured or semi-structured data.
  Labcoat generates code, such as python code, that can be easily deployed to operationalize the models that are created. Any step or manipulation that is done in the development environment can be packages up as usable code. This code can be scoring code (applying a finished model to a specific customer or transaction) or it could be model development code (to create a new model on the fly) or any combination.
  Precog also provides some model management in that it can be configured to update models on a schedule. You can set cache settings that determine how long it will cache models or model results and so cause it to re-run the modeling steps based on these settings. The same cache metaphor can be applied to all the modeling activities in Labcoat.
• ReportGrid
  The final element is a tool for building reports and dashboards. There is a visual component as well as an API. This is a classic tool for selecting a data source, displaying charts etc. The charts can be saved as code that runs in a dashboard or as a fixed chart.

Precog is going to be one of the vendors in our Decision Management Systems Platform Technologies Report, and you can get more information on them here.