Update – SAS Model Manager

I got an update from SAS on SAS Model Manager recently. The new release came out August 17 alongside a new release of SAS Enterprise Miner and has some interesting new features. SAS Model Manager, as I noted in my previous review, supports the analytic model deployment lifecycle (from registering candidate models through validation, deployment, performance monitoring and requesting a new or updated model). SAS Model Manager is integrated with other SAS products like SAS Enterprise Miner (reviewed at the same time) and SAS/STAT. The new release (version 2.3) has some nice new features around deployment and monitoring.

The publish function in SAS Model Manager has been extended in the new release to support Netezza and IBM DB2 for in-database scoring. This uses integration with the SAS Scoring Accelerator to allow deployment of models directly to these warehouse/database environments. This integration has a nice feature that allows immediate access to the data in Teradata/Netezza/ IBM DB2 while you deploy the models. This allows you to run the model against your data to see how it will work. You can see what kinds of results you might get from a sample before deploying it. This lets you do a final test on your scoring code before deployment and without having to develop a test harness etc. Very nice.

The performance reporting in SAS Model Manager has also been extended to support dashboards. SAS Model Manager allows you to report against various features or characteristics of a model. Now you can add an indicator from the list supported in the reporting environment to a dashboard. For instance you can track decay in the lift generated by a model or track variables (predictors) that vary from the expected values. Having assembled the models/metrics that you care about you can either build an HTML dashboard from this or generate a SAS dataset. The HTML version quickly gives you an interactive report on these indicators for all the models managed in SAS Model Manager. This allows a view across the various models deployed to see which ones might have problems, helping analytic teams focus on models that are decaying or showing problems. Some companies have lots of models deployed and this is a nice way to keep an eye on them all.

Alternatively, and more interestingly, SAS Model Manager can generate a SAS dataset of this data. This allows it to be integrated with any another dashboard environment. This would allow this information to be merged with information, for instance, from a rules environment or from a business perspective. I really liked this as it would allow you to develop a single decisioning dashboard showing business, analytic and rules information. For instance, if you had a customer retention decisioning environment you could develop a dashboard that showed:

• Which offers were made to retain customers and which ones worked (from your rules engine)
• How overall churn results and income from retained customers look (from your standard performance management environment)
• The specifics of the model performance in this context (from SAS Model Manager)

Very nice.

This release is now available and longer term plans includes more support for monitoring the analytic workflow, increased R support (all the way from registering models to monitoring them), model retraining from within SAS Model Manager and Champion/Challenger back testing.