Custom vs. Automated Analytics - The Lost Luggage Rule Again

It is one of the few certainties in life that an Analytics professional will be asked in a sales pitch: "is your solution packaged or custom?" The answer is seldom less straight forward or general than we would like: all solutions involve some degree of automation and some of customization, and it is generally unwise to try and steer analytical services to either extreme.

Given that, there are two basic aspects to consider when deciding to automate analytics:

1. Apply the lost luggage method - as indicated in my previous post, decide what to automate, map out all your processes and make that your product, then present that to clients as ask them how different from that do they want their solution to be. This approach provides the benefits of standardization and automation but still provides for some customization and the additional value it brings to clients.
2. Focus on automating what is more amenable to automation, to that end consider the accompanying figure.This image also doubles as an indication of what analytical areas are more amenable to automation: those on the left, that depend more on leaf level data tend to lend themselves better to automation than optimization or media mix modeling; we can automate a segmentation routine to run periodically or a get a scoring algorithm to re-set its coefficients with increments in client acquisition but we'd be hard pressed to optimize a time-series model that decomposes media contributions to a business.

The takeaway for clients of analytical services is: make sure that you're getting the knowledge and decision support you need and less about whether it comes from an off-the-shelf package or is custom-designed. There are generally trade-offs between price, functionality and level of automation, make sure that you specify your needs clearly and are quoted a reasonable price. Automated solutions may seem cheaper but the trade-off may be less flexibility, accuracy, or a higher cost of maintenance (i.e. higher total cost of operation/ownership), at other extreme, custom solutions may be too costly unless some of the more common features are automated to some extent.

The takeaway for analytics providers is: automate the low value tasks and customize the high value ones. For example, data collection and transformation is as critical as it is low value, this is generally a good are to automate; also, reporting is a great candidate for automation, this is an area that OLAP vendors have defined very well, in their view reports can be classified into:

1. standard - reports that are pre-defined, standardized and nearly never changed unless their subject changes or is discontinued
2. impromptu - limited functionality that allows report users to generate some simple custom reports and drill down on the data
3. custom - reports that either rely on very specific and complex analysis processes, are based of infrequent and    or that are one-offs that serve a very specific need that is not repeated

this classification serves as a great guideline for partially automating reports.

Comments

[Tony]

I like this model and nice content! The big question is ALWAYS about going with an Out-of-the-box solutions or highly customized. The second question is ALWAYS do we try this in-house or go with a vendor.

Keep up the excellent work!