Scoop's ability to automatically snapshot data is a powerful tool generally not present in operational reporting the comes built into business applications, and far more flexible than the very few cases in may be found. This page will first discuss what snapshotting is in detail and the enormous analytical benefit it provides. It will then explain how snapshotting works in Scoop.

What is a snapshot? and what can it be used for?

Business applications typically store two different categories of data:

• Transactions: transactions are basically things that happened and now represent history. Think of things like web logs, store purchases, etc. These items, once created, generally do not change. The primary way to analyze them is to simply aggregate them by whatever attributes may be associated with them: e.g. web page, product id, etc.
• Business entities: these are items where the application is tracking the status of something that can change over time. Think for example a sales opportunity, a marketing lead, a service request, etc. The application has attributes for these items that can change over time, often as part of some sort of a life cycle. In addition to being able to aggregate these items by their attributes, it is very often much more powerful to analyze how these items change over time and move through their life cycle. This is what snapshotting enables.

Snapshotting works the way it sounds, by taking regular snapshots of the current state of each item in a population. Doing this has historically required complex database and analytical work to create the data structures required to store these snapshots and the logic required to analyze them properly. Scoop completely automates this for you. In addition, Scoop adds advanced change analysis processing that enables you to analyze how these items change over time, their conversion rates, their velocity, their effectiveness and their cycle times.

How do you setup snapshots?

If you want to analyze how particular business items change over time, you can leverage snapshot datasets. You do this by selecting Snapshot when you create a new dataset.

What should your source report contain?

Unique Key

In order for snapshot analysis to provide its most value, one must be able to compare the same item across snapshots. This means it is important that one column in your source report contain a unique identifying value, called a unique key. For example for sales opportunities in a CRM system, it would typically be the opportunity ID. Scoop will scan your dataset to detect your unique key. If it finds one, it will enable all of the change analysis and process analysis possible by being able to compare the status of each opportunity/deal across all snapshots. If that unique key is not present, then change analysis is not possible. When you look at the details of your report dataset, if Scoop found a unique column you will see a Key symbol in the key column next to it.

If your report does not have a unique key and you are not sure why because you think a given column should be unique, you can click on the "View Profile" option next to the column you think should be unique. There is a slider called "Analyze Uniqueness" on that page. If you select it, it will identify the top non-unique values in that column so you can see why it is not unique.

Sometimes you can create your own unique column by combining other columns into a unique key. See Adding Calculated Columns for more information about adding new calculated columns to your dataset.

Filters

Generally speaking, when setting up your source report, you want to make sure it has all the items you care about and need to track. Many applications have a similar pattern. There are currently a set of "active" items that you are looking to track through a life cycle. Let's use sales opportunities as an example. At any point in time there are only a certain number of sales opportunities which are "open" or "active" at any one point. So instead of including all opportunities that exist in your source application, you would include in the report only the opportunities that are currently open. Additionally, since you want to track when an item ends its life cycle by "closing" you will need to also add to your filter to include any record that has changed open status today. Doing this will ensure you snapshot all meaningful changes in your dataset and will enable powerful process analysis with those changes. See Process Analysis for more.

📘

To summarize, your filter should include all opportunities that are either currently open or have been modified in the last day (or maybe last few days)

Additional Intelligence in Snapshots

Sometimes an item like a sales opportunity can become inactive (or closed) and may therefore no longer be included in the daily report. However, it is possible in the future for that opportunity to re-open and it would then start appearing in the dataset. However, there will be a gap in prior Scoop snapshots with that item not having been in the reports while it was inactive. Scoop will detect this situation and automatically fill in the missing data for you so that you can track this item during the entire history.

Change analysis

For snapshot datasets, Scoop automatically analyzes your data for changes and then allows you to analyze those changes to do some powerful analysis. Scoop automatically tracks all changes and identifies when attributes switch values, numbers increase or decrease or dates move forward or backward.

You can utilize this change data a couple of different ways:

1. Process Analysis (Process Diagram or Sankey Chart): Process analysis helps you visualize the life cycle of a business entity and then ask meaningful questions about it to understand where your processes are performing well or not so well. See Process Analysis
2. In Explorer: Here, you can actually directly analyze the change data to see things like how conversions from stage A to stage B have been performing over time.