Snapshot datasets capture the state of your business entities at regular intervals, enabling powerful analysis of how things change over time. Unlike transactional data that records events, snapshot data tracks the evolving state of items like sales opportunities, support tickets, inventory levels, and customer accounts.

Why Snapshots Matter

Most BI tools can only show you current state or aggregate historical transactions. Snapshots unlock a fundamentally different type of analysis—understanding how individual entities evolve through their lifecycle.

Two Types of Data

Business applications store fundamentally different categories of data:

Transactional Data

Events that happened and don't change:

• Web page visits
• Store purchases
• Email sends
• Log entries

Analysis approach: Aggregate by attributes (date, product, region)

Entity Data (Requires Snapshots)

Items with evolving status:

• Sales opportunities moving through stages
• Support tickets progressing to resolution
• Inventory items being ordered, received, sold
• Customer accounts changing tiers

Analysis approach: Track state over time, analyze transitions

How Scoop Snapshots Work

Day 1: Opportunity A = "Prospecting" → Snapshot stored
Day 2: Opportunity A = "Prospecting" → Snapshot stored (no change)
Day 3: Opportunity A = "Qualification" → Snapshot stored (change detected!)
Day 4: Opportunity A = "Proposal" → Snapshot stored (change detected!)
...

Scoop automatically:

1. Stores each snapshot with its timestamp
2. Detects changes between consecutive snapshots
3. Tracks transitions (what changed from what to what)
4. Calculates durations (time spent in each state)
5. Enables analysis of velocity, conversion rates, and patterns

Setting Up Snapshot Datasets

Step 1: Create the Dataset

When creating a new dataset, select Snapshot as the dataset type:

This tells Scoop to:

• Store each data load as a point-in-time snapshot
• Compare consecutive snapshots to detect changes
• Enable change analysis and process visualization

Step 2: Ensure You Have a Unique Key

For snapshot analysis to work, Scoop must track the same entity across snapshots. This requires a unique key—a column that uniquely identifies each record.

How Scoop Detects Keys

Scoop automatically scans your data to find unique columns. When found, a key icon appears next to the column:

Debugging Uniqueness Issues

If Scoop doesn't detect a key you expected:

1. Click View Profile next to the column
2. Enable Analyze Uniqueness
3. Review the top non-unique values

Common causes of non-uniqueness:

• Duplicate records in source data
• Multiple line items per parent record
• Historical data included (same ID appears multiple times)

Creating Composite Keys

If no single column is unique, create a composite key using Calculated Columns:

=CONCATENATE(A2,"-",B2)  // Combine Order ID + Line Number

Step 3: Configure Source Filters

Your source report should include:

1. All currently active items — Items in the lifecycle you want to track
2. Recently closed items — Items that completed their lifecycle today/yesterday

📘

Recommended filter pattern: Include all records that are either currently open OR were modified in the last 1-2 days

Why Include Recently Closed Items?

Day 1: Opportunity in "Proposal" stage → Snapshot captured
Day 2: Opportunity moves to "Closed Won" → If not in today's report,
                                            Scoop can't record the transition!

By including recently modified items, you capture the final state change.

Filter Examples by Application

Intelligent Gap Filling

Scoop handles a common real-world scenario: items that temporarily disappear from reports.

Example: A sales opportunity is closed (removed from "Open Opportunities" report), then later re-opened.

Day 1-10:  Opportunity in report → Snapshots captured
Day 11-20: Opportunity closed, not in report → Gap in data
Day 21+:   Opportunity re-opened, back in report → Snapshots resume

Scoop automatically:

1. Detects the gap
2. Fills missing snapshots with the last known state
3. Maintains continuous history for analysis

This ensures you can track items through their entire lifecycle, even if they temporarily fall out of your source reports.

Change Analysis

For snapshot datasets, Scoop automatically analyzes changes between consecutive snapshots:

Types of Changes Detected

Using Change Data

Process Analysis

Visualize lifecycle flows with Sankey diagrams:

           ┌─────────────┐
Prospecting│ 100 started │
           └──────┬──────┘
                  ▼
           ┌─────────────┐
Qualification│ 80 (20 lost)│
           └──────┬──────┘
                  ▼
           ┌─────────────┐
  Proposal │ 50 (30 lost)│
           └──────┬──────┘
                  ▼
           ┌─────────────┐
 Closed Won│ 30 (20 lost)│
           └─────────────┘

See Process Analysis for details.

Explorer Analysis

Query change data directly:

• "How many opportunities moved from Proposal to Closed Won last month?"
• "What's the average time in Qualification stage?"
• "Which reps have the fastest stage progression?"

Best Practices

Snapshot Frequency

Data Volume Considerations

Snapshots accumulate data over time. For high-volume datasets:

• Use appropriate filters to limit active items
• Consider archiving old snapshots
• Monitor dataset size periodically

Common Pitfalls

Troubleshooting

No Key Detected

• Verify your data has a truly unique column
• Check for duplicate IDs in source data
• Consider creating a composite key

Changes Not Appearing

• Confirm dataset type is "Snapshot" not "Transactional"
• Verify data is being loaded daily
• Check that items exist in consecutive snapshots

Unexpected Gaps

• Review source report filters
• Ensure recently modified items are included
• Check for data loading failures on specific dates

Related Topics

• Process Analysis - Visualize lifecycle flows
• Intelligent Date Handling - Date intelligence
• Adding Calculated Columns - Create composite keys
• Filtering Data - Filter datasets during load