Intelligent Handling Dates in Data

Flexible date analysis with automatic temporal tracking

Scoop provides intelligent date handling that lets you analyze data across multiple time dimensions without complex configuration. Every dataset can be analyzed by different date columns, and Scoop automatically tracks when records change over time.

Understanding Date Columns in Scoop

When you load data, Scoop identifies and tracks several types of dates:

Date TypeSourcePurpose
SCOOP_TSTAMPSystem-generatedWhen the record was loaded into Scoop
Business datesYour data columnsCreated date, close date, order date, etc.
Calculated datesFormulasDerived from other dates

System Timestamp (SCOOP_TSTAMP)

Every row in Scoop has a SCOOP_TSTAMP column recording when it was loaded. This enables:

  • Tracking how your data changes over time
  • Comparing snapshots from different days
  • Identifying when records were added or modified

Changing the Analysis Date

When analyzing your data, you can switch which date column Scoop uses. Click the three-dot menu next to any metric:

Example: Sales Pipeline

Your pipeline data has multiple dates:

  • Created Date: When the opportunity was created
  • Close Date: Expected close date
  • Last Modified: When the record changed
  • SCOOP_TSTAMP: When Scoop captured this snapshot

You can analyze the same data by any of these:

  • "Revenue by Created Date" → When deals entered the pipeline
  • "Revenue by Close Date" → When deals are expected to close
  • "Revenue by SCOOP_TSTAMP" → How the pipeline looked on each snapshot day

Date Handling in Snapshot Datasets

Snapshot datasets capture the state of your data at regular intervals (typically daily). Scoop adds powerful logic to make date-based analysis work seamlessly.

Accumulating Records Over Time

Scoop tracks all unique IDs it has ever seen across all snapshots, keeping the latest values for each. This solves a common problem:

The Problem:

  • You want to analyze opportunities by their Created Date
  • But your daily report only shows currently open opportunities
  • Opportunities that closed and fell off the report would be missing

Scoop's Solution:

  • Scoop remembers every opportunity it has ever seen
  • When you analyze by Created Date, you get ALL opportunities
  • Each opportunity shows its latest known state

Example: Opportunity Analysis

Day 1 Snapshot: Opp A (Created Jan 1), Opp B (Created Jan 5)
Day 2 Snapshot: Opp A (Closed), Opp C (Created Jan 10)
Day 3 Snapshot: Opp C, Opp D (Created Jan 15)

Analysis by Created Date includes:
- Jan 1: Opp A (latest state from Day 2)
- Jan 5: Opp B (latest state from Day 1)
- Jan 10: Opp C (latest state from Day 3)
- Jan 15: Opp D (latest state from Day 3)

Even though Opp A and B are no longer in the current report, Scoop includes them when analyzing by Created Date.

How This Works Technically

For each unique key in your dataset:

  1. Scoop maintains the most recent version of each record
  2. When analyzing by a business date (like Created Date), Scoop uses this accumulated view
  3. When analyzing by SCOOP_TSTAMP, Scoop shows the data as it appeared on each load date

Date Analysis Best Practices

Choosing the Right Date

Analysis GoalDate Column to Use
Pipeline changes over timeSCOOP_TSTAMP
When deals were createdCreated Date
Revenue forecast by close dateClose Date
Activity timingActivity Date
Process durationClose Date - Created Date

Time-Based Comparisons

Scoop makes it easy to compare different time periods:

  • Year-over-year: Compare this January to last January
  • Period-over-period: Compare this month to last month
  • Point-in-time: Compare today's pipeline to 30 days ago

Multiple Date Columns

When your data has many date columns, consider:

  • Which date best answers your business question?
  • Should you create calculated date columns (e.g., Days to Close)?
  • Would multiple visualizations with different date axes be helpful?

Date Formats

Scoop automatically recognizes common date formats:

  • 2024-01-15 (ISO)
  • 01/15/2024 (US)
  • 15/01/2024 (International)
  • January 15, 2024 (Long form)
  • 1/15/24 (Short)

If Scoop doesn't recognize your date format, you can specify the format during dataset setup or use calculated columns to parse non-standard formats.

Troubleshooting Date Issues

Dates Not Recognized

  • Check the format is consistent across all rows
  • Verify there are no mixed formats in the column
  • Look for non-date values (like "TBD" or "N/A")

Wrong Time Period Shown

  • Confirm which date column is selected for the analysis
  • Check if filters are limiting the date range
  • Verify the dataset has data for the expected period

Duplicate Records

  • Ensure unique key configuration is correct
  • Check if the source report has duplicates
  • Verify SCOOP_TSTAMP filtering if analyzing snapshots

Related Topics