Intelligent Handling Dates in Data
Flexible date analysis with automatic temporal tracking
Scoop provides intelligent date handling that lets you analyze data across multiple time dimensions without complex configuration. Every dataset can be analyzed by different date columns, and Scoop automatically tracks when records change over time.
Understanding Date Columns in Scoop
When you load data, Scoop identifies and tracks several types of dates:
| Date Type | Source | Purpose |
|---|---|---|
| SCOOP_TSTAMP | System-generated | When the record was loaded into Scoop |
| Business dates | Your data columns | Created date, close date, order date, etc. |
| Calculated dates | Formulas | Derived from other dates |
System Timestamp (SCOOP_TSTAMP)
Every row in Scoop has a SCOOP_TSTAMP column recording when it was loaded. This enables:
- Tracking how your data changes over time
- Comparing snapshots from different days
- Identifying when records were added or modified
Changing the Analysis Date
When analyzing your data, you can switch which date column Scoop uses. Click the three-dot menu next to any metric:
Example: Sales Pipeline
Your pipeline data has multiple dates:
- Created Date: When the opportunity was created
- Close Date: Expected close date
- Last Modified: When the record changed
- SCOOP_TSTAMP: When Scoop captured this snapshot
You can analyze the same data by any of these:
- "Revenue by Created Date" → When deals entered the pipeline
- "Revenue by Close Date" → When deals are expected to close
- "Revenue by SCOOP_TSTAMP" → How the pipeline looked on each snapshot day
Date Handling in Snapshot Datasets
Snapshot datasets capture the state of your data at regular intervals (typically daily). Scoop adds powerful logic to make date-based analysis work seamlessly.
Accumulating Records Over Time
Scoop tracks all unique IDs it has ever seen across all snapshots, keeping the latest values for each. This solves a common problem:
The Problem:
- You want to analyze opportunities by their Created Date
- But your daily report only shows currently open opportunities
- Opportunities that closed and fell off the report would be missing
Scoop's Solution:
- Scoop remembers every opportunity it has ever seen
- When you analyze by Created Date, you get ALL opportunities
- Each opportunity shows its latest known state
Example: Opportunity Analysis
Day 1 Snapshot: Opp A (Created Jan 1), Opp B (Created Jan 5)
Day 2 Snapshot: Opp A (Closed), Opp C (Created Jan 10)
Day 3 Snapshot: Opp C, Opp D (Created Jan 15)
Analysis by Created Date includes:
- Jan 1: Opp A (latest state from Day 2)
- Jan 5: Opp B (latest state from Day 1)
- Jan 10: Opp C (latest state from Day 3)
- Jan 15: Opp D (latest state from Day 3)
Even though Opp A and B are no longer in the current report, Scoop includes them when analyzing by Created Date.
How This Works Technically
For each unique key in your dataset:
- Scoop maintains the most recent version of each record
- When analyzing by a business date (like Created Date), Scoop uses this accumulated view
- When analyzing by SCOOP_TSTAMP, Scoop shows the data as it appeared on each load date
Date Analysis Best Practices
Choosing the Right Date
| Analysis Goal | Date Column to Use |
|---|---|
| Pipeline changes over time | SCOOP_TSTAMP |
| When deals were created | Created Date |
| Revenue forecast by close date | Close Date |
| Activity timing | Activity Date |
| Process duration | Close Date - Created Date |
Time-Based Comparisons
Scoop makes it easy to compare different time periods:
- Year-over-year: Compare this January to last January
- Period-over-period: Compare this month to last month
- Point-in-time: Compare today's pipeline to 30 days ago
Multiple Date Columns
When your data has many date columns, consider:
- Which date best answers your business question?
- Should you create calculated date columns (e.g., Days to Close)?
- Would multiple visualizations with different date axes be helpful?
Date Formats
Scoop automatically recognizes common date formats:
2024-01-15(ISO)01/15/2024(US)15/01/2024(International)January 15, 2024(Long form)1/15/24(Short)
If Scoop doesn't recognize your date format, you can specify the format during dataset setup or use calculated columns to parse non-standard formats.
Troubleshooting Date Issues
Dates Not Recognized
- Check the format is consistent across all rows
- Verify there are no mixed formats in the column
- Look for non-date values (like "TBD" or "N/A")
Wrong Time Period Shown
- Confirm which date column is selected for the analysis
- Check if filters are limiting the date range
- Verify the dataset has data for the expected period
Duplicate Records
- Ensure unique key configuration is correct
- Check if the source report has duplicates
- Verify SCOOP_TSTAMP filtering if analyzing snapshots
Related Topics
- Snapshot Datasets - Understanding snapshot data capture
- Advanced Dataset Options - Additional configuration options
- Charting Time Series Data - Visualizing data over time
Updated 4 days ago