Reprocessing Data

Rebuild your dataset with updated calculations and structure

When you make structural changes to a dataset—like adding calculated columns, modifying formulas, or changing column mappings—you need to reprocess the dataset to apply those changes to all your historical data.

What Reprocessing Does

Reprocessing rebuilds your dataset from scratch:

  1. Clears existing data - All processed rows are removed from the dataset
  2. Replays source reports - Every source file/report is re-ingested in chronological order
  3. Applies current calculations - Your new or modified calculated columns run against all data
  4. Rebuilds snapshots - If snapshotting is enabled, snapshot history is preserved and recalculated

This ensures your entire dataset—including historical data—reflects your current column definitions and formulas.

When to Reprocess

You need to reprocess when you:

  • Add a new calculated column
  • Modify an existing calculated column formula
  • Change column include/exclude settings
  • Update a VLOOKUP table used in calculations
  • Fix a formula error

You don't need to reprocess when you:

  • New source data arrives (this processes automatically)
  • You create charts or visualizations
  • You change dataset name or description
  • You modify column display settings (like hiding columns in Explorer)

How to Reprocess

Step 1: Make Your Changes

First, make your structural changes in the dataset settings:

  • Go to your dataset
  • Click SettingsCalculated Columns
  • Add or modify your columns and formulas
  • Click Save Changes

Step 2: Initiate Reprocessing

After saving, click the Reprocess Data button.

Step 3: Monitor Progress

A progress indicator appears on the Datasets page showing:

  • Number of source reports being processed
  • Current processing stage
  • Estimated time remaining

Step 4: Verify Results

Once complete:

  1. Go to the Preview Data tab to spot-check your calculations
  2. Open Explorer to verify your new columns appear correctly
  3. Check that historical data has the expected values

Processing Time

Reprocessing time depends on:

FactorImpact
Number of source reportsMore reports = longer processing
Dataset size (rows)Larger datasets take more time
Formula complexityComplex VLOOKUPs add processing time
External sheet referencesGoogle Sheets lookups add latency

Typical processing times:

  • Small dataset (< 10K rows, 10 reports): 30 seconds - 2 minutes
  • Medium dataset (10K-100K rows, 50 reports): 2-10 minutes
  • Large dataset (100K+ rows, 100+ reports): 10-30 minutes

For very large datasets, consider processing during off-peak hours.

Best Practices

Test Before Full Reprocess

Before reprocessing a large dataset:

  1. Use the Preview Data feature to test your formula on sample rows
  2. Verify the formula logic is correct before committing to a full reprocess
  3. Fix any errors in preview mode—it's much faster than reprocessing twice

Plan for Blended Datasets

If your dataset is a source for other blended datasets:

  • Those downstream datasets may also need reprocessing
  • Check the "Reprocess when source updates" option on blended datasets to automate this

Snapshot Considerations

Reprocessing a snapshot dataset:

  • Preserves your snapshot history
  • Recalculates values within each snapshot based on new formulas
  • Does NOT change which records were captured in each snapshot

Avoid Reprocessing Loops

If Dataset A blends with Dataset B, and both have "auto-reprocess" enabled:

  • Be careful not to create circular dependencies
  • One dataset should be the "source" and the other the "consumer"

Troubleshooting

Reprocessing Takes Too Long

  • Simplify complex formulas where possible
  • Consider breaking large calculated column sheets into smaller, focused columns
  • Check if external VLOOKUP references are slowing things down

Calculated Column Shows Errors After Reprocess

  • Open the Calculated Columns editor and check the Values preview row
  • Look for #REF!, #VALUE!, or #N/A errors
  • Verify that column references still exist (columns may have been renamed in source)

Data Looks Different After Reprocess

  • Check if source report format changed between uploads
  • Verify column mappings are still correct
  • Review any VLOOKUP tables to ensure they have the expected values

Related Topics