One of the powerful features of Scoop is how it can intelligently handle changes to your source reports. If, for some reason, in the future you decide that you need to add a field to a report for example, that does not break your dataset. Scoop will recognize that the report with the new column is essentially the same report and actually modify the underlying dataset to add that new column. Going forward, that column will be included in your data and can be used.

What Scoop Does to Match Reports

When a new report comes in, Scoop does the following:

1. It identifies the columns that are present in the data.
2. It analyzes the report to see if any one of the columns is a unique key for that report. A unique key is a column that uniquely identifies an item (like a sales opportunity ID, or a lead number), and only appears once in each report. When Scoop discovers a unique key, it can make more assumptions about what it can do with your data.
3. If it is the first time you have ingested a report into a report dataset, then it will just create a new dataset table for that report.
4. If, however, you have previously loaded reports into that report dataset, it will see if it matches any of the tables that are there. If the columns match, then it adds the new data from this report to the dataset table. If, however, the columns are a bit different, it checks to see if it can alter an existing dataset to match this new report.
   1. If there is a unique key in this report, and there is a matching unique key in an existing table, it adjusts the table to match the new report (adding or removing columns accordingly)
   2. If there is not a unique key, and the new columns are numbers, it also adjusts the existing table to match
   3. However, if either of the previous conditions are not true, it is possible that the new report is aggregated differently and Scoop creates a new dataset table to handle this new report

Examining Why a Report may not Match

If Scoop unexpectedly creates a new dataset for a new report and doesn't ingest the data into an existing one, that is because the report format does not match. It's possible a column was accidentally added or removed that breaks it. Or, possibly, some larger change was made to the underlying report. Scoop provides a handy tool to help identify those cases. Going to the Dataset page for Scoop and selecting the dataset at issue, you can click on the Tables tab within the dataset.

Clicking on Compare Tables will show you the format of the two different tables side by side. You can then see what the differences are and identify if one of them is incorrect.

Deleting Bad Reports

If you discover that a bad report has been loaded into Scoop, you can delete it. On the Scoop History tab within your dataset, you can see each time a report was ingested into Scoop. There is a trash can next to each report. If you click on that you can remove that report from Scoop's history and then reprocess the dataset. That report will no longer be included in Scoop's processed data.