How Unity processes queries and large datasets
Understand DuckDB, source calculations, consistent text rules, large inputs, result limits, paging, and read progress.
Unity calculates report results with read-only DuckDB SQL over the report's saved model: it reads the source outputs it needs and finishes the calculation in its own engine.
A query can process millions of rows and return a few numbers. Those input rows are different from the rows displayed in a chart, returned by an inspection tool, or loaded into a table window.
Where calculations run
| Scenario | Execution |
|---|---|
| A calculation over a SQL Server source | Unity's engine completes it; SQL Server can first reduce the outputs it sends (below). |
| Text comparisons, grouping, or sorting | Unity's engine applies its consistent text rules. Source collation is not inspected or cached for these reads. |
| A join across different databases, files, or API outputs | Unity reads the referenced outputs and computes the combined result. |
| An imported CSV | DuckDB queries the stored typed Parquet dataset. |
| Snowflake or Databricks entities in new report SQL | The source supplies its saved entity-query output; Unity completes the report calculation. |
Source entity queries use their connector's native dialect or API request descriptor. Report calculations use named model entities and DuckDB SQL. See Semantic models and Query inspection.
Reduce work at the source
SQL Server can do useful work first. Proven source reductions can select only needed columns, apply compatible restrictions, or return partial sums and counts that Unity combines. If a reduction cannot be proved safe, Unity reads the full entity output instead.
This can reduce data transfer, but it does not make every query fast. Joins, distinct counts, median calculations, wide text, and sorting can require more work than a simple sum.
Understand input and output limits
| Quantity | Meaning |
|---|---|
| Source-input rows | No fixed 250,000-row cap. Available memory, staging/spill disk, and execution safety budgets bound the work. |
| A complete report-query response | Bounded by the endpoint's row and byte limits. An oversized complete result fails explicitly. |
| SQL Playground preview | Defaults to 1,000 returned rows; removing that preview limit still leaves runtime output limits. |
| Non-paged HTML dataset | Must return a complete result of at most 10,000 rows. |
| Matrix hierarchy | Its query computes every level and the grand total from their own rows. Any number of grouped rows: the complete result stays on the server and the matrix reads the rows it shows. |
| Detail table | Retrieves windows rather than sending its entire result to the browser. |
| CSV upload | Default file-size limit is 2 GB; there is no fixed row-count cap. The dialog shows the configured limit. |
For example, SELECT SUM(amount) FROM orders can calculate over millions of orders while returning one row. A table window showing 200 orders does not contain the data needed to sum all orders in browser JavaScript. Use a separate aggregate query for that total.
Page through large results
Native and HTML tables can reuse a temporary ordered result for later windows. It belongs to the same account, report, model revision, SQL, and parameters. Changed filters or sorting require a different read; expired or unavailable results are read again. QueryReportData reads every page of a table, matrix, paged dataset or trial query from one retained result, kept until ten minutes after its last page, so its pages agree.
The retained result is a snapshot from when it was built. Refresh to request current data. It is not a permanent import or a transaction shared by every visual in the report. An authored LIMIT remains part of the analytical query.
Follow a slow calculation
Long browser reads continue as background jobs after the initial wait. Loading progress can show rows processed and a total when known. A source can be computing before it has any rows to send, so an unknown total or a pause before the first row is not itself an error.
Use Cancel to stop a read and refresh to start another. Healthy work can continue beyond a minute, within configured resources and the safety ceiling. The default read ceiling is 15 minutes; a streaming read with no progress for two minutes can stop earlier. Jobs with nobody polling them are also stopped. See Background work.
Compare results at the same scope
Match the report model, measure, row grain, filters, text rules, and source time before comparing a chart, table, HTML visual, or chat answer. Inspection samples and table windows are partial results, while old chat answers retain their original data.
Large row counts alone do not establish a deployment's capacity. The source plan, row width, available memory/disk, and simultaneous work determine how much the backend can process. A resource error is reported rather than quietly returning a partial aggregate.
Questions about your environment? Contact us.