orchardcore-data-sources

Orchard Core Data Sources (Data Pipelines Core)

The Data Pipelines Core feature (OrchardCore.DataPipelines.Core, also DataPipelinesConstants.CoreFeatureId) ships with Orchard Core 4.0 in the Data Pipelines module. It is the data layer that data pipelines and other features, such as reports, share:

• data sources that expose site data as typed, tabular data sets, read in batches with the access of a user;
• file formats rows are written to and read from: CSV, Excel, JSON and JSON Lines;
• the formula language and filter operators (see orchardcore-data-formulas);
• bounded-memory data operations (sort, distinct, group, join) that spill to temporary files;
• shared links (see orchardcore-data-pipeline-steps).

The feature has no admin screen and is EnabledByDependencyOnly: features such as OrchardCore.DataPipelines enable it. Using pipelines: orchardcore-data-pipelines.

Packages and dependencies

<!-- Web/startup project -->
<PackageReference Include="OrchardCore.DataPipelines" Version="4.*" />

<!-- Module project that reads or adds data sources -->
<PackageReference Include="OrchardCore.DataSources.Abstractions" Version="4.*" />
<PackageReference Include="OrchardCore.DataSources.Core" Version="4.*" />

ContentDataSourceOptions lives in the OrchardCore.DataPipelines package; IDataTimeZoneResolver in OrchardCore.DataPipelines.Core.

A module that reads or adds data sources either depends on the feature in its manifest (enabling the module enables the feature), or registers its services in a [RequireFeatures("OrchardCore.DataPipelines.Core")] startup (registered once another feature, such as Data Pipelines, enabled it):

[assembly: Module(
    Name = "Support Reports",
    Version = "1.0.0",
    Dependencies = ["OrchardCore.DataPipelines.Core"]
)]

Concepts

Concept	Description
Data source	An IDataSource scoped service with a stable static Name (such as Contents).
Data set	One table of a data source. DataSetDescriptor: Name (stable), DisplayName, Description, Group, DefaultDateField (its main date field, if any), References (data sets its fields link to).
Schema	DataSetSchema: the descriptor and its ordered DataFields (Name, may contain dots such as TitlePart.Title; DisplayName, Type, Group, Description, IsIdentifier, IsKeyFilterable, References). FindField(name).
Field type	DataFieldType: Text (string), Integer (long), Decimal (decimal), Boolean (bool), Date (a calendar date, DateTime) and DateTime (UTC, or a local time without zone after a conversion or from a file). A missing value is null.
Batch	DataBatch (Fields, Rows of object[] in field order, Count, IndexOf(name)), 500 rows by default (DataSourceQuery.DefaultBatchSize).