Per-record data packages

Available from sfp v51. This format is in beta — the layout and commands may still change.

A data package normally stores its records as one flat <sObject>.csv per object. The per-record format stores each record as its own file under a records/ folder instead. The package still deploys through SFDMU in exactly the same way — only how the records are kept in git changes.

Use it when a data package is edited by hand or reviewed in pull requests. One file per record means:

• Reviewable diffs — a change to one record touches one file, not a 200-row csv cell.
• No merge conflicts between records — two people editing different records edit different files.
• Large fields as real files — rich-text and long-text bodies become .html/.txt files instead of giant csv cells.

Enabling the per-record format

Add "dataFormat": "perRecord" to the package descriptor in sfdx-project.json:

{
    "path": "data/accounts",
    "package": "account-data",
    "type": "data",
    "dataFormat": "perRecord",
    "versionNumber": "1.0.0.0"
}

The package still needs its export.json — the per-record files are generated from, and flattened back to, the csv that SFDMU reads.

Layout

data/accounts/
  export.json
  records/
    Account/
      _schema.yaml          # the full exported column set, in order
      Acme-Inc.yaml         # one record per file
      Globex.yaml
      _content/
        Acme-Inc.Description.html   # extracted large / rich-text field bodies

A record file holds the record's fields directly. Fields longer than about 2 KB, or detected as rich text/HTML, are extracted to a file under _content/ and referenced with a $file pointer:

Name: Acme Inc
Industry: Manufacturing
Description:
  $file: _content/Acme-Inc.Description.html

The file name (Acme-Inc) is derived from the record's external id, so the same record lands in the same file every time it is pulled.

Records are id-free by default

Salesforce ids differ between every org and sandbox, so per-record files omit them by default to keep records identical wherever they were pulled from. This is driven by SFDMU's native excludeIdsFromCSVFiles setting, which sfp's tooling defaults to true:

• The record's own Id and audit fields (CreatedDate, LastModifiedById, SystemModstamp, …) are stripped.
• Raw lookup id columns are stripped when their __r. relationship column is present — SFDMU resolves the lookup from the relationship on deploy.
• Upserts match on the object's external id, never on Id.

To keep ids instead, set excludeIdsFromCSVFiles to false in export.json, or pass --keep-ids to the convert command below.

Working with a per-record package

These commands run from the project root and route automatically to the data path when -p names a data package.

Pull records from an org

sfp pull -p account-data -o myOrg

Exports the records from the org and writes them as the records/ layout. For a per-record package, the intermediate csv files are removed afterwards — records/ is the source of truth in git.

Preview a pull without changing anything

sfp pull -p account-data -o myOrg --preview

Shows what a pull would change — records added, modified, deleted, or unchanged — without touching the working tree. Add --json for machine-readable output.

Push records to an org

sfp push -p account-data -o myOrg

Flattens records/ back to csv, deploys through SFDMU, and cleans up the generated csv afterwards.

Convert an existing csv package

sfp package data convert is an offline transform — no org, no build. Use it to migrate an existing csv data package to the per-record layout, or to inspect the flattened csv:

# csv -> per-record YAML
sfp package data convert -p account-data --to records

# per-record YAML -> csv
sfp package data convert -p account-data --to csv

# convert every data package in the project
sfp package data convert --all --to records

Flag	Effect
--to records / --to csv	Direction of the conversion (required).
--all	Convert every data package in the project.
--keep-ids	Keep org-specific Salesforce ids instead of stripping them.
--keep-csv	Keep the source <sObject>.csv alongside records/ after converting to records.

At build time

When sfp builds the package, the records/ layout is flattened back to <sObject>.csv and that csv is what goes into the versioned artifact. The records/ folder is the git source of truth and is not shipped in the artifact, so installation behaves exactly like any other data package.