Update Rules Reference

This page is the full reference for the rules language used by Bulk Update files. It is written so that an AI assistant can author correct rules from it.

Working with an AI assistant? Give it the plain-Markdown version of this page: /bulk-operations/rules-reference/agents.md. Ask the assistant to fetch that address, or if it cannot fetch pages, download the file and attach it to your conversation. Both are more reliable than copying and pasting this web page.

Operators rarely need this page directly. For when rules are needed at all, and the workflow around them, see Bulk Update and Bulk Action Lists. The same language also drives CDR Processing Rules, which documents the CDR-specific parts.

For the AI Assistant: Context and Ground Rules

You are writing a YAML ruleset for the SAFE Billing Platform’s bulk update engine. The engine reads an uploaded spreadsheet (CSV or Excel), maps its columns to platform fields, resolves each row to a platform record by name or id, and stages the changes for a human to test and apply. Rules never change data directly; a separate test step shows the human the outcome of every row before anything is saved.

Ground rules that prevent most mistakes:

1. Rules are an overlay, not a replacement. The platform first derives a complete configuration automatically from the file’s column headers. Your YAML is merged over it. Write rules only for what the automatic recognition gets wrong; do not restate what already works. The exact merge behaviour is documented under “How the Overlay Merges” below.
2. Validation is strict. Unknown keys, misspelt transform names and malformed structures are errors, reported with the YAML path (for example fields.duration: declare exactly one of header, column). Never invent keys or values not listed on this page.
3. Quote header names exactly as they appear in the file. Header matching trims whitespace, ignores case and ignores a trailing colon, but nothing else.
4. Declare version: 1 and kind: bulk-action at the top. They are optional, but declaring them makes clear which schema the ruleset follows.
5. Decision-only columns get temp. names. A column that should influence the rules but not the update, such as a report’s “(Information Only)” context columns, is mapped to a temp.-prefixed field. Give it an aka with the natural short name (aka: 'Number Type') and refer to it by that in conditions. See “Decision-Only Columns and Temporary Fields” below.
6. Compare numbers numerically. equals is a string comparison, so equals: ['0'] does not match a cell holding 0.00. Use over/under/atLeast/atMost for numeric tests; “exactly zero” is allOf with atLeast: 0 and atMost: 0.
7. Name every skip and warn rule for the operator. The name: appears verbatim in the user-visible run log (“Rows skipped: 3 Recurring Charge is zero (lines …)”), so write a short plain description. Give each distinct reason its own named rule rather than one catch-all, so the log counts each reason separately.
8. Ask for the file’s header row and a few sample data rows if they have not been provided. Rules written blind are usually wrong.
9. Tell the user to paste the finished rules into the “Advanced - rules override” box and click “Validate rules”. If validation reports an error, they should paste the error back to you.

Document Structure

A bulk-action ruleset may contain any of these top-level keys. All are optional; an empty overlay is valid.

Key	Purpose
version	Optional. Only 1 is accepted.
kind	Optional. bulk-action for update files.
target	Optional. May restate {object: ..., action: ...}; validated against the file’s chosen object and action, never used to change them.
file	How to read the file: format, encoding, delimiter, header handling.
fields	Column-to-field mapping and per-field value handling.
ignore	List of column headings to discard without warnings.
rowRules	Row-level skip, stop and flag logic.
lookups	The record-matching chain. Auto-derived; override only when necessary.
lookupOptions	Options for the matching behaviour of named lookups.
emit	What each resolved row writes. Auto-derived; rarely overridden.

Field names for the target object use the form object.fieldName, for example feature.serviceCharge or number.site. The platform’s automatic recognition tells the operator about unrecognised columns, which is usually the list of things your rules need to map.

file: - Reading the File

Key	Default	Meaning
format	csv	csv, excel/xlsx, or xls/legacy excel.
encoding	utf-8-sig	Text encoding for CSV files, for example latin-1 or windows-1252.
delimiter	,	Single character, or tab.
quote	"	CSV quote character.
header	true	Whether the first row (after any skipping) is a header row. With header: false, fields must use positional column: sources.
skipLinesBeforeHeader	0	Lines to discard before the header row.
skipLinesUntil	-	Skip lines until one contains this text; that line becomes the header.
skipLinesBeforeData	0	Lines to discard between the header and the data.
fixedWidths	-	List of column widths for fixed-width files, for example [10, 20, 8].
missingColumns	process	What to do when a row is shorter than the mapped columns: process reads the missing cells as empty; skip discards the row and reports it.

fields: - Mapping and Shaping Values

Each entry maps one logical field to a source in the file and describes how to clean its value:

fields:
  feature.serviceCharge:
    header: 'Monthly Fee'
    transforms:
      - remove: currency symbols

Sources (exactly one per field)

Source	Example	Meaning
header	header: 'Monthly Fee'	The column with this heading.
headers	headers: ['DDI 1', 'DDI 2']	Several columns joined together. Empty cells are skipped. Use join: to set the joining text (default is a single space).
column	column: 3	Column by position, counting from 1. Needed for header-less files.
columns	columns: [1, 2]	Several positional columns joined, as headers.
value	value: 'Broadband'	A constant for every row. The form value: {date: 'first of last month'} resolves a date expression (see Date Expressions).
source	source: filename	The uploaded file’s name.

Other per-field keys

Key	Meaning
optional	true suppresses the “expected column was not found” warning when the column may legitimately be absent.
default	Value substituted when the cell is empty.
translations	A mapping of raw value to replacement, matched case-insensitively on the trimmed cell, applied before conditions and transforms. Example: translations: {done: OK}.
skipRowValues	List of values (matched trimmed and case-insensitively) that cause the whole row to be skipped. Example: skipRowValues: ['-1', 'n/a'].
transforms	Ordered list of value transforms, applied after row rules are evaluated. See below.
format	Shorthand for a final date parse: format: '%d/%m/%Y' is a date transform with that pattern.
aka	Alternative name or names this field may be referred to by in conditions.

Processing order for each row: cells are read and translated; row rules and skipRowValues are evaluated against these translated values; then transforms run to produce the final values. Conditions therefore see values before transforms.

Decision-Only Columns and Temporary Fields

Not every column in a file is there to update the record. Three concepts cover columns that inform a decision instead:

• “(Information Only)” columns. A heading suffixed (Information Only) is context for whoever, or whatever, is working with the file; platform exports use the suffix heavily. The automatic recognition never maps or writes these columns. To use one in your rules, map it explicitly, normally to a temporary field (below). Mapping one to a real target field turns it back into an ordinary update column, so only do that deliberately.
• Temporary (temp.) fields. A field whose logical name does not belong to the target object is parsed, translated, transformed and visible to every condition, but is never written to the record. Prefer a temp. prefix so the intent is obvious: temp.contractEnd. Give the field an aka with the natural short name so conditions can refer to it by that rather than the internal name. Typical uses: skipping rows on a value the update does not carry, or comparing two columns with sameAs. Note that the misspelt-field warning only covers names on the target object (feature.xyz); any other prefix is quietly treated as temporary, which is another reason to use temp. deliberately rather than arrive at it by accident.
• Locator (“Existing”) columns. A record is found by its identifying field, so that column cannot normally carry a new value at the same time. A heading marked with Existing, Current, Old, Previous or Original, as a leading word (Existing Description) or a trailing parenthetical (Description (Current)), is used only to find the record, freeing the plain same-named column to carry the new value. The automatic recognition detects these markers on identifier and narrowing columns. In hand-written rules the same concept is the reserved __lookup suffix: feature.description__lookup locates and feature.description updates. The markers are ignored, with a warning, in add files: a new record has no existing value to find.

Skipping rows on an information-only column via a temporary field:

fields:
  temp.contractEnd:
    header: 'Contract End Date (Information Only)'
    aka: 'Contract End Date'
rowRules:
  skip:
    - name: Still in contract
      when:
        - {field: 'Contract End Date', present: true}

How the Overlay Merges

Your YAML is merged over the automatically derived configuration:

• fields: merge per field. An entry for a field the recognition already mapped keeps its source unless you declare one; you can add transforms, translations, skipRowValues and so on to a recognised column without repeating its header.
• An overlay’s transforms: replaces that field’s automatic transforms. Recognised date columns carry an automatic UK-date parse, so to change a date format, supply the full replacement: transforms: [{date: '%m/%d/%Y'}].
• Mapping an unrecognised header is enough on its own. A fields: entry naming a plain field of the target object, such as fields: {feature.site: {header: 'Location'}}, is written to the record exactly as if the header had been recognised. Date and datetime fields gain the automatic UK-date parse when you declare no transforms. Locators (__lookup names), the identifier a lookup matches on, id fields and parent foreign keys keep their locate-only roles and are never written. A name that looks like a target field but is not one (a typo) produces a warning on the file’s log.
• ignore: lists union. rowRules:, file: and lookupOptions: merge key by key, with your value winning per key; emit: merges per entry (see emit: below). lookups: is replaced wholesale if you supply it. version, kind and target never override the derived values.

Transforms

Each list entry is a single name: option pair. Names ignore spaces, hyphens, underscores and case, so remove suffix and removeSuffix are the same. They run in the order written.

Transform	Option	Effect
trim	true	Strip surrounding whitespace.
case	lower, upper or title	Change case.
remove prefix	text	Remove the prefix if present.
add prefix	text	Add the prefix unless already present.
remove suffix	text	Remove the suffix if present.
add suffix	text	Add the suffix unless already present.
remove	one of, or a list of: currency symbols, spaces, non digits, punctuation	Strip those characters. currency symbols removes £$€ and thousands commas.
number	normalise, normalise long, international, e164, remove44, or leading zero; or {mode: ..., defaultPrefix: ...}	Telephone-number normalisation.
duration	normalise (H:M:S or “2 hours 3 mins” forms to seconds) or minutes (minutes to seconds)	Duration conversion. Non-zero durations never round down to zero.
date	a pattern such as '%d/%m/%Y', or true for automatic parsing (ISO first, then day-first)	Parse to an ISO date. Empty, null, none and n/a become empty; unparseable values pass through unchanged for the test step to report.
datetime	as date	Parse keeping the time of day.
decimal	true	Strip currency symbols, commas and whitespace from a money value.
substitute	{pattern: regex, with: replacement}	Regular-expression replacement.
extract	regex	Keep the first captured group (or the whole match if no groups). No match leaves the value unchanged.
scale	number	Multiply a numeric value, for example scale: 60 to turn minutes into seconds or scale: 0.01 to turn pence into pounds.
round	nearest, up, down, or a number of decimal places	Numeric rounding.
pad	a length, or {length: N, with: text, side: left/right}	Pad a short value out to the length: pad: 11 restores leading zeros a spreadsheet stripped from a phone number. Defaults: pad with 0 on the left. Empty values stay empty.
dateAdjust	first of month, last of month, first of next month, first of previous month, or {days: N} / {months: N}	Adjust the field’s own date: snap it within its month, or offset it by days or months (month offsets clamp to the target month’s last day, and negative offsets go backwards). A value that does not hold a date passes through unchanged.

Conditions

Conditions are used in rowRules (and, for CDR rules, in several other places). A condition group is a list of conditions, any of which may match (OR). For AND, NOT, or to nest, use the explicit wrappers allOf: (every member matches), anyOf: (any member matches) and noneOf: (no member matches):

when:
  allOf:
    - {field: duration, equals: ['0']}
    - anyOf:
        - {field: band, prefix: [FREMA]}
        - {field: band, contains: [Inbound]}
    - noneOf:
        - {field: status, equals: [keep]}

Each condition names a field: and exactly one operator:

Operator	Argument	Matches when
equals	value or list	The trimmed value equals any listed value, ignoring case. A string comparison: equals: ['0'] does not match 0.00. Use the numeric operators for numbers.
prefix	value or list	The value starts with any listed value, ignoring case.
suffix	value or list	The value ends with any listed value, ignoring case.
contains	value or list	The value contains any listed value, ignoring case.
matches	regular expression	The regex matches anywhere in the value. This is the only case-sensitive operator.
over / under	number	Strict numeric comparison. A non-numeric value never matches.
atLeast / atMost	number	Inclusive numeric comparison (>= / <=). A non-numeric value never matches.
before / after	date, or a named date expression	The value holds a date strictly before or after the bound. Whole days are compared; a value that does not parse as a date never matches.
onOrBefore / onOrAfter	date, or a named date expression	The inclusive forms of before and after.
lengthOver / lengthUnder	integer	Comparison of the value’s length in characters.
sameAs	another field reference	The two fields’ values are equal, trimmed and ignoring case.
empty	true or false	true: the value is blank. false: it is not.
present	true or false	The opposite spelling of empty.

The date operators accept a fixed date (before: '2026-04-01') or a named expression from the Date Expressions section (before: first of this month). Relative expressions are anchored to the file’s upload date, not the moment of processing, so re-testing or re-generating an old file gives the same result as its original run.

Two special conditions take no field:: {always: true} matches every row, and {columnCountAbove: N} / {columnCountBelow: N} test how many columns the raw row has.

A field: reference (and the sameAs: argument) may be either the column heading as it appears in the file (Recurring Charge), a field’s aka name, or the internal field name (feature.serviceCharge), matched with the same normalisation as headers. Prefer the heading the operator sees in the file; the rules then read naturally to whoever reviews them. Long headings can be shortened by giving the field an aka and referencing that. Fall back to the internal name only when a reference would otherwise be ambiguous: a reference matching more than one field is a validation error naming the candidates. A reference that matches nothing reads as empty and produces a validation warning.

A condition can only see columns that are mapped to a field. “(Information Only)” columns are never mapped automatically, so a condition on one silently reads as empty; map each one you need to a temp. field first (see “Decision-Only Columns and Temporary Fields”).

rowRules: - Skipping, Stopping and Flagging

rowRules:
  endProcessing:
    when:
      - {field: 'Account Number', equals: ['Total']}
  processLine:
    when:
      - {field: 'Record Type', equals: ['DETAIL']}
  skip:
    - name: Internal traffic
      when:
        - {field: 'Charge Band', equals: ['Internal']}
      unless:
        - {field: 'Status', equals: ['keep']}
  warn:
    - name: Unusually large charge
      when:
        - {field: 'Recurring Charge', atLeast: 100}

Key	Meaning
endProcessing	Stop reading the file when a row matches; that row and everything after it is ignored. Use for totals footers.
processLine	Process only rows that match; skip everything else.
skip	A list of named skip groups. A row is skipped when a group’s when matches and its unless (optional) does not. The name appears verbatim in the user-visible log, so write it as a short plain description of the reason (“Recurring Charge is zero”), capitalised as it should read there, and give each distinct reason its own rule.
warn	The same {name, when, unless} shape as skip, but a matching row is flagged, not skipped: it still processes normally and is counted under “Rows flagged” in the run summary, with its line numbers. Only rows that survived every skip rule are checked. Use for rows worth a second look, such as unusually large amounts.

Skipped rows are counted per reason, with source line numbers, in the file’s log. They produce no per-row result. Rows that fail to match a record are different: they appear in the results grid with an error, so the operator can review them individually.

lookups:, lookupOptions: and emit: - Record Matching

The platform derives these automatically from the file’s columns and the chosen object type: which column identifies the customer, the number within the customer, the feature within the number, whether id columns identify records directly, and which columns narrow an ambiguous match. Prefer adding a heading via fields: and letting the automatic chain do the matching. Overriding lookups: replaces the whole derived chain.

For completeness, a lookup step looks like this:

lookups:
  - name: number
    object: number
    match:
      column: number          # the object's database field to match on
      using: number.number    # the logical field carrying the value
      caseInsensitive: true
    restrictedBy:
      - {parent: customer, foreignKey: customerID}
    onMissing: fail

Key	Meaning
name	How other steps and emit refer to this lookup.
object	The platform object to look up.
match.column	The object’s field to match against.
match.using	The logical field whose value is matched.
match.idUsing	A logical field carrying an explicit record id; when present and non-blank it wins over the name match.
match.caseInsensitive	Case-insensitive matching.
restrictedBy	Parent lookups this match is scoped within, as {parent: name, foreignKey: column}.
disambiguate	Columns used to narrow an ambiguous match, as {column: site, using: number.site__lookup, label: Site}.
onMissing	When the lookup does not resolve: fail (default) fails the row, skip skips it, ignore leaves the step unresolved, add turns an unmatched target row into an add (used by the tariff-rate objects, where an unknown rate is a new rate), and create creates the missing record at Apply, permission-checked, so a Test run creates nothing (used to create tariffs from carrier files). For fail/skip/ignore, a supplied value that matches nothing still fails the row.
lookup	Inline matching options, as lookupOptions below.

lookupOptions: is keyed by lookup name and passes options to the matchers, for example {number: {method: digits, multipleMatches: first active}} for the telephone-number matcher.

emit: controls the output row: values maps logical fields to the columns written ({feature.site: site}), targetID names the lookup whose match is the record to change, parentForeignKeys links new records to resolved parents, and captureRevision: true stamps the record’s revision for the stale-edit check.

emit: merges key by key with the derived configuration, so supplying only values keeps the derived targetID, parentForeignKeys and captureRevision. The values and parentForeignKeys maps themselves merge per entry: your entry wins for that logical field, and the derived entries are kept. Setting an entry to null removes the derived one, which is the way to have a column parsed and available to conditions but never written. Plain target-object fields you map in fields: join values automatically, so an explicit emit.values is only needed to redirect or suppress a column.

A field name with the reserved __lookup suffix (for example number.site__lookup) is a locator: it finds the record but is never written as a value. A locator with no transforms of its own inherits the base field’s transforms, so the value used to find a record stays in step with the value written.

Rate Files

Tariff rate and fixed fee tariff rate files (see Bulk Rate Updates for the operator workflow) route each row to the tariff named in a “Tariff Name” column, then locate the rate within it: a call tariff rate by call type (AKA-aware), call time and start date, a fixed fee rate by its applies-to lists compared as sets plus the start date. Their derived rate step carries onMissing: add, so a rate the tariff does not have becomes a new rate. Two optional behaviours are set with rules:

Prefer a specific carrier’s call-type aliases, and record unallocated alias entries for unknown destination names ready for mapping afterwards:

lookupOptions:
  callType:
    carrier: 'Carrier Name'
    addAKA: true

Create tariffs that do not exist yet by setting onMissing: create on the tariff step. Creation happens at Apply only and is permission-checked, so a Test run creates nothing. Supplying lookups: replaces the derived chain, so restate it in full:

lookups:
  - name: tariff
    object: tariff
    match: {column: name, using: tariff.name}
    onMissing: create
  - name: tariffDetail
    object: tariffDetail
    match: {column: callTypeID, using: tariffDetail.callTypeID}
    restrictedBy:
      - {parent: tariff, foreignKey: tariffID}
    disambiguate:
      - {column: callTimeID, using: tariffDetail.callTimeID, label: Call Time}
      - {column: startDate, using: tariffDetail.startDate, label: Start Date}
    onMissing: add

Date Expressions

Anywhere a date expression is accepted (value: {date: ...}, and the before/after/onOrBefore/onOrAfter operators), you may use an ISO or UK date, or a named expression: today, yesterday, tomorrow, first of this month, last of this month, first of last month, last of last month, first of 2 months ago. Named expressions are resolved against the file’s upload date, so a re-run gives the same answer as the original run.

Do not confuse these with the dateAdjust transform: a date expression produces a date relative to the upload date, while dateAdjust moves the date the field itself holds.

Errors, Warnings and the Skip Report

• Configuration errors stop the whole file with a message giving the YAML path. The “Validate rules” button reports these without uploading.
• Warnings do not stop the file: a missing expected column, a condition referencing an unknown field, and unrecognised file columns are reported on the file’s log for the operator to review.
• Row-level problems never stop the file. Unmatched or ambiguous rows appear in the results with a message; skipped rows are tallied by reason with line numbers.

Worked Examples

Map columns whose headings the platform does not recognise (a feature edit file with “Monthly Fee (ex VAT)” and “Start” columns; the customer and feature identifier columns use recognised names, so the record matching is automatic). The fields: mappings alone are enough; the mapped columns are written like recognised ones:

fields:
  feature.serviceCharge:
    header: 'Monthly Fee (ex VAT)'
    transforms:
      - remove: currency symbols
  feature.startDate:
    header: 'Start'
    transforms:
      - date: '%m/%d/%Y'
rowRules:
  skip:
    - name: Subtotal row
      when:
        - {field: 'Description', prefix: ['Subtotal']}
ignore: ['Internal Ref']

Clean a recognised charge column and drop unusable rows by value (the file writes “£12.34 pm”, or “n/a” when unknown). The column is already recognised, so no source or emit is needed:

fields:
  feature.serviceCharge:
    transforms:
      - remove suffix: ' pm'
      - decimal: true
      - round: 2
    skipRowValues: ['n/a', 'tbc']
    default: '0'

Translate a status vocabulary before matching, and stop at the totals footer:

fields:
  number.statusID:
    header: Status
    translations:
      live: Active
      ceased: Dropped
rowRules:
  endProcessing:
    when:
      - {field: 'Account Number', equals: ['Total', 'Grand Total']}

A header-less file (a number edit updating each number’s site). With no header row there is nothing to recognise automatically, so the ruleset must be complete: positional fields, the lookup chain, and the emit block:

file:
  header: false
  missingColumns: skip
fields:
  customer.accountNumber: {column: 1}
  number.number: {column: 2}
  number.site: {column: 3}
lookups:
  - name: customer
    object: customer
    match: {column: accountNumber, using: customer.accountNumber}
  - name: number
    object: number
    match: {column: number, using: number.number}
    restrictedBy:
      - {parent: customer, foreignKey: customerID}
emit:
  targetID: number
  captureRevision: true
rowRules:
  skip:
    - name: Row too short
      when:
        - {columnCountBelow: 3}

Build one value from two columns and add a constant (an add file creating numbers under each customer; the “Account Number” column is recognised, so the customer chain is automatic):

fields:
  number.number:
    headers: ['DDI 1', 'DDI 2']
    join: ''
  number.description:
    value: 'Broadband DDI'
ignore: ['Internal Ref']

Work only the rows due this month, flag doubtful ones instead of dropping them, and restore leading zeros a spreadsheet stripped (a number edit file; “Renewal Date” only informs the rules, so it is a temporary field):

fields:
  temp.renewalDate:
    header: 'Renewal Date'
  number.number:
    transforms:
      - pad: 11
rowRules:
  skip:
    - name: Already renewed
      when:
        - {field: 'Renewal Date', before: first of this month}
    - name: Not yet due
      when:
        - {field: 'Renewal Date', after: today}
  warn:
    - name: Unusually large charge
      when:
        - {field: 'Recurring Charge', atLeast: 100}
    - name: Number does not start with 0
      when:
        noneOf:
          - {field: 'Number', prefix: ['0']}
          - {field: 'Number', empty: true}

A real request from testing: “skip lines where the Recurring Charge is empty or 0, where the Number Type is India Toll Free, or where the calculated charge has a value but the explicit charge is empty” (an annual price increase file: the context columns are all “(Information Only)”, so each one used in a condition is mapped to a temporary field first, with an aka giving it its natural name). Note the zero test is numeric, so it catches 0, 0.0 and 0.00 alike, and every reason has its own operator-readable name:

version: 1
kind: bulk-action
fields:
  temp.numberType:
    header: 'Number Type (Information Only)'
    aka: 'Number Type'
  temp.calculatedCharge:
    header: 'Feature Calculated Recurring Charge (Information Only)'
    aka: 'Calculated Recurring Charge'
  temp.explicitCharge:
    header: 'Feature Explicit Recurring Charge (Information Only)'
    aka: 'Explicit Recurring Charge'
rowRules:
  skip:
    - name: Recurring Charge is empty
      when:
        - {field: 'Recurring Charge', empty: true}
    - name: Recurring Charge is zero
      when:
        allOf:
          - {field: 'Recurring Charge', atLeast: 0}
          - {field: 'Recurring Charge', atMost: 0}
    - name: India Toll Free number type
      when:
        - {field: 'Number Type', equals: ['India Toll Free']}
    - name: Calculated charge without an explicit charge
      when:
        allOf:
          - {field: 'Calculated Recurring Charge', present: true}
          - {field: 'Explicit Recurring Charge', empty: true}

Related Pages

• Bulk Update and Bulk Action Lists: the workflow these rules plug into, including validation and saved rulesets
• CDR Processing Rules: the same language applied to carrier call data, with the CDR-specific keys
• Bulk Edit and Bulk Add: the CSV conventions for platform-exported files, which rarely need rules