CDR Processing Rules

Overview

Every carrier sends call data in its own format: different columns, different date styles, different ways of marking call direction. Processing rules tell the platform how to read each carrier’s files, so the calls inside them can be rated and billed.

Each carrier has a set of processing rules per file type (daily CDRs, monthly CDRs, fixed fee files). The rules cover:

• Recognising files as they arrive: which carrier and billing period a file belongs to, and whether it should be billed, held or rejected
• Reading the file: encoding, delimiters, header rows and junk lines
• Mapping columns to call fields such as the number, date, duration, charge band and cost
• Producing call records: some files describe two call legs per row, or need calls split by direction

Processing rules are platform configuration, managed by your administrator or the support team. They are not a self-service screen: if a carrier changes its file format, or you add a new carrier, support will set up or adjust the rules. This page explains how they behave, and documents the rules language itself.

Two Formats: Legacy and YAML

The platform supports two rules formats, and each carrier ruleset uses one or the other:

• A legacy format used by long-standing carrier configurations. It still works unchanged.
• The YAML rules language documented here, shared with Bulk Update rules. New rulesets use YAML, and legacy rulesets are migrated one at a time. A converter produces a draft YAML ruleset from a legacy one, together with a review report listing anything that needs a human decision.

The platform detects the format automatically, so migration is gradual and per carrier.

File Recognition and Safety Holds

Before a file is read, the recognise: rules classify it:

• Monthly, daily or reject rules match on the file’s name, path or content. A rejected file is marked Rejected and not processed. The billing setting decides whether a matched file feeds billing, statistics or is held for review.
• Required headers protect you from silent format changes. If a supplier renames or drops an expected column, the file is Held rather than half-processed, and the queue report says why. This stops a quiet supplier-side change producing wrong bills.
• Duplicate handling controls what happens when a file arrives with a familiar name or identical contents: import, ignore, hold or reject. See Potential Duplicate Files for the operator’s view.

After changing recognition rules, re-recognise pending files to apply them to files already received. Review each file’s Processing Log and resulting status.

---

The Rules Language for CDR Files

The rest of this page is a reference for the YAML rules language as it applies to CDR and fixed fee files. It is written so that an AI assistant can draft a ruleset from a sample file, given the file’s header row and a few sample rows.

Working with an AI assistant? Give it the plain-Markdown version of this page: /telecommunications/cdr-processing-rules/agents.md. Ask the assistant to fetch that address, or if it cannot fetch pages, download the file and attach it to your conversation. Both are more reliable than copying and pasting this web page.

For the AI assistant: you are writing a kind: cdr (or kind: transactions) YAML ruleset for the SAFE Billing Platform’s CDR import engine. The shared language basics are identical to the Update Rules Reference: file: framing, fields: with sources (header, headers, column, value, source: filename), translations, skipRowValues, transforms, rowRules (including warn groups, which flag rows in the processing report without skipping them), ignore, and the condition grammar (anyOf/allOf/noneOf; operators equals, prefix, suffix, contains, matches, over, under, atLeast, atMost, before, after, onOrBefore, onOrAfter, lengthOver, lengthUnder, sameAs, empty, present, always, columnCountAbove, columnCountBelow). The date operators take a fixed date or a named expression such as first of last month, anchored to the file’s upload date. Validation is strict: unknown keys are errors, so never invent keys. version: and kind: are optional; if the ruleset is for fixed fee (carrier transaction) files use kind: transactions, otherwise cdr.

What differs for CDR rules:

1. Field names are call fields, and you may use their friendly spellings: Date, Time, Duration, Target (the billed number), Called Number, Calling Number, Charge Band, Call Type, Cost, Extension, and so on. A field name the call record does not use, such as Temporary Field 1 or any temp.-prefixed name, is scratch space: you can read, transform and test it in conditions, but it is not written to the call.
2. records: produces the output calls. Without it, each row produces one call. With it, one row can produce several call legs, each with its own direction and column bindings. A condition can take an optional record: beside field: to read a field as another record binds it.
3. flags:, settings: and recognise: are available; lookups: and emit: are not (they belong to bulk update rules).

records: - One Row, Several Calls

Each entry describes one output record. All shared fields: apply to every record; a record can override or add its own.

Key	Meaning
name	Required, unique. Used in the drop report.
prefix	Column-prefix sugar: with prefix: A, a shared field bound to header Number also binds to a column headed A Number for this record. This is the usual way to map per-leg columns.
fields	Per-record field overrides. An override that declares a source replaces the shared one; an override with only transforms keeps the shared source.
direction	inbound, outbound, aleg, bleg, neither, determine, or determine b leg. The determine forms compare the target against the called and calling numbers.
directionWhen	A list of {direction: ..., when: <conditions>} entries; the first match wins over the static direction.
when	Emit this record only when the conditions match.
skip	Drop this record when the conditions match (the row’s other records still emit).
formatted	{when: ...} and/or {skip: ...} evaluated against the final values just before insert, after transforms.
useAlternate	Decide a field’s value from conditions on the row: Charge Band: {from: Alternate Charge Band, when: [...]}, or a list of cases. See below.
surcharge	{when: <conditions>}: marks the record as a surcharge leg; the platform derives the surcharge charge band from the call’s origin and destination.

Records that are declared but not emitted are counted per reason in the processing report, so nothing is dropped silently.

Which leg a condition reads

A condition reads a field as the scope it sits in binds it. Inside a record that is the record’s own binding: the column its prefix: points at, or its own entry in fields:. In rowRules:, which runs once for the whole line, it is the shared fields: block alone.

That leaves one thing out of reach: a gate on another leg’s column. A condition on Duration inside the B record can only mean the B leg’s duration, however the file spells it.

A condition can name the record whose binding to read, with record: alongside field::

records:
  - name: aLeg
    prefix: A
    direction: aLeg
    fields:
      Duration: {header: 'A Seconds'}
  - name: bLeg
    prefix: B
    direction: bLeg
    fields:
      Duration: {header: 'B Seconds'}
      Called Number: {header: 'B Called'}
    skip:
      - {field: Duration, equals: ['-1']}
      - {field: Duration, record: aLeg, equals: ['-1']}
rowRules:
  skip:
    - name: Message Direct
      when:
        - {field: 'Called Number', record: bLeg, prefix: ['01223']}

The B leg is dropped when its own duration is -1, and again when the A leg’s is. The row-level skip reaches a column only the B record maps, which a rule at that level could not otherwise test.

A field the shared fields: block already binds needs no qualifier, and a qualifier on one is harmless: it resolves to the shared field. Naming a record or a field that does not exist is an error rather than the “reads as empty” warning an unqualified typo gets, because a qualifier is a statement about where the field lives.

Only the parse-stage keys travel with a qualified field: its source, join, default, translations and translationsFrom. Transforms and formatting belong to a later stage that conditions never see.

Legacy rulesets spelled these gates with their leg-prefixed column names, and the converter used to drop them silently. A conversion now emits record: wherever the legacy rule tested another leg’s column, so the converted ruleset skips the rows the legacy engine really skipped. Rulesets converted before this are unchanged until they are converted again.

Deciding a value with useAlternate

useAlternate sets a field’s value from conditions on the row. In its simplest form it takes one alternative:

useAlternate:
  Charge Band: {from: Alternate Charge Band, when: [{field: Service, equals: CPS}]}

It also takes a list of cases, and a case can state a literal value: instead of copying from: another column. Values can be date expressions:

useAlternate:
  Charge Band:
    - {value: Mobile, when: [{field: Service, equals: MOB}]}
    - {from: Alternate Charge Band, when: [{field: Service, equals: CPS}]}
  Billing Period:
    - {value: first of last month, when: [{field: Period, empty: true}]}

The first matching when: wins. Where none match, the field keeps whatever its own source set.

Put value logic here rather than in a second record. A record declares an output row, so using one per case emits the line twice wherever the cases overlap, and those duplicate rows share their identifiers and mis-pair at leg matching. useAlternate changes the value without changing how many rows the line produces.

flags: - Per-Call Booleans

Each flag is {when: <conditions>}, evaluated per record. Declared flags are set true or false; undeclared flags are left for the platform to work out where it can.

Flag conditions are written against the shared fields: block, but evaluated for each record in turn, so a shared field the record overrides reads that record’s value. To test one leg’s column whichever record is being evaluated, name it with record:.

Available flags: inbound, outbound, dataCall, mobileOrigination, payphoneOrigination, callRecording, sms, surcharge, numberWithheld.

flags:
  mobileOrigination:
    when:
      - {field: 'Call Type', equals: ['M', 'MOB']}

settings: - File-Wide Behaviour

Key	Meaning
autoMapHeaders	true binds any column whose heading matches the platform’s built-in call field names automatically. Explicit fields: always win. Most rulesets set this and map only the exceptions.
cost	{scale: 100, decimalSeparator: ','}: divide costs supplied in pence, or fix a decimal comma.
numbers	{mode: normalise} (also international, e164, remove44): normalisation applied to the call’s number fields.
dates	{date: '%d/%m/%Y', time: '%H:%M:%S', datetime: ...}: the file’s date and time patterns.
data	Data-call handling: {round: ..., scale: ..., useDuration: true} treats a data volume as the duration.
normalise	Per number-role fine tuning (billed, origination, termination, called): lookup order and minimum lengths.
generateCallIDs	default, per-line or grouped-lines, for files without a usable unique call id.
skipOldCallsBefore	A date expression; calls starting before it are dropped and reported.
timezone	The file’s timezone, where it differs from the platform’s.
processAfter	{fileTypeIDs: [12], carrierID: 34}: hold this ruleset’s files back until the named file types have been processed. carrierID defaults to the file’s own carrier.

Waiting for another file

Some carriers split a call across two files, so a file loaded on its own arrives with columns it cannot fill. processAfter names the file types this one depends on:

settings:
  processAfter:
    fileTypeIDs: [12]

While a file of a named type is outstanding - unprocessed, held, failed, or flagged as a possible duplicate - this file waits. It stays visible in the pending list rather than loading with blank columns, and it is picked up in the same processing pass once its dependency has been handled, so one press of Process CDR Files deals with both. Where no matching file is outstanding, nothing waits.

backfill: - Filling Fields from Other Rows

Some carriers supply information a row needs in a different file, or in a different row of the same file. backfill: fills those fields in after the file’s rows are loaded but before allocation and rating, so the calls reach billing complete.

It is a top-level block in a CDR ruleset, holding a list of steps:

backfill:
  - name: outbound targets from inbound legs
    from: previousFiles
    source:
      fileTypeIDs: [1]
      recentBillruns: 2
    match:
      - field: callGroup
      - {field: aLegCarrierID, sourceField: carrierCallID}
    when: [{field: target, empty: true}]
    nearest: {withinMinutes: 10}
    set:
      target: {from: target, ifEmpty: true}

Key	Meaning
name	Names the step in the processing report.
from	previousFiles to read already-processed calls, or thisFile to read other rows of the same import.
source	Which rows to consider. fileTypeIDs (required for previousFiles), carrierID (defaults to the file’s own carrier), recentBillruns (defaults to 2) and when conditions.
match	The equality keys joining a source row to an import row. field alone uses the same column on both sides; add sourceField where the columns are named differently, or sourcePart: {delimiter, index} to match on one segment of a column.
when	Which import rows the step updates.
nearest	{withinMinutes: N}: bound the match by time, and use closeness in time to break ties.
preferNonEmpty	A secondary tie-break, preferring a source row that has a value in the named field.
set	The fields to write: target: callGroup copies a column, target: {from: target, ifEmpty: true} fills only where the file supplied nothing.

Conditions use the same shape as elsewhere in the rules, but only the operators that translate into a database query are available: equals, prefix, suffix, contains, empty and present, with the same part: {delimiter, index} modifier. Fields can be named by any of their usual spellings, so Charge Band works as readily as chargeBand.

Steps read and write the text columns of a call. Typed columns such as dates, costs and flags, and the results of allocation, are deliberately out of reach.

A step can read and write aLegCarrierID, which is what lets a carrier delivering its A and B legs in separate files be loaded from rules: each leg keeps its own call identifier while the B leg picks up the billed number, call reference and A-leg identifier it needs for leg matching later.

Each step reports how many rows it updated in the processing report.

Note

backfill: does not apply where module.file sends the file to a bespoke class. Either remove the module dispatch, or put the equivalent logic in the class.

recognise: - Classifying Arriving Files

Key	Meaning
monthly / daily / reject	Match blocks with filename, exceptFilename, path, exceptPath (regular expressions), headerMatches (regex tested against the first lines), contentMatches: {line: N, pattern: ...}, and billing (billing, stats, hold, manual or none).
requiredHeaders	Column headings that must be present, or the file is held for review.
duplicates	{filename: ..., contents: ...} each one of import, ignore, reject, hold; plus minimumSize in bytes.
housekeeping	Retention of processed files, for example {dailyFiles: delete}.
filenames	Permitted and blocked filename lists; see below.

Filename allow and deny lists

Where a carrier puts the billed number in the filename, a supplier typo can bill a whole file against a number nobody owns. filenames: checks the name against permitted and blocked lists as soon as the ruleset recognises the file, before any of it reaches billing.

Key	Meaning
extract	A regular expression picking the significant part of the name, for example the billed number out of 01612345678_202607.csv.
allowed / denied	The permitted and blocked lists, written in the ruleset.
allowedFrom / deniedFrom	The same lists held in a platform setting instead.
match	regex where names vary; entries otherwise match exactly and case-insensitively.
onUnlisted / onDenied	What to do with a file that is not on the permitted list, or is on the blocked list: hold it, or reject it.

Holding keeps the file against its carrier and file type so it can be reviewed, corrected and reset. Rejecting drops it with a line in the import report. A file whose name cannot be checked at all is held rather than admitted.

Worked Example: Two Call Legs with Direction Logic

One row per call, with A-prefixed and B-prefixed columns for the two legs. The B leg is only produced when its column is filled, and an alternate charge band is used for CPS and WLR calls:

kind: cdr
settings:
  autoMapHeaders: true
  cost:
    scale: 100
  numbers:
    mode: normalise
  dates:
    date: '%d/%m/%Y'
fields:
  Duration:
    header: Call Length
    transforms:
      - duration: normalise
  Call Status:
    header: Status
    translations:
      OK: answered
      ENGAGED: busy
rowRules:
  skip:
    - name: Internal calls
      when:
        - {field: 'Charge Band', equals: ['Internal', '']}
records:
  - name: aLeg
    prefix: A
    fields:
      Target:
        header: Caller
    directionWhen:
      - direction: inbound
        when:
          - {field: 'Charge Band', prefix: ['IN']}
    useAlternate:
      Charge Band:
        from: Alternate Charge Band
        when:
          - {field: 'Charge Band', equals: ['CPS', 'WLR']}
  - name: bLeg
    prefix: B
    when:
      - {field: 'B Number', present: true}
    fields:
      Target:
        header: Called

Worked Example: Header-less File with Conditional Direction

kind: cdr
file:
  header: false
fields:
  Call Type: {column: 1}
  Target: {column: 2}
records:
  - name: leg
    direction: outbound
    directionWhen:
      - direction: inbound
        when:
          - {field: 'Call Type', equals: ['I', 'A']}

Fixed Fee Files (kind: transactions)

Fixed fee (SDR) files use the same language with kind: transactions. The field roster is the carrier transaction’s fields, including the number references used to match each charge to a customer, number or feature, and the transaction value, which respects settings.cost. The records:, flags: and recognise: keys work the same way. See Fixed Fee Billing Files for how these files are matched and billed.

---

Externally Maintained Lists

Some of the values a ruleset works from change more often than the ruleset itself: a customer’s number added to a permitted filename list, or a supplier’s new spelling of a charge band. Those lists can live in platform settings instead, so they can be maintained by someone who does not write rulesets and without revalidating one.

Two settings hold them, both per carrier and file type under CDR Processing Settings, and both needing elevated access:

Setting	Holds
CDR rule value lists	Lists of values, one per line
CDR rule translations	Value translations, one Old Name:New Name per line

Group entries into sections with a [Section Name] heading. You choose the section names to suit the ruleset that uses them. Lines before any heading form the unnamed section, which a ruleset refers to as parameter. Blank lines are ignored, and a line beginning with # is a comment.

[Permitted Numbers]
01612345678
01613334444

[Charge Bands]
Mobile Term:Mobile
MOB-TERM:Mobile

A ruleset names the section it wants:

Rule key	Reads from
recognise.filenames.allowedFrom	A section of CDR rule value lists
recognise.filenames.deniedFrom	A section of CDR rule value lists
A field’s skipRowValuesFrom	A section of CDR rule value lists
A field’s translationsFrom	A section of CDR rule translations

Where a ruleset has its own entries as well, filename lists and skipRowValues combine the two. Translations written in the ruleset take precedence over the setting. A section the setting does not contain is named in the run log, rather than being treated as an empty list.

Related Pages

• Supply Files: how CDR and fixed fee files reach the platform, duplicates and bundles
• CDR Processing: the end-to-end processing workflow
• Call and CDR Configuration: call types, overrides and the CDR file records
• Update Rules Reference: the shared rules language in full, applied to bulk update files