Balanced offset records sum a File's Debits or Credits to zero in a single EntryDetail within a Batch. These are used to offset transactions within an ODFI.

Issue: https://github.com/moov-io/ach/issues/531

(The original issue was https://github.com/moov-io/ach/issues/236.)

TraceNumber (and likely other fields) have known edge cases and bugs.

• TraceNumber is a 15 digit value whose prefix is an ABA routing number.
  • These ABA routing numbers often have a leading zero.
• int in Go is only defined: "int is a signed integer type that is at least 32 bits in size"

We should change this type to be a string (perhaps a TraceNumber type) to prevent both bugs. This is a breaking change however, so we should release it sooner rather than later.

We should also scan the other fields (int types) for similar problems.

I would like to help out adding more batch types, but based on the way the library is currently written I am not sure how you were planning on adding support for batch types other than PPD.

Can you point me in the right direction?

Moov ACH exposes a RESTful endpoint to create ach files. We should supply a curl or postman based example of interacting with the API for file creation.

Ports the human-readable formatting from the command line achcli to the webui.

Note: It might be worth changing the input and output textareas to vertical rather than horizontal to accommodate the wrapping that can happen with the human-readable content.

cc @adamdecaf

ACH Version: v1.15.0

What were you trying to do? Add an offset to a return batch. This was done by calling

batch.WithOffset(&ach.Offset{
	RoutingNumber: "DUMMY",
	AccountNumber: "DUMMY",
	AccountType: "checking",
	Description: "S",
})
batch.Create()

What did you expect to see? Successful creation

What did you see? Error:

batch #1 (WEB) Addenda99 is a mandatory field and has a default value

While debugging and stepping through the code, we see that the error is being thrown by the OFFSET entry that gets added during the build step.

The offset entry is created internally as a last step in the build method that is called by batch.Create() and is given the same category as the first entry in the batch, which in this case is CategoryReturn. However, since it's an OFFSET record, it doesn't get an Addenda99 record.

When Validate() gets called on the batch, it loops through all the entries (including the OFFSET record) and errors out on the addendaFieldInclusionReturn when validating the OFFSET record.

How can we reproduce the problem? Very simple example with dummy data.

package main

import (
	"fmt"
	"time"

	"github.com/moov-io/ach"
)

func main() {
	// Batch Header
	batchHeader := ach.NewBatchHeader()
	batchHeader.ServiceClassCode = 200
	batchHeader.CompanyName = "COMPANY"
	batchHeader.CompanyIdentification = "IDENTIFICATION"
	batchHeader.StandardEntryClassCode = ach.WEB
	batchHeader.CompanyEntryDescription = "DESCRIPTION"
	batchHeader.CompanyDescriptiveDate = time.Now().Format("060102")
	batchHeader.EffectiveEntryDate = time.Now().Add(time.Hour * 24).Format("060102")
	batchHeader.ODFIIdentification = "1"

	// Addenda99
	addenda99 := ach.NewAddenda99()
	addenda99.ReturnCode = "R07"
	addenda99.OriginalTrace = "99912340000015"
	addenda99.AddendaInformation = "Authorization Revoked"
	addenda99.OriginalDFI = "9101298"

	// Entry
	entry := ach.NewEntryDetail()
	entry.TransactionCode = 22
	entry.RDFIIdentification = "1"
	entry.CheckDigit = "3"
	entry.DFIAccountNumber = "DUMMY"
	entry.Amount = 1000
	entry.IndividualName = "DOE, JOHN"
	entry.Category = ach.CategoryReturn
	entry.Addenda99 = addenda99
	entry.AddendaRecordIndicator = 1

	// Batches
	batch1 := ach.NewBatchWEB(batchHeader)
	batch1.AddEntry(entry)

	// Prints nil
	fmt.Printf("batch1 error: %v\n", batch1.Create())

	batch2 := ach.NewBatchWEB(batchHeader)
	batch2.AddEntry(entry)
	batch2.WithOffset(&ach.Offset{
		RoutingNumber: "100000007",
		AccountNumber: "DUMMY",
		AccountType:   "checking",
		Description:   "S",
	})

	// Prints error
	fmt.Printf("batch2 error: %v\n", batch2.Create())
}

Go playground link: https://go.dev/play/p/EwWCywnMq_1

NACHA files are limited to 10,000 lines. But when creating file with localhost:8080/files/create , then result that view with localhost:8080/files/{fileID}/contents can be longer than 10,000 lines.

moovach

Converts a stream of ACH input data (TXT, JSON, CSV) to a designated output format.

Installation

pre-built binaries are available under releases.

If you have a working golang install, you can use go get.

go get github.com/moov-io/ach/cmd/ach

Usage

usage: moovach
    -i /path/to/input.json (optional; default is stdin)
    -o /path/to/output.csv (optional; default is stdout)
    // -b build a valid ach file from the input stream // TODO: see comments below
    -f [TXT,JSON,CSV] defines format of input and output
    --version
    -h This help

To convert:

{"files":[{"id":"fmucllmgqkaxcjowdoko","fileHeader":{"id":"fmucllmgqkaxcjowdoko","immediateDestination":"231380104","immediateOrigin":"121042882","fileCreationDate":"2018-10-08T00:00:00Z","fileCreationTime":"2018-10-08T00:00:00Z","fileIDModifier":"A","immediateDestinationName":"Federal Reserve Bank","immediateOriginName":"My Bank Name"},"batches":[{"batchHeader":{"id":"fmucllmgqkaxcjowdoko","serviceClassCode":200,"companyName":"Your Company, Inc","companyIdentification":"121042882","standardEntryClassCode":"PPD","companyEntryDescription":"Online Order","effectiveEntryDate":"2018-10-09T00:00:00Z","ODFIIdentification":"12104288","batchNumber":1},"entryDetails":[{"id":"fmucllmgqkaxcjowdoko","transactionCode":22,"RDFIIdentification":"23138010","checkDigit":"4","DFIAccountNumber":"81967038518      ","amount":100000,"identificationNumber":"#83738AB#      ","individualName":"Steven Tander         ","discretionaryData":"  ","addendaRecordIndicator":1,"traceNumber":121042880000001,"category":"Forward"}],"batchControl":{"id":"fmucllmgqkaxcjowdoko","serviceClassCode":200,"entryAddendaÇount":0,"entryHash":0,"totalDebit":0,"totalCredit":0,"companyIdentification":"121042882","ODFIIdentification":"12104288","batchNumber":1}}],"IATBatches":null,"fileControl":{"id":"","batchCount":1,"blockCount":1,"entryAddendaCount":0,"entryHash":0,"totalDebit":0,"totalCredit":0},"NotificationOfChange":null,"ReturnEntries":null}],"error":null}

to:

101 231380104 1210428821810080000A094101Federal Reserve Bank   My Bank Name                   
200Your Company, In                    121042882 PPDOnline Ord      181009   0121042880000001
2223138010481967038518      0000100000#83738AB#      Steven Tander           1121042880000001
2000000000000000000000000000000000000000000121042882                          121042880000001
9000001000001000000000000000000000000000000000000000000                                       
9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999

you would either

ach  -i achfile.json -o achfile.txt

or

cat achfile.json | ach -f TXT > achfile.txt

These changes overhaul the current Github Pages ACH documentation site by applying a customized Jekyll theme and reorganizing existing documentation. The theme is located at https://github.com/nlakritz/bulma-clean-theme.

Project settings under GitHub Pages need to be set to master and the docs folder to publish the site successfully at https://moov-io.github.io/ach/. Additionally, we’ll need to update the url in _config.yml from https://nlakritz.github.io to https://moov-io.github.io before merging. The site is currently live at https://nlakritz.github.io/ach.

Improvements to consider for the future:

• Embed API documentation
• Embed ACH demo
• Link to community site when it’s up

I wanted to see a same-day ACH file creation example and documentation. I also want to ensure that our library supports same-day ACH.

The use case example and file format is identified in this update from NACHA

Same Day ACH: FAQ

Our API docs currently state (as part of POST /files/create's response):

{
  "fileCreationDate": "102318",
  "fileCreationTime": "1601"
}

Source: https://api.moov.io/#operation/addFile

However, this format might be annoying for folks. Should we support RFC 3339 and YYMMDD / HHMM?

Our api returns RFC 3339 now, which is Go's default).

$ curl -s -H "cookie: moov_auth=..." https://api.moov.io/v1/ach/files/docrlxnuinuoouhilfws | jq '.file.fileHeader' | grep fileCreation
  "fileCreationDate": "2018-10-08T00:00:00Z",
  "fileCreationTime": "2018-10-08T00:00:00Z", 

Examples:

• https://github.com/moov-io/ach/blob/master/fileHeader.go#L48
• https://github.com/moov-io/ach/blob/master/addenda99.go#L151
• https://github.com/moov-io/ach/blob/master/iatBatchHeader.go#L128

This PR contains the following updates:

| Package | Type | Update | Change | |---|---|---|---| | github.com/aws/aws-lambda-go | require | patch | v1.36.0 -> v1.36.1 | | github.com/mattn/go-isatty | require | patch | v0.0.16 -> v0.0.17 |

Release Notes

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

• [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.

ValidateOpts has a lot of options To add all of the options using cli flags is uncomfortable, because cli has a lot of flags

Added one flag that specify validation opts config file

#achcli --help

achcli is a tool for displaying Nacha formatted ACH files in a human readable format.

USAGE
   achcli [-mask] [-pretty] path/to/file.ach

EXAMPLES
  achcli -diff first.ach second.ach    Show the difference between two ACH files
  achcli -mask file.ach                Print file details with personally identifiable information partially removed
  achcli -reformat=json first.ach      Convert an incoming ACH file into another format (options: ach, json)        
  achcli -version                      Print the version of achcli (Example: v1.26.4)
  achcli 20060102.ach                  Summarize an ACH file for human readability

FLAGS
  -diff
        Compare two files against each other

  -flatten
        Flatten batches in each file

  -mask
        Mask/hide full account numbers and individual names
  -mask.accounts
        Mask/hide full account numbers
  -mask.corrections
        Mask/Hide Corrected Data in Addenda98 records      
  -mask.names
        Mask/hide full individual names

  -merge
        Merge files before describing

  -pretty
        Display all values in their human readable format  
  -pretty.amounts
        Display human readable amounts instead of exact values

  -reformat string
        Reformat an incoming ACH file to another format

  -v    Print verbose details about each ACH file

  -validate string
        Path to config file in json format to enable validation opts
  -validate.allowInvalidCheckDigit
        Allow the CheckDigit field in EntryDetail to differ from the expected calculation
  -validate.allowMissingFileControl
        Allow a file to be read without a FileControl record
  -validate.allowMissingFileHeader
        Allow a file to be read without a FileHeader record
  -validate.allowUnorderedBatchNumbers
        Allow a file to be read with unordered batch numbers
  -validate.allowZeroBatches
        Allow the file to have zero batches
  -validate.bypassCompanyIdentificationMatch
        Allow batches in which the Company Identification field in the batch header and control do not match
  -validate.bypassDestinationValidation
        Skip validation for the ImmediateDestination file header field
  -validate.bypassOriginValidation
        Skip validation for the ImmediateOrigin file header field
  -validate.customReturnCodes
        Skip validation for the Return Code field in an Addenda99
  -validate.customTraceNumbers
        Disable Nacha specified checks of TraceNumbers
  -validate.requireABAOrigin
        Enable routing number validation over the ImmediateOrigin file header field
  -validate.unequalAddendaCounts
        Skip checking that Addenda Count fields match their expected and computed values
  -validate.unequalServiceClassCode
        Skip equality checks for the ServiceClassCode in each pair of BatchHeader

  -version
        Print moov-io/ach cli version

if use invalid config file, display example config file

#achcli -validate=unknown sample.ach

Specify validation config file in json foramt to enable valiation opts.

EXAMPLE:
{
  "requireABAOrigin": false,
  "bypassOriginValidation": false,
  "bypassDestinationValidation": false,
  "customTraceNumbers": false,
  "allowZeroBatches": false,
  "allowMissingFileHeader": false,
  "allowMissingFileControl": false,
  "bypassCompanyIdentificationMatch": false,
  "customReturnCodes": false,
  "unequalServiceClassCode": false,
  "allowUnorderedBatchNumbers": false,
  "allowInvalidCheckDigit": false,
  "unequalAddendaCounts": false
}

With error flags / wrapping we should validate fields according to these requirements instead of what we currently have. Often we have to relax validations (see #1150 as an example) when our validations aren't aware of Required fields.

ACH Version

v1.26.1

What were you trying to do?

I would like to be able to parse an ACH file without running any validations on the records. Often files are so badly formatted that working through validation errors causes a lot of overhead.

What did you expect to see?

A ValidateOpts option to disable all validations.

ACH Version

1.18

What were you trying to do?

I am trying to adapt the reader implementation to allow for a streamed approach that could keep from having a whole file in memory at once. The use case is that in large-volume applications with a service-based architecture reading a whole file into memory at once may not be possible. This would make some validations impossible of course and that is a trade-off consumers would need to make.

What did you expect to see?

// new signature proposed
func (r *Reader) Stream(ctx context.Context) (chan interface{}, <-chan error) { ... }
// Usage sample:

achReader := moov_ach.NewReader(reader)

opts := &moov_ach.ValidateOpts{
	// ...
}
achReader.SetValidation(opts)

fileRecords, errs := Stream(ctx)

var moreRecord, moreErr bool = true, true
var record interface{}
var err error
select {
case record, moreRecord = <-fileRecords:
	if header, ok := record.(moov_ach.FileHeader); ok {
		// handle header
	} else if batchHeader, ok := record.(moov_ach.BatchHeader); ok {
		// handle batch header
	} else if entry, ok := record.(moov_ach.EntryDetail); ok {
		// handle entry record
	} else if record, ok := record.(moov_ach.RecordType); ok {
		// handle other records
	}
case err, moreErr = <-errs:
	// process err
case <-ctx.Done():
	return ctx.Err()
}