Describe the bug I noticed when I benchmarked ledger, I sometimes would get a 200 response with no content in the response body, but the ledger logs would indicate: internal errors executing script: conflict error on reference.

After doing some digging, the true cause is duplicate key value violates unique constraint "transactions_id_key" when inserting into transactions table (possibly same issue could be present with other tables too).

The issue happens becomes you seem to be incrementing the ID in code, and not by using a db auto increment feature. Since there is no locking around this this kind of conflict is going to arise sooner or later when handling concurrent requests.

To Reproduce Steps to reproduce the behavior: Just have a script/benchmarking tool do a few requests to ledger at the same time.

For instance, I use Bombardier:

bombardier -c 50 -m POST -f bench.json -d 10s http://localhost:3068/example/script

Where bug.json can be any script posting, but for instance:

{"plain":"vars {\n  monetary $money_authorized\n  account $destination\n}\n\nsend $money_authorized (\n  source = @world\n  // @user:c36a3385-d94e-485d-9ccd-3682507d0fc2:authorized:b394c68d-0e4b-4e09-a634-3449d054edd5\n  destination = 
$destination\n)","vars":{"money_authorized":{"amount":2,"asset":"EUR/2"},"destination":"user:bf11c59beaca47cda9d477cd62181e55:authorized:47aa2ce0db61498bb02d9f0ca2728425"}}

Expected behavior I expect a bunch of things:

1. Ledger should be able to handle this case, worst case by locking when incrementing the id. But perhaps a better solution would be to use UUIDs as IDs in the database OR use the database auto increment feature
2. For such kind of fatal issues that do still occur, he HTTP request should not yield a 200 OK response with an empty body, but probably a 500 error

Environment:

• Numary Version 1.8.0

Add generator config to generate Ledger java client

I’ve tried to mimic what you did for the SDKs for other languages. There’s a few open things though which are probably best for Formance to pick up.

• [ ] Formance needs to define release process (.github/workflows/release.yml.mustache) + create repo
• [ ] Formance needs to define licenseName/licenseUrl (in additionalProperties)
• [ ] Please define servers url in swagger.json so that when generating not the hardcoded “localhost” is used. This needs to be fixed before the SDK can really be used by others. Please see https://stackoverflow.com/a/72182063/1422070 for a good approach I believe
• [ ] For some reason, a few classes are named like GetAccount200Response by default. I use inlineSchemaNameMappings setting to manually override this to a normal name (like GetAccountResponse). There are also classes for error status codes like GetAccount400Response, but although these are generated they are not used by other classes, so I don’t bother with renaming those at this time. But perhaps we can skip generation somehow?
• [ ] Tests are generated but they don’t actually do anything. I can define some manual tests, just not sure what your strategy is here. Just define them manually and include them in the final repo? It seems that’s what you do for the Python SDK

Type of change

• [ ] Bug fix (non-breaking change which fixes an issue)
• [X] New feature (non-breaking change which adds functionality)
• [ ] Refactoring / Technical debt

What parts of the code are impacted ?

No direct impact, since the github workflows still need to be modified to call the generation task for Java.

Checklist:

• [X] My code follows the style guidelines of this project
• [X] I have performed a self-review of my code
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] My changes generate no new warnings
• [ ] I have added tests that prove my fix is effective or that my feature works
• [X] New and existing unit tests pass locally with my changes

Hi there,

i tried displaying the UI right now and it comes up with this:

there is data in the quickstart ledger: {"cursor":{"page_size":15,"has_more":false,"total":3,"remaning_results":0,"data":[{"address":"world","contract":"default"},{"address":"users:001","contract":"default"},{"address":"central_bank","contract":"default"}]},"ok":true}

Is your feature request related to a problem? Please describe. A way to represent negative amounts, i.e. deficiency of funds.

Summary If the ledger is used to mirror real world accounts (shortage of) assets, it should support negatives.

Solution proposal A "limit" attribute on accounts. The transaction engine would guarantee I don't exceed that limit (just like it currently guarantees the limit of 0).

Additional context Credits would expand the ledger's realm of use-cases, to performing FX between currencies with double-entry book-keeping (which @world can provide in a sense, albeit at the cost of complexity), interest calculations and cash optimisation.

This feature allows us to filters the accounts we get using the /{ledger}/accounts route by their balance

The request now takes two more optionnal parameters, and <balance_operator>

• balance : int64 that represents the balance volume of the account
• balance_operator : comparison operator, can be gt, gte, e, lt, lte

I also tried to make the tests easier to read by bracketing them

HI there,

I tried opening the Dashboard via the SSH link i have, but it just came up with the following error: 2021/07/22 12:57:08 exec: "xdg-open": executable file not found in $PATH I have already changed the HTML bind address to the external IP. Is there some URL I can visit to open the Dashboard or does it have to be started manually?

thanks, Philip

Title

Test already depends on vendor task, which already runs go mod vendor

Type of change

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [X] Refactoring / Technical debt

What parts of the code are impacted ?

Taskfile for Go SDK

Checklist:

• [X] My code follows the style guidelines of this project
• [X] I have performed a self-review of my code
• [X] I have commented my code, particularly in hard-to-understand areas
• [X] I have made corresponding changes to the documentation
• [X] My changes generate no new warnings
• [X] I have added tests that prove my fix is effective or that my feature works
• [X] New and existing unit tests pass locally with my changes

Describe the bug

No words guys:

$ numary --telemetry-enabled=false server start
  loading config file: Config File "numary" Not Found in "[/Users/r0ck3t/.numary /etc/numary]"
  segment 2022/11/15 23:58:59 ERROR: sending request - Post "https://api.segment.io/v1/batch": dial tcp: lookup api.segment.io: no such host
  segment 2022/11/15 23:58:59 ERROR: sending request - Post "https://api.segment.io/v1/batch": dial tcp: lookup api.segment.io: no such host
  segment 2022/11/15 23:58:59 ERROR: sending request - Post "https://api.segment.io/v1/batch": dial tcp: lookup api.segment.io: no such host

To Reproduce

1. Run numary --telemetry-enabled=false server start
2. See logs in firewall or in wireshark.

Expected behavior

Disable telemetry by default!

Environment (please complete the following information):

• MacOS Monterey, Macbook on M1

• Numary Version:

  Version: 1.8.0 
  Date: 2022-10-31T08:29:29Z 
  Commit: a55756b 
  

Additional context

You are doing something wrong guys, telemetry must be disabled by default in opensource apps, especially if they were build from source code. It's super stupid.

Describe the bug Ledger is making a connection to segment even when Segment integration is disabled. I use Pi-Hole blocker and this results in an error log printed in the log.

To Reproduce Block api.segment.io DNS resolution via some means like Pi-Hole etc.

Expected behavior Since the integration is disabled, there should be no client created / connection attempted.

Logs segment 2022/10/17 21:34:23 ERROR: sending request - Post "https://api.segment.io/v1/batch": dial tcp [::]:443: connect: connection refused segment 2022/10/17 21:34:23 ERROR: sending request - Post "https://api.segment.io/v1/batch": dial tcp [::]:443: connect: connection refused segment 2022/10/17 21:34:23 ERROR: sending request - Post "https://api.segment.io/v1/batch": dial tcp [::]:443: connect: connection refused

Environment (please complete the following information):

• OS: Osx 12.5.1
• Numary Version Version: 1.7.4 Date: 2022-09-02T12:29:05Z Commit: ceeef7f

Additional context the problem goes away if I disable Pi-Hole blocking

Is your feature request related to a problem? Please describe. About the fact that accounts can't have negative values, is it possible to overcome this in some way ? (In the formance documentation: Accounts in Formance Ledger cannot go negative! (Except for the special @world account).)

SDK : Add java

From #261

How to deploy

Create a sonatype account and follow the guide : https://central.sonatype.org/publish/publish-guide/#introduction

Once sonatype account configured, you have to create a file ~/.m2/settings.xml :

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
    <servers>
      <server>
          <id>ossrh</id>
          <username>******</username>
          <password>******</password>
      </server>
  </servers>
</settings>

To deploy, you can use the following command to deploy snapshot version :

 mvn -B deploy -Psign-artifacts -DaltDeploymentRepository=ossrh::default::https://s01.oss.sonatype.org/content/repositories/snapshots/

To deploy a release version :

 mvn -B deploy -Psign-artifacts -DaltDeploymentRepository=ossrh::default::https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/

Remember already initialized ledgers

Just remember what ledgers has already been initialized and avoid sql queries.

Type of change

• [X] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Refactoring / Technical debt

Is your feature request related to a problem? Please describe. In some scenarios it is needed to create transactions that happened in the past and not in the moment they were committed.

Summary

Solution proposal Make the api accept a backdate property

{
  "postings": [
    {
      "source": "alice",
      "destination": "teller",
      "amount": 100,
      "asset": "COIN"
    },
    {
      "source": "teller",
      "destination": "alice",
      "amount": 5,
      "asset": "GEM"
    }
  ],
  backdate: $timestamp
}

This should update the balances accordingly or throw an error if subsequent transactions cannot be applied to avoid leaving the system in an invalid state

I'm trying to learn and use Nix, so to test that and numary at the same time I've proposed adding numary to the Nixos package directory.

https://github.com/NixOS/nixpkgs/pull/191106

I've followed the instructions for building present at the current Dockerfile but I could use some help with the tests.

feat: use sharedanalytics package

This basically extract the analytics code to go-libs library (https://github.com/numary/go-libs/pull/9). Please review go-libs before.

Type of change

• [ ] Bug fix (non-breaking change which fixes an issue)
• [X] New feature (non-breaking change which adds functionality)
• [ ] Refactoring / Technical debt

Remove post commit volumes and pre commit volumes calculation from transaction batches. Aggregation is provided by the core using functions AggregatePostCommitVolumes() and AggregatePreCommitVolumes(). This simplify a bit the transaction processing and remove duplicate codes.