cloud infra tooling for lnd provisioning/unlocking

lndinit: a wallet initializer utility for lnd

This repository contains the source for the lndinit command. The main purpose of lndinit is to help automate the lnd wallet initialization, including seed and password generation.

Requirements

Most commands of this tool operate independently of lnd and therefore don't require a specific version to be installed.

The commands wait-ready and init-wallet only work with lnd v0.14.2-beta and later though.

A recent version of Kubernetes is needed when interacting with secrets stored in k8s. Any version >= v1.8 should work.


Subcommands

Most commands work without lnd running, as they are designed to do some provisioning work before lnd is started.

gen-password

gen-password generates a random password (no lnd needed)

gen-seed

gen-seed generates a random seed phrase

No lnd needed, but seed will be in lnd-specific aezeed format

load-secret

load-secret interacts with kubernetes to read from secrets (no lnd needed)

store-secret

store-secret interacts with kubernetes to write to secrets (no lnd needed)

init-wallet

init-wallet has two modes:

  • --init-type=file creates an lnd specific wallet.db file
    • Only works if lnd is NOT running yet
  • --init-type=rpc calls the lnd RPC to create a wallet
    • Use this mode if you are using a remote database as lnd's storage backend instead of bolt DB based file databases
    • Needs lnd to be running and no wallet to exist

wait-ready

wait-ready waits for lnd to be ready by connecting to lnd's status RPC

  • Needs lnd to run, eventually

Example Usage

Example use case 1: Basic setup

This is a very basic example that shows the purpose of the different sub commands of the lndinit binary. In this example, all secrets are stored in files. This is normally not a good security practice as potentially other users or processes on a system can read those secrets if the permissions aren't set correctly. It is advised to store secrets in dedicated secret storage services like Kubernetes Secrets or HashiCorp Vault.

1. Generate a seed without a seed passphrase

Create a new seed if one does not exist yet.

$ if [[ ! -f /safe/location/seed.txt ]]; then
    lndinit gen-seed > /safe/location/seed.txt
  fi

2. Generate a wallet password

Create a new wallet password if one does not exist yet.

$ if [[ ! -f /safe/location/walletpassword.txt ]]; then
    lndinit gen-password > /safe/location/walletpassword.txt
  fi

3. Initialize the wallet

Create the wallet database with the given seed and password files. If the wallet already exists, we make sure we can actually unlock it with the given password file. This will take a few seconds in any case.

$ lndinit -v init-wallet \
    --secret-source=file \
    --file.seed=/safe/location/seed.txt \
    --file.wallet-password=/safe/location/walletpassword.txt \
    --init-file.output-wallet-dir=$HOME/.lnd/data/chain/bitcoin/mainnet \
    --init-file.validate-password

4. Start and auto unlock lnd

With everything prepared, we can now start lnd and instruct it to auto unlock itself with the password in the file we prepared.

$ lnd \
    --bitcoin.active \
    ...
    --wallet-unlock-password-file=/safe/location/walletpassword.txt

Example use case 2: Kubernetes

This example shows how Kubernetes (k8s) Secrets can be used to store the wallet seed and password. The pod running those commands must be provisioned with a service account that has permissions to read/create/modify secrets in a given namespace.

Here's an example of a service account, role provision and pod definition:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: lnd-provision-account


---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: lnd-update-secrets-role
  namespace: default
rules:
  - apiGroups: [ "" ]
    resources: [ "secrets" ]
    verbs: [ "get", "list", "watch", "update" ]


---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: lnd-update-secrets-role-binding
  namespace: default
roleRef:
  kind: Role
  name: lnd-update-secrets-role
  apiGroup: rbac.authorization.k8s.io
subjects:
  - kind: ServiceAccount
    name: lnd-provision-account
    namespace: default


---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: lnd-pod
spec:
  strategy:
    type: Recreate
  replicas: 1
  template:
    spec:
      # We use the special service account created, so the init script is able
      # to update the secret as expected.
      serviceAccountName: lnd-provision-account

      containers:
        # The main lnd container
        - name: lnd
          
          # The lndinit image is an image based on the main lnd image that just
          # adds the lndinit binary to it. The tag name is simply:
          #   <lndinit-version>-lnd-<lnd-version>
          image: lightninglabs/lndinit:v0.1.0-lnd-v0.14.2-beta
          env:
            - name: WALLET_SECRET_NAME
              value: lnd-wallet-secret
            - name: WALLET_DIR
              value: /root/.lnd/data/chain/bitcoin/mainnet
            - name: CERT_DIR
              value: /root/.lnd
            - name: UPLOAD_RPC_SECRETS
              value: '1'
            - name: RPC_SECRETS_NAME
              value: lnd-rpc-secrets
          command: [ '/init-wallet-k8s.sh' ]
          args: [
              '--bitcoin.mainnet',
              '...',
              '--wallet-unlock-password-file=/tmp/wallet-password',
          ]

The /init-wallet-k8s.sh script that is invoked in the example above can be found in this repository: example-init-wallet-k8s.sh The script executes the steps described in this example and also uploads the RPC secrets (tls.cert and all *.macaroon files) to another secret so apps using the lnd node can access those secrets.

1. Generate a seed passphrase (optional)

Generate a new seed passphrase. If an entry with the key already exists in the k8s secret, it is not overwritten, and the operation is a no-op.

$ lndinit gen-password \
    | lndinit -v store-secret \
    --target=k8s \
    --k8s.secret-name=lnd-secrets \
    --k8s.secret-key-name=seed-passphrase

2. Generate a seed using the passphrase

Generate a new seed with the passphrase created before. If an entry with that key already exists in the k8s secret, it is not overwritten, and the operation is a no-op.

$ lndinit -v gen-seed \
    --passphrase-k8s.secret-name=lnd-secrets \
    --passphrase-k8s.secret-key-name=seed-passphrase \
    | lndinit -v store-secret \
    --target=k8s \
    --k8s.secret-name=lnd-secrets \
    --k8s.secret-key-name=seed

3. Generate a wallet password

Generate a new wallet password. If an entry with that key already exists in the k8s secret, it is not overwritten, and the operation is a no-op.

$ lndinit gen-password \
    | lndinit -v store-secret \
    --target=k8s \
    --k8s.secret-name=lnd-secrets \
    --k8s.secret-key-name=wallet-password

4. Initialize the wallet, attempting a test unlock with the password

Create the wallet database with the given seed, seed passphrase and wallet password loaded from a k8s secret. If the wallet already exists, we make sure we can actually unlock it with the given password file. This will take a few seconds in any case.

$ lndinit -v init-wallet \
    --secret-source=k8s \
    --k8s.secret-name=lnd-secrets \
    --k8s.seed-key-name=seed \
    --k8s.seed-passphrase-key-name=seed-passphrase \
    --k8s.wallet-password-key-name=wallet-password \
    --init-file.output-wallet-dir=$HOME/.lnd/data/chain/bitcoin/mainnet \
    --init-file.validate-password

The above is an example for a file/bbolt based node. For such a node creating the wallet directly as a file is the most secure option, since it doesn't require the node to spin up the wallet unlocker RPC (which doesn't use macaroons and is therefore un-authenticated).

But in setups where the wallet isn't a file (since all state is in a remote database such as etcd or Postgres), this method cannot be used. Instead, the wallet needs to be initialized through RPC, as shown in the next example:

$ lndinit -v init-wallet \
    --secret-source=k8s \
    --k8s.secret-name=lnd-secrets \
    --k8s.seed-key-name=seed \
    --k8s.seed-passphrase-key-name=seed-passphrase \
    --k8s.wallet-password-key-name=wallet-password \
    --init-type=rpc \
    --init-rpc.server=localhost:10009 \
    --init-rpc.tls-cert-path=$HOME/.lnd/tls.cert

NOTE: If this is used in combination with the --wallet-unlock-password-file= flag in lnd for automatic unlocking, then the --wallet-unlock-allow-create flag also needs to be set. Otherwise, lnd won't be starting the wallet unlocking RPC that is used for initializing the wallet.

The following example shows how to use the lndinit init-wallet command to create a watch-only wallet from a previously exported accounts JSON file:

$ lndinit -v init-wallet \
    --secret-source=k8s \
    --k8s.secret-name=lnd-secrets \
    --k8s.seed-key-name=seed \
    --k8s.seed-passphrase-key-name=seed-passphrase \
    --k8s.wallet-password-key-name=wallet-password \
    --init-type=rpc \
    --init-rpc.server=localhost:10009 \
    --init-rpc.tls-cert-path=$HOME/.lnd/tls.cert \
    --init-rpc.watch-only \
    --init-rpc.accounts-file=/tmp/accounts.json

5. Store the wallet password in a file

Because we now only have the wallet password as a value in a k8s secret, we need to retrieve it and store it in a file that lnd can read to auto unlock.

$ lndinit -v load-secret \
    --source=k8s \
    --k8s.secret-name=lnd-secrets \
    --k8s.secret-key-name=wallet-password > /safe/location/walletpassword.txt

Security notice:

Any process or user that has access to the file system of the container can potentially read the password if it's stored as a plain file. For an extra bump in security, a named pipe can be used instead of a file. That way the password can only be read exactly once from the pipe during lnd's startup.

# Create a FIFO pipe first. This will behave like a file except that writes to
# it will only occur once there's a reader on the other end.
$ mkfifo /tmp/wallet-password

# Read the secret from Kubernetes and write it to the pipe. This will only
# return once lnd is actually reading from the pipe. Therefore we need to run
# the command as a background process (using the ampersand notation).
$ lndinit load-secret \
    --source=k8s \
    --k8s.secret-name=lnd-secrets \
    --k8s.secret-key-name=wallet-password > /tmp/wallet-password &

# Now run lnd and point it to the named pipe.
$ lnd \
    --bitcoin.active \
    ...
    --wallet-unlock-password-file=/tmp/wallet-password

6. Start and auto unlock lnd

With everything prepared, we can now start lnd and instruct it to auto unlock itself with the password in the file we prepared.

$ lnd \
    --bitcoin.active \
    ...
    --wallet-unlock-password-file=/safe/location/walletpassword.txt

Logging and idempotent operations

By default, lndinit aborts and exits with a zero return code if the desired result is already achieved (e.g. a secret key or a wallet database already exist). This can make it hard to follow exactly what is happening when debugging the initialization. To assist with debugging, the following two flags can be used:

  • --verbose (-v): Log debug information to stderr.
  • --error-on-existing (-e): Exit with a non-zero return code (128) if the result of an operation already exists. See example below.

Example:

# Treat every non-zero return code as abort condition (default for k8s container
# commands).
$ set -e

# Run the command and catch any non-zero return code in the ret variable. The
# logical OR is required to not fail because of above setting.
$ ret=0
$ lndinit --error-on-existing init-wallet ... || ret=$?
$ if [[ $ret -eq 0 ]]; then
    echo "Successfully initialized wallet."
  elif [[ $ret -eq 128 ]]; then
    echo "Wallet already exists, skipping initialization."
  else
    echo "Failed to initialize wallet!"
    exit 1
  fi
Comments
  • Feature Request: show seed pubkey

    Feature Request: show seed pubkey

    It would be really great if this app could allow you to input an aezeed and respond with the node pubkey, as a way of checking a seed on an offline machine and matching to a node.

  • Helm Chart

    Helm Chart

    Hello, I love your products, thanks for it and sorry that I have to ask here in lack of other places. I really like that you seem to support K8s indicated by this: https://github.com/lightninglabs/lndinit#example-use-case-2-kubernetes What would be great is a helm chart with which I can deploy an lnd stack on an existing cluster. Are you planning something like this? BG

  • [Suggestion] Fetch key via fingerprint in release notes

    [Suggestion] Fetch key via fingerprint in release notes

    This cmd fetches your key via its fingerprint:

    gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys F4FC70F07310028424EFC20A8E4256593F177720
    

    It's is only slightly longer than the current

    curl https://keybase.io/guggero/pgp_keys.asc | gpg --import
    

    ... but it removes keybase as a trusted party.

  • lnd:v0.14.3-beta + releash.sh path fix

    lnd:v0.14.3-beta + releash.sh path fix

    This PR updates all lnd references to v0.14.3-beta.

    I saw an existing tag but it looks like it still used some 0.14.2 components. When I updated the tag to the contents of this branch, release.sh encountered a path error due to the tag containing docker/.... I'm unsure if this happens during the github build, but replacing / in the tag with _ allowed it to succeed. If this change is unnecessary, please let me know and I'll remove that commit.

  • --k8s.base64 flag causes double encoding

    --k8s.base64 flag causes double encoding

    It appears that when using the --k8s.base64 flag, secrets are encoded in base64 twice. This causes issues with decoding.

    Kubernetes Verison 1.22 / 1.23

    To Reproduce

    From a pod within a k8s context (has role to create secrets). Create two secrets, one with base64 flag and one without..

    # create secret using base64 flag
    $ lndinit gen-password \
      | lndinit -v store-secret \
      --target=k8s \
      --k8s.base64 \
      --k8s.namespace="default" \
      --k8s.secret-name="lnd-wallet-secret-b64" \
      --k8s.secret-key-name=walletpassword
    
    # create secret without base64 flag
    $ lndinit gen-password \
      | lndinit -v store-secret \
      --target=k8s \
      --k8s.namespace="default" \
      --k8s.secret-name="lnd-wallet-secret" \
      --k8s.secret-key-name=walletpassword
    

    Then from your host, decode secrets, you will see the base64 flagged one requires double decoding..

    # decode base64 flagged secret
    $ k get secret lnd-wallet-secret-b64 -o json | jq -r .data.walletpassword | base64 -d
    bWF6ZS1wb3J0aW9uLWlkZWEtc29kYS1uZXN0LXJhaW4tZm9vZC1wb3B1bGFy
    
    # double decode base64 flagged secret
    $ k get secret lnd-wallet-secret-b64 -o json | jq -r .data.walletpassword | base64 -d | base64 -d
    maze-portion-idea-soda-nest-rain-food-popular
    
    # decode non base64 flagged secret
    $ k get secret lnd-wallet-secret -o json | jq -r .data.walletpassword | base64 -d
    slice-random-school-energy-cart-proof-notice-nest
    

    Expected behavior

    Using --k8s.base64 flag would not require double decoding. This makes unlocking the wallet fail, as you would imagine.

  • master should become main

    master should become main

    in this line, I think main should be used instead of master as that's the actual branch name and, when I try to build this Dockerfile "out of the box", I get the error below:

    #7 7.310 error: pathspec 'master' did not match any file(s) known to git
    
  • What type of database is used for wallet.db

    What type of database is used for wallet.db

    I can generate wallets in an automated flow. There are three files generated - seed, password and wallet.db. What is the type of database used for wallet.db? Is this something I can decode and see having seed and password?

  • Is LND node running needed for lndinit to work?

    Is LND node running needed for lndinit to work?

    It's not an issue, but just to get a better understanding. For some reason I thought lndinit works via running requests to an lnd node but then ran without having a node and it still worked. So I suppose lndinit just needs some data from me like macaroons and cert and then does everything internally?

    Thanks!

  • Add lndinit binary

    Add lndinit binary

    Copied over code from https://github.com/lightningnetwork/lnd/pull/5150.

    A docker image can be found at guggero/lnd-with-lndinit:v0.0.1-lnd-v0.14.2-beta-test2.

  • readme: add Command Summary, Table of Contents

    readme: add Command Summary, Table of Contents

    • Command summary taken from @guggero's reply in #11
    • Examples moved under "Example Usage" heading
    • Table of Contents linking to top 2 levels of headings
  • store-secret: read up to EOF from stdin

    store-secret: read up to EOF from stdin

    I should have tested everything on an actual cluster before merging #4 :see_no_evil: Unfortunately the interaction with k8s is a bit tricky to test in unit tests, so we don't have all functionality covered yet. But I'll look into ways of simulating a k8s backend so we can add more unit tests.

    I now manually built and deployed this version on a cluster and made sure everything works!

check-cert: Go-based tooling to check/verify certs

check-cert: Go-based tooling to check/verify certs

Dec 6, 2022
Webserver I built to serve Infura endpoints. Deployable via k8s and AWS EKS. Load testable via k6 tooling, and montiorable via prometheus and grafana

Infura Web Server Welcome to my verion of the take home project. I've created a webserver written in go to serve Infura api data over 3 possible data

Nov 15, 2022
An ATNA (Audit Trail and Node Authentication) Cloud Backup Utility
An ATNA (Audit Trail and Node Authentication) Cloud Backup Utility

ATNA Vault ATNA Vault allows you to maintain a secure long-term archive for all your IHE audit messages. IHE vendors who can provide "filter forward"

Mar 13, 2022
Market-20151101 - Alibaba Cloud Market SDK for Go

English | 简体中文 Alibaba Cloud Market SDK for Go Requirements It's necessary for y

Jan 17, 2022
DORY is a tool who enables people to recover their access to an Active Directory service, by changing, resetting or unlocking their account.

DORY - Server Expose a simple API to manipulate AD. Password reinitialization Password changer Account Unlocking You must have LDAPS (port 636) active

Oct 3, 2022
OPG sirius supervision firm deputy hub: Managed by opg-org-infra & Terraform

OPG sirius supervision firm deputy hub: Managed by opg-org-infra & Terraform

Jan 10, 2022
Go api infra - Infrastructure for making api on golang so easy

Go Infra Api Infrastructre methods and types for make api simple Response abstra

Jun 18, 2022
⚡ 🖥️ 👾 Host your own Lightning Address on LND

⚡ ??️ ?? Host your own Lightning Address on LND Lighting Wallets like BlueWallet, Blixt and many more allow us to send sats to Lighting Addresses like

Dec 22, 2022
Portico Exchange LND

Portico Exchange LND This repository contains a (https://github.com/PorticoExchange) client for LND. It supports Normal Submarine Swaps (from onchain

Jan 31, 2022
Wrapper for Lightning Network Daemon (lnd). It provides separate accounts with minimum trust for end users.

LndHub.go Wrapper for Lightning Network Daemon (lnd). It provides separate accounts with minimum trust for end users. LndHub compatible API implemente

Dec 21, 2022
provide api for cloud service like aliyun, aws, google cloud, tencent cloud, huawei cloud and so on

cloud-fitter 云适配 Communicate with public and private clouds conveniently by a set of apis. 用一套接口,便捷地访问各类公有云和私有云 对接计划 内部筹备中,后续开放,有需求欢迎联系。 开发者社区 开发者社区文档

Dec 20, 2022
FaaSNet: Scalable and Fast Provisioning of Custom Serverless Container Runtimes at Alibaba Cloud Function Compute (USENIX ATC'21)

FaaSNet FaaSNet is the first system that provides an end-to-end, integrated solution for FaaS-optimized container runtime provisioning. FaaSNet uses l

Jan 2, 2023
Easy cloud instance provisioning

post-init (work in progress) Post-Init is a set of tools that allows you to easily connect to, provision, and interact with cloud instances after they

Dec 6, 2021
Tooling to validate HTTPS Certificates and Connections Around Web 🕷️
Tooling to validate HTTPS Certificates and Connections Around Web 🕷️

Cassler - SSL Validator Tool If your read fast, it's sounds like "Cassia Eller" Tooling to validate HTTPS Certificates and Connections Around Web ??️

Sep 14, 2022
A strongly typed HTML templating language that compiles to Go code, and has great developer tooling.
A strongly typed HTML templating language that compiles to Go code, and has great developer tooling.

A language, command line tool and set of IDE extensions that makes it easier to write HTML user interfaces and websites using Go.

Dec 29, 2022
Support CI generation of SBOMs via golang tooling.

Software Package Data Exchange (SPDX) is an open standard for communicating software bill of materials (SBOM) information that supports accurate identification of software components, explicit mapping of relationships between components, and the association of security and licensing information with each component.

Jan 3, 2023
Search and analysis tooling for structured logs

Zed The Zed system provides an open-source, cloud-native, and searchable data lake for semi-structured and structured data. Zed lakes utilize a supers

Jan 5, 2023
Personal notetaking tooling
Personal notetaking tooling

jot CLI Task List and Journal jot is a simple journaling program for CLI that helps you keep track of your life. Config File ~/.jot.yaml is read on st

Aug 9, 2022
my own mm2 client / tooling

mm2-client my own mm2 client / tooling Future tasks: Wallet + encryption seed cancel_all_order setprice buy sell my_recent_swaps my_orders prompt use

Feb 23, 2022
Hermit - uniform tooling for Linux and Mac

Hermit installs tools for software projects in self-contained, isolated sets, so your team, your contributors, and your CI have the same consistent tooling.

Jan 5, 2023