Secure software enclave for storage of sensitive information in memory.

As we move closer to v1.0.0, do you have any suggestions for improvements to the API?

I've used the library myself in my own projects, so primarily that's the only indication I have of the suitability, ease-of-use, and just general quality of the API.

So, to those of you that use memguard (or not), add a comment with your opinions or suggestions.

Program:

package main

import (
	"fmt"
	"github.com/libeclipse/memguard"
)

func main() {
	for i := 0; i < 100; i++ {
		key := []byte("123456768901234567890123456789012")
		safeKey, err := memguard.NewFromBytes(key, true)
		if err != nil {
			panic(err.Error())
		}
		fmt.Printf("%v: %v\n", i, safeKey)
	}
}

Output on a CentOS Linux release 7.3.1611 VM:

0: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
1: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
2: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
3: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
4: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
5: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
6: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
7: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
8: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
9: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
10: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
11: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
12: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
13: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
14: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
15: &{{0 0} [49 50 51 52 53 54 55 54 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50] true false}
panic: memguard.memcall.Lock(): could not acquire lock on 0x7f7711a67000 [Err: cannot allocate memory]

goroutine 1 [running]:
github.com/libeclipse/memguard/memcall.Lock(0x7f7711a67000, 0x1000, 0x2000)
	~/go/src/github.com/libeclipse/memguard/memcall/memcall_unix.go:18 +0x16c
github.com/libeclipse/memguard.New(0x21, 0x4cd800, 0x21, 0x30, 0x21)
	~/go/src/github.com/libeclipse/memguard/memguard.go:78 +0xf6
github.com/libeclipse/memguard.NewFromBytes(0xc420010bd0, 0x21, 0x30, 0xc420010b01, 0x21, 0x30, 0x0)
	~/go/src/github.com/libeclipse/memguard/memguard.go:113 +0x34
main.main()
	try.go:11 +0x8a

It happens on every run on this host. I get the same results on a CentOS release 6.9 (Final) VM. It does not fail on my Mac.

Maybe I should have thought this through a bit more, but it seems to me that there is too much package level variables. I think that prevents memguard from being used by multiple dependencies of a Go program. For example; consider a program "package a" that depends on memguard/core and also on "package b", which in turn depends on memguard/core. Suppose both "a" and "b" try to set core.Interval, then there is a data race as it is not known which store to core.Interval happened last and which will affect the memguard funcs.

• Refactoring of functionality into core package.
• Addition of Enclave objects which store encrypted data.
• Rewrite of signals handling. Should make implementing further data catching much easier.
• Rewrite and simplification of the API.
• Addition of examples package.
• Patched data races in interrupt handling functionality.
• Conceal mapped memory from core dumps on OpenBSD.
• Additional functionality added such as safe panicking.
• Disable core dumps by default. Users that want them can enable themselves.
• Use local entropy pool for fast random bytes reading.
• Use a variable sized canary value with 32 <= len(canary) <= ~pagesize.
• Other optimisations and improvements.
• ...

We encountered an infrequent problem at runtime in one of Ubuntu docker containers:

panic: runtime error: slice bounds out of range [:32] with capacity 0
goroutine 147009 [running]:
github.com/xxx/secretsclient.EncryptMapData(0xc03ca81c80, 0x4, 0xc0209ba1b0, 0xc0004d5200, 0x45, 0x5e, 0x0, 0x0)
	/go/pkg/mod/github.com/xxx/[email protected]/encrypt.go:125 +0x1fae
...

The code segment around encrypt.go:125 is (with an added line # and comments)

(119)	keyIVBuf, err := encKeyIV.Open()  // encKeyIV was from keyiv.Seal() with keyiv.Size() == KeyIVSIZE(48)
	if err != nil {
		return err
	}
	defer keyIVBuf.Destroy()

(125)	block, err := aes.NewCipher(keyIVBuf.Bytes()[:cKeySize])  // cKeySize = 32
	if err != nil {
		return err
	}

Couldn't figure out what went wrong... would a low memlock ulimit possibly cause this? Thanks. (we do have ulimit setting

/ # cat /etc/security/limits.conf
* soft memlock 512000
* hard memlock 1024000

Somehow the ulimit command still shows the default (this would encounter problems quickly for us):

/ # ulimit -l
64

so a little puzzled here too.) Appreciate your help!!

Additional info: version github.com/awnumar/memguard v0.22.2 OS: Ubuntu 16.04 docker image Go 1.13

We are also seeing the following error now in one of the docker instances (same instance as above):

panic: runtime error: index out of range [0] with length 0
goroutine 6911 [running]:
github.com/awnumar/memguard/core.(*Coffer).View(0xc00001ec90, 0x0, 0x0, 0x0)
	/go/pkg/mod/github.com/awnumar/[email protected]/core/coffer.go:112 +0x207
github.com/awnumar/memguard/core.Open(0xc00979aa20, 0x661f81, 0xc00000ab40, 0x0)
	/go/pkg/mod/github.com/awnumar/[email protected]/core/enclave.go:101 +0x60
github.com/awnumar/memguard.(*Enclave).Open(0xc009482150, 0xc034b6d278, 0x4, 0xc009482150)
	/go/pkg/mod/github.com/awnumar/[email protected]/enclave.go:43 +0x32
github.com/xxx/secretsclient.EncryptMapData(0xc034b6d278, 0x4, 0xc02fd7cea0, 0xc00016d6c0, 0x16, 0x1c, 0x0, 0x0)
	/go/pkg/mod/github.com/xxx/[email protected]/encrypt.go:119 +0x9d
...

Is your feature request related to a problem? Please describe. When testing cryptographic systems it's always useful to be able to have a predictable source of randomness so you can support test vectors. For NewImmutableRandom, NewMutableRandom, FillRandomBytes, FillRandomBytesAt it would be nice to be able to provide predictable randomness during a test phase.

Describe the solution you'd like One way of solving the problem would be to have a way of temporarily setting a rand Reader for the package temporarily. Another solution would be that each LockedBuffer has an io.Reader associated with it, that can be set (although that doesn't help for NewImmutableRandom and NewMutableRandom. Another solution would be to add functions that take a Reader for the randomness source.

Describe alternatives you've considered I don't really have any good alternatives to how to solve this.

As suggested in https://www.reddit.com/r/golang/comments/6s84ky/memory_security_in_go/dlcmzp3/

Some cosmetic/ux changes may be needed. LockedBuffers() was removed to allow implementing automatic destruction. This shouldn't be much of a problem since I can't think of any good use cases outside of DeleteAll().

Is your feature request related to a problem? Please describe. Currently the guard pages use noaccess protection, but the inner region (canary + data) is still readable. This leaves the inner region vulnerable to memory scans (have verified this locally).

Describe the solution you'd like It would be nice to have an option to make the inner region always be set to noaccess and only make it read-only when it needs to be read. Making this a default behavior would likely require a breaking API change. This is also similar to what's described in https://download.libsodium.org/doc/memory_management#guarded-heap-allocations in the sodium_mprotect_noaccess related info (specifically, This function can be used to make confidential data inaccessible except when actually needed for a specific operation.).

We have Java and C# implementations that are very similar to your project and libsodium (we only came across your project when our Go implementation was started). The approach we've taken is:

• accepting a lambda/function from the user that operates on the data
• call mprotect with readonly before the lambda invocation and then mprotect with noaccess after the lambda completes
• we use a lock and access counter to minimize the mprotect readonly and noaccess system calls to only be done when needed (readonly after 1st access, noaccess after last caller exits)

You can refer to how we've implemented this in https://github.com/godaddy/asherah/blob/4ead9ca1803056d2fe2ea006aaa771b3301ae28b/languages/java/secure-memory/src/main/java/com/godaddy/asherah/securememory/protectedmemoryimpl/ProtectedMemorySecret.java#L89-L113.

Describe alternatives you've considered

1. We considered using Enclave instead of LockedBuffer but we expect that for our use case, it will likely result in higher unmanaged memory usage (lot of caching and concurrent reads of keys).
2. We've considered seeing if we can replicate the page boundary calculations to get an unsafe handle on the inner region start, but this seems very messy and error prone.
3. We're considering replicating our Secure Memory implementation in Go, similar to what we did for Java/C#, but we're hoping to use your implementation for Go instead. The only other thing that may be an issue for us is whether the guard pages cause too much unmanaged memory in our workloads, but we'll cross that bridge when we get there.

Additional context Think I covered all the details. Let me know if you have any questions on this. I can add the script that was used to verify the memory is readable if you'd like.

I'm implementing a transparently encrypted in-memory filesystem, and I'm storing the data of in-memory files as pointers to Enclaves *memguard.Enclave. I'm not sure what to do when I need to delete a file, or truncate a file to 0. I know that LockedBuffers can be destroyed with the Destroy method, but Enclaves don't have such a method.

I'm also aware that when memguard.Purge is called, Enclaves are rendered useless by rolling the decryption key for them, which makes sense. How then can I safely invalidate a single enclave at will? Is that possible?

See HN article for more detail. TL;DR, golang does not specify any particular semantics about memory addresses. Calling mlock on a go-managed memory address provide little to nothing in the way of guarantees, because that memory will be copied and moved around as the runtime sees fit.

This is pretty much unavoidable. The address you call mlock on isn't even guaranteed to be the address passed in via memlock.Lock(b).

Hi, I am just wasting some time so I thought I'd compare performance requirements and I am getting panic from enclave.Open(). Any idea why?

package whatever

import (
	"github.com/awnumar/memguard"
	"testing"
)

func BenchmarkEnclaveOpen(b *testing.B) {
	secret := []byte("secret secret secret")
	e := memguard.NewEnclave(secret)

	for i := 0; i < b.N; i++ {
		e.Open()
	}
}

func BenchmarkEnclaveOpenRead(b *testing.B) {
	secret := []byte("secret secret secret")
	e := memguard.NewEnclave(secret)

	for i := 0; i < b.N; i++ {
		lb, _ := e.Open()
		lb.String()
	}
}

func BenchmarkLBRead(b *testing.B) {
	secret := []byte("secret secret secret")
	lb := memguard.NewBufferFromBytes(secret)

	for i := 0; i < b.N; i++ {
		lb.String()
	}
}

Describe the bug Importing github.com/awnumar/memguard causes a panic when run in a systemd-nspawn container.

To Reproduce

go.mod:

module test

go 1.19

require github.com/awnumar/memguard v0.22.3

main.go:

package main

import _ "github.com/awnumar/memguard"

func main() {}

Run:

go mod tidy
go run .

Expected behaviour The command should complete with no panic.

Screenshots

$ go run .
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x49af1f]

goroutine 1 [running]:
github.com/awnumar/memguard/core.Purge.func1(0xc0000aea60)
	/root/go/pkg/mod/github.com/awnumar/[email protected]/core/exit.go:23 +0x3f
github.com/awnumar/memguard/core.Purge()
	/root/go/pkg/mod/github.com/awnumar/[email protected]/core/exit.go:51 +0x25
github.com/awnumar/memguard/core.Panic({0x4a69c0, 0xc000014370})
	/root/go/pkg/mod/github.com/awnumar/[email protected]/core/exit.go:85 +0x25
github.com/awnumar/memguard/core.NewBuffer(0x20)
	/root/go/pkg/mod/github.com/awnumar/[email protected]/core/buffer.go:73 +0x2d5
github.com/awnumar/memguard/core.NewCoffer()
	/root/go/pkg/mod/github.com/awnumar/[email protected]/core/coffer.go:30 +0x34
github.com/awnumar/memguard/core.init.0()
	/root/go/pkg/mod/github.com/awnumar/[email protected]/core/enclave.go:15 +0x2e
exit status 2

System (please complete the following information):

• OS and Kernel Versions: arch
• Memguard Version: v0.22.3
• Go Version: go1.19.4 linux/amd64

Additional context

We recently started using this project in telegraf in our new secret store implementation. A user reported this issue in https://github.com/influxdata/telegraf/issues/12403 and I was able to reproduce it outside of telegraf with the above steps.

Is your feature request related to a problem? Please describe. Using guard pages has a tradeoff of taking up more unmanaged memory pages, which could be a potential concern in high traffic scenarios.

Describe the solution you'd like Would be nice if there was an option to skip use guard pages when creating LockedBuffers. Can obviously keep using them by default but allow the user to pass in an option to avoid them. Whether this is exposed as part of the Enclave related usage should be considered, but could probably be handled separately, if needed.

Describe alternatives you've considered Considered using Enclave, but in the expected traffic/usage patterns (cached keys, multiple concurrent users), it would likely lead to more unmanaged memory usage.

Additional context N/A