Scribble - A tiny Golang JSON database

Experimental feature for add Options to the New method. As example, i added ExistDir option, which is related with https://github.com/nanopack/yoke/issues/10 . If ExistDir is true, it not creates new dir, before starting.

The following is not found: godoc.org / github.com/nanobox-io/golang-hatchet from:

For an example of a qualified logger see here.

Using a nil Logger results in lumber for a console log, ok, for playing around.

Also, there isn't an Open for an existing database, so something like this would be nice:

// Open an existing scribble database at the desired directory location, and
// return a *Driver to then use for interacting with the database
func Open(dir string, logger Logger) (driver *Driver, err error) {
    dir = filepath.Clean(dir)

    // ensure the database location does exist
    if _, err := os.Stat(dir); err != nil {
        fmt.Printf("Database, '%s' does not exist.\n", dir)
        os.Exit(1)
    }

    if logger == nil {
        logger = lumber.NewConsoleLogger(lumber.INFO)
    }

    driver = &Driver{
        dir:     dir,
        mutexes: make(map[string]sync.Mutex),
        log:     logger,
    }

    logger.Info("Opening scribble database at '%v'...\n", dir)

    return driver, nil
}

... or perhaps that isn't a valid use case (well, it is for me). Thanks for scribble.

usage eg.:

func userGetList() []*userS {
    usersIDs, err := db.List("user")
    if err != nil {
        panic(err)
    }

    users := make([]*userS, len(usersIDs))
    for i, userID := range usersIDs {
        if err := db.Read("user", userID, &users[i]); err != nil {
            panic(err)
        }
    }
}

I tried to add scribble to my code using

glide get github.com/nanobox-io/golang-scribble

I am prompted for my GitHub username and password but I get the after providing which I get the following error

[WARN]	Unable to checkout github.com/nanobox-io/golang-scribbl
[ERROR]	Update failed for github.com/nanobox-io/golang-scribbl: Unable to get repository
[ERROR]	Failed to checkout packages: Unable to get repository

Can we have this package available through glide as well?

Hi, I changed the code to simply return an empty array if the folder for the collections doesn't already exists. Otherwise we would have to check in the code each time if it exists, which I find counterintuitiv and also not perfect for performance. Otherwise I would like to add a methode to create the collections add the beginning.

The examples show how to read everything from a database:

// Read all fish from the database, unmarshaling the response.
records, err := db.ReadAll("fish")
if err != nil {
  fmt.Println("Error", err)
}

fish := []Fish{}
if err := json.Unmarshal([]byte(records), &fish); err != nil {
  fmt.Println("Error", err)
}

However, db.ReadAll returns a []string, which cannot be cast to []byte in the call to json.Unmarshal.

The get getOrCreateMutex function copies mutexes which the golang docs specifically tell us not to do: https://golang.org/pkg/sync/#Mutex Changed the mutex map to pointers to avoid this happening.

Also, the delete function was creating a mutex for the resource, not the collection.

just general clean up of a few things that will eventually cause issues.

• files are now written to a temporary location first. This fixes a situation where a file is truncated, and the scribble process exists before completing the write. DATALOSS
• mutex locks are now unlocked in defer statements, this fixes situations where a lock is held and the not unlocked because errors cause the function to return early.

This PR fixes issues #26 and #28.

It's super small, adding a newline with a one line change to fix #26.

In another it changes a function call to no longer "reference" a variable that is already a pointer. This fixes #28.

Passing a receiving variable by value to scribble.Read() results in a silent failure to populate it. The underlying json.Unmarshall() handles correctly returns an error, but it does not propagate.

I believe this happens because scribble.read() "references" (puts ampersand in front of) the v variable). This means that json.Unmarshall() gets the pointer it expects and populates it, but in reality, it is a pointer to a local copy of v inside scribble.read().

I believe the fix is to remove the ampersand from the last line of scribble.read(). There is no need to include it, because the value must already be a pointer, and it should be passed as is to json.Unmarshall().

Basically, change:

func read(record string, v interface{}) error {

	b, err := ioutil.ReadFile(record + ".json")
	if err != nil {
		return err
	}

	// unmarshal data
	return json.Unmarshal(b, &v)
}

to

func read(record string, v interface{}) error {

	b, err := ioutil.ReadFile(record + ".json")
	if err != nil {
		return err
	}

	// unmarshal data
	return json.Unmarshal(b, v)
}

After applying this fix locally, all my unit tests of scribble still pass, but passing a v variable by value results in an error as expected and desired.

I noticed that the change to add a newline to the end of json files, that got committed in 40263ff was reverted again without any comment in 4116320 (the otherwise very nice pull request #21).

Given that there was no note regarding that, I'm thinking that it was an inadvertent regression. I really like having the newline.

Looks like a very simple fix, just to add that code back at the end of write() prior to writing the marshaled data to temp file.

	// marshal the pointer to a non-struct and indent with tab
	b, err := json.MarshalIndent(v, "", "\t")
	if err != nil {
		return err
	}

	// add newline to the end
	b = append(b, byte('\n'))

	// write marshaled data to the temp file
	if err := ioutil.WriteFile(tmpPath, b, 0644); err != nil {
		return err

scribble.Driver.Delete returns an opaque error type for non-existent keys which makes it hard for the clients to distinguish between different kind of errors. This behavior is also inconsistent with the scribble.Driver.Read API.

Please add some context around the TODO item "Better support for concurrency." What exactly are the current weaknesses when it comes to concurrency?