go implementation of fissions web-native file system

wnfs-go

go language implementation of the fission web-native file system, using the typescript implementation as a reference.

Development Status: Work-In-Progress

This repo is very much a work-in-progress, and doesn't yet produce proper WNFS data. Don't rely on this.

Owner
Qri.io
"query" - free & open-source dataset versioning software
Qri.io
Comments
  • Feat/public merge

    Feat/public merge

    Merge method for public directories & basic test suite.

    public.Merge just might take the cake as the longest it's ever taken me to write a function. Much refactor.

  • perf(ratchet): switch to sha3

    perf(ratchet): switch to sha3

    faster.

    $ benchcmp master.txt sha3.txt
    benchmark                                                old ns/op      new ns/op      delta
    BenchmarkIPFSCat10MbFile-4                               23847          25574          +7.24%
    BenchmarkIPFSWrite10MbFile-4                             57115          52016          -8.93%
    BenchmarkIPFSCat10MbFileSubdir-4                         62323          59235          -4.95%
    BenchmarkIPFSWrite10MbFileSubdir-4                       121900         114753         -5.86%
    BenchmarkIPFSCp10DirectoriesWithOne10MbFileEach-4        2504300        2151781        -14.08%
    BenchmarkPublicCat10MbFile-4                             160940         139502         -13.32%
    BenchmarkPublicWrite10MbFile-4                           467663096      413785247      -11.52%
    BenchmarkPublicCat10MbFileSubdir-4                       444089         281957         -36.51%
    BenchmarkPublicWrite10MbFileSubdir-4                     673108614      726374279      +7.91%
    BenchmarkPublicCp10DirectoriesWithOne10MbFileEach-4      6692854643     6931491407     +3.57%
    BenchmarkPrivateCat10MbFile-4                            554663         122874         -77.85%
    BenchmarkPrivateWrite10MbFile-4                          321439286      296391631      -7.79%
    BenchmarkPrivateCat10MbFileSubdir-4                      1106848        178223         -83.90%
    BenchmarkPrivateWrite10MbFileSubdir-4                    414538861      381436433      -7.99%
    BenchmarkPrivateCp10DirectoriesWithOne10MbFileEach-4     3189553722     2754943154     -13.63%
    
    benchmark                                                old allocs     new allocs     delta
    BenchmarkIPFSCat10MbFile-4                               53             53             +0.00%
    BenchmarkIPFSWrite10MbFile-4                             258            258            +0.00%
    BenchmarkIPFSCat10MbFileSubdir-4                         324            324            +0.00%
    BenchmarkIPFSWrite10MbFileSubdir-4                       663            663            +0.00%
    BenchmarkIPFSCp10DirectoriesWithOne10MbFileEach-4        3017           3017           +0.00%
    BenchmarkPublicCat10MbFile-4                             113            113            +0.00%
    BenchmarkPublicWrite10MbFile-4                           968            940            -2.89%
    BenchmarkPublicCat10MbFileSubdir-4                       268            268            +0.00%
    BenchmarkPublicWrite10MbFileSubdir-4                     1503           1514           +0.73%
    BenchmarkPublicCp10DirectoriesWithOne10MbFileEach-4      14672          14831          +1.08%
    BenchmarkPrivateCat10MbFile-4                            101            101            +0.00%
    BenchmarkPrivateWrite10MbFile-4                          860            895            +4.07%
    BenchmarkPrivateCat10MbFileSubdir-4                      165            165            +0.00%
    BenchmarkPrivateWrite10MbFileSubdir-4                    1176           1242           +5.61%
    BenchmarkPrivateCp10DirectoriesWithOne10MbFileEach-4     10362          11235          +8.43%
    
    benchmark                                                old bytes     new bytes     delta
    BenchmarkIPFSCat10MbFile-4                               50278         50289         +0.02%
    BenchmarkIPFSWrite10MbFile-4                             18063         18060         -0.02%
    BenchmarkIPFSCat10MbFileSubdir-4                         66160         66158         -0.00%
    BenchmarkIPFSWrite10MbFileSubdir-4                       53960         53987         +0.05%
    BenchmarkIPFSCp10DirectoriesWithOne10MbFileEach-4        350421        347952        -0.70%
    BenchmarkPublicCat10MbFile-4                             97473         97484         +0.01%
    BenchmarkPublicWrite10MbFile-4                           270720        180541        -33.31%
    BenchmarkPublicCat10MbFileSubdir-4                       110801        110787        -0.01%
    BenchmarkPublicWrite10MbFileSubdir-4                     152432        153520        +0.71%
    BenchmarkPublicCp10DirectoriesWithOne10MbFileEach-4      2335648       2086472       -10.67%
    BenchmarkPrivateCat10MbFile-4                            86010         85959         -0.06%
    BenchmarkPrivateWrite10MbFile-4                          1470024       1411742       -3.96%
    BenchmarkPrivateCat10MbFileSubdir-4                      95656         95623         -0.03%
    BenchmarkPrivateWrite10MbFileSubdir-4                    2141714       2242216       +4.69%
    BenchmarkPrivateCp10DirectoriesWithOne10MbFileEach-4     27745320      27145656      -2.16%
    
  • refactor(HAMT): replace MMPT with go-hampt-ipld

    refactor(HAMT): replace MMPT with go-hampt-ipld

    using the HAMT for now as my terrible MMPT from WNFS v1 is buggy. I'm using a fork b/c the HAMT API should really have a write method, and would love it if cid.CID implemented the CBOR marshaling interface natively.

    This is also showing performance improvements over even the buggy MMPT, I suspect b/c this one only writes when the root is updated. Benchmarks are comparing to master @ e087580fb7716a88a9a36a3d41e6a711a801ec17

    benchmark                                                old ns/op     new ns/op     delta
    BenchmarkIPFSCat10MbFile-4                               24099         21202         -12.02%
    BenchmarkIPFSWrite10MbFile-4                             41901         38335         -8.51%
    BenchmarkIPFSCat10MbFileSubdir-4                         41545         29462         -29.08%
    BenchmarkIPFSWrite10MbFileSubdir-4                       83816         77827         -7.15%
    BenchmarkIPFSCp10DirectoriesWithOne10MbFileEach-4        2032281       2069801       +1.85%
    BenchmarkPublicCat10MbFile-4                             59938         28857         -51.86%
    BenchmarkPublicWrite10MbFile-4                           334974        257696        -23.07%
    BenchmarkPublicCat10MbFileSubdir-4                       98454         49662         -49.56%
    BenchmarkPublicWrite10MbFileSubdir-4                     568033        419323        -26.18%
    BenchmarkPublicCp10DirectoriesWithOne10MbFileEach-4      4707          4134          -12.17%
    BenchmarkPrivateCat10MbFile-4                            46211         38446         -16.80%
    BenchmarkPrivateWrite10MbFile-4                          3433741       3340183       -2.72%
    BenchmarkPrivateCat10MbFileSubdir-4                      54347         53768         -1.07%
    BenchmarkPrivateWrite10MbFileSubdir-4                    4140070       3866860       -6.60%
    BenchmarkPrivateCp10DirectoriesWithOne10MbFileEach-4     2788          2689          -3.55%
    
    benchmark                                                old allocs     new allocs     delta
    BenchmarkIPFSCat10MbFile-4                               42             42             +0.00%
    BenchmarkIPFSWrite10MbFile-4                             204            204            +0.00%
    BenchmarkIPFSCat10MbFileSubdir-4                         123            123            +0.00%
    BenchmarkIPFSWrite10MbFileSubdir-4                       524            524            +0.00%
    BenchmarkIPFSCp10DirectoriesWithOne10MbFileEach-4        2473           2472           -0.04%
    BenchmarkPublicCat10MbFile-4                             108            108            +0.00%
    BenchmarkPublicWrite10MbFile-4                           1168           1186           +1.54%
    refactor(HAMT): replace MMPT with go-hampt-ipld
    BenchmarkPublicCat10MbFileSubdir-4                       243            243            +0.00%
    BenchmarkPublicWrite10MbFileSubdir-4                     2029           2047           +0.89%
    BenchmarkPublicCp10DirectoriesWithOne10MbFileEach-4      20             18             -10.00%
    BenchmarkPrivateCat10MbFile-4                            93             93             +0.00%
    BenchmarkPrivateWrite10MbFile-4                          1756           1276           -27.33%
    BenchmarkPrivateCat10MbFileSubdir-4                      154            154            +0.00%
    BenchmarkPrivateWrite10MbFileSubdir-4                    2555           1653           -35.30%
    BenchmarkPrivateCp10DirectoriesWithOne10MbFileEach-4     10             9              -10.00%
    
    benchmark                                                old bytes     new bytes     delta
    BenchmarkIPFSCat10MbFile-4                               49793         49834         +0.08%
    BenchmarkIPFSWrite10MbFile-4                             11445         11382         -0.55%
    BenchmarkIPFSCat10MbFileSubdir-4                         53420         53419         -0.00%
    BenchmarkIPFSWrite10MbFileSubdir-4                       26986         27150         +0.61%
    BenchmarkIPFSCp10DirectoriesWithOne10MbFileEach-4        253011        253030        +0.01%
    BenchmarkPublicCat10MbFile-4                             53589         53595         +0.01%
    BenchmarkPublicWrite10MbFile-4                           67518         68305         +1.17%
    BenchmarkPublicCat10MbFileSubdir-4                       62011         61997         -0.02%
    BenchmarkPublicWrite10MbFileSubdir-4                     117304        118048        +0.63%
    BenchmarkPublicCp10DirectoriesWithOne10MbFileEach-4      1193          1095          -8.21%
    BenchmarkPrivateCat10MbFile-4                            72976         73035         +0.08%
    BenchmarkPrivateWrite10MbFile-4                          3053406       2870977       -5.97%
    BenchmarkPrivateCat10MbFileSubdir-4                      80932         80902         -0.04%
    BenchmarkPrivateWrite10MbFileSubdir-4                    4109331       3756259       -8.59%
    BenchmarkPrivateCp10DirectoriesWithOne10MbFileEach-4     2471          2373          -3.97%
    
  • use counters for small & medium ratchet fields

    use counters for small & medium ratchet fields

    very nice pickup from the old approach of serializing ceiling shasums. encoded ratchets drop from 216 -> 133 bytes. That'll save 8.3Kib over 100 files.

    example old serialized ratchet:

    jiAjzIubJ5xfbrA5OKv5Nd3pO-m_3ABqD1cFNf2oLviZAJ4prGVG6845XUubOkwO6vqLOoFMQ0Ohp6R4k2VeaQRtEJddpyHw6SojKe8S5uU2F6I7q_ZSsWZT6wWXdQdYwYqNp_UdIye5cS9jw1e9Yxbfc33wSTpLGOxpHezlWckjYF35WHDewaLeUePegN0ILY3e07gEHzRmfhzOikjj0Q==
    

    example new serialized ratchet:

    uFwYqNp_UdIye5cS9jw1e9Yxbfc33wSTpLGOxpHezlWckAmQCeKaxlRuvOOV1LmzpMDur6izqBTENDoaekeJNlXmkAjiAjzIubJ5xfbrA5OKv5Nd3pO-m_3ABqD1cFNf2oLvg
    

    Also, faster:

    benchmark                                old ns/op     new ns/op     delta
    BenchmarkRatchetAdvance256-4             376107        63957         -82.99%
    BenchmarkRatchetDeserializeAdvance-4     1014          878           -13.36%
    
    benchmark                                old allocs     new allocs     delta
    BenchmarkRatchetAdvance256-4             4              3              -25.00%
    BenchmarkRatchetDeserializeAdvance-4     3              3              +0.00%
    
    benchmark                                old bytes     new bytes     delta
    BenchmarkRatchetAdvance256-4             256           176           -31.25%
    BenchmarkRatchetDeserializeAdvance-4     560           368           -34.29%
    
  • perf(benchmarks): add IPFS & Private benchmarks

    perf(benchmarks): add IPFS & Private benchmarks

    getting an early benchmark suite up that compare raw UnixFS, WNFS public, and WNFS private. Results so far:

    goos: darwin
    goarch: amd64
    pkg: github.com/qri-io/wnfs-go
    cpu: Intel(R) Core(TM) i7-6567U CPU @ 3.30GHz
    
    BenchmarkIPFSCat10MbFile-4                             	   61572	     20811 ns/op	   49822 B/op	      42 allocs/op
    BenchmarkIPFSWrite10MbFile-4                           	   30804	     47417 ns/op	   11432 B/op	     204 allocs/op
    BenchmarkIPFSCat10MbFileSubdir-4                       	   36136	     30758 ns/op	   53418 B/op	     123 allocs/op
    BenchmarkIPFSWrite10MbFileSubdir-4                     	   14757	     85009 ns/op	   27069 B/op	     524 allocs/op
    BenchmarkIPFSCp10DirectoriesWithOne10MbFileEach-4      	     652	   1832513 ns/op	  253688 B/op	    2472 allocs/op
    
    BenchmarkPublicCat10MbFile-4                           	   34389	     34300 ns/op	   53607 B/op	     108 allocs/op
    BenchmarkPublicWrite10MbFile-4                         	    4340	    251079 ns/op	   66669 B/op	    1168 allocs/op
    BenchmarkPublicCat10MbFileSubdir-4                     	   20268	     52665 ns/op	   62005 B/op	     243 allocs/op
    BenchmarkPublicWrite10MbFileSubdir-4                   	    2546	    417772 ns/op	  116442 B/op	    2029 allocs/op
    BenchmarkPublicCp10DirectoriesWithOne10MbFileEach-4    	  238276	      5297 ns/op	    1156 B/op	      19 allocs/op
    
    BenchmarkPrivateCat10MbFile-4                          	   23559	     61366 ns/op	  115821 B/op	     119 allocs/op
    BenchmarkPrivateWrite10MbFile-4                        	    2440	    515472 ns/op	  156549 B/op	    2287 allocs/op
    BenchmarkPrivateCat10MbFileSubdir-4                    	   17660	     68230 ns/op	  124090 B/op	     187 allocs/op
    BenchmarkPrivateWrite10MbFileSubdir-4                  	    1707	    755976 ns/op	  228413 B/op	    3289 allocs/op
    BenchmarkPrivateCp10DirectoriesWithOne10MbFileEach-4   	     162	   6474506 ns/op	 1246602 B/op	   10928 allocs/op
    
  • Flakiness in Private Merge

    Flakiness in Private Merge

    I believe it's related to choosing the wrong ratchet during merge. Needs addressing before private merge can be considered even remotely stable

    $ go test --run TestTreeMergeCommit/remote_overwrites_local_file --count 3 -v
    === RUN   TestTreeMergeCommit
    === RUN   TestTreeMergeCommit/remote_overwrites_local_file
    2021-10-27T15:26:41.982-0400	DEBUG	wnfs	private/private.go:381	Tree.Add	{"path": "hello.txt"}
    2021-10-27T15:26:41.984-0400	DEBUG	wnfs	ratchet/store.go:44	writing ratchet	{"name": "_3xDEII-eMyaUQlvmgG4m7pEWPLDRrW4EDE6bMs2Rb4="}
    2021-10-27T15:26:41.984-0400	DEBUG	wnfs	private/private.go:1069	File.Put	{"name": "", "cid": "bafkreieaujplcex3zm76jgrudvrn5v53jqg2k3sp7gyjcj4zm5a4sl5rvy", "size": 6}
    2021-10-27T15:26:41.984-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 1}
    2021-10-27T15:26:41.985-0400	DEBUG	wnfs	ratchet/store.go:44	writing ratchet	{"name": "bJcG5wFzWYUQeGiXZFf4oVLYszw74OqMzJwuF-qkBOE="}
    2021-10-27T15:26:41.985-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreifiz3alz25gq7txk5jjvwrdha56chq75rfskorehsnkhg2u7wlvji", "size": 6}
    2021-10-27T15:26:41.985-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzacebn6le2na7eloclahpr4naz2pd4rwlowlzkrjf26sugwi5u4ebkc6", "key": "Q3goYsxfWBAzOUSdLAgrWjZ5M5jlrF76WQtL5I910G4="}
    2021-10-27T15:26:41.985-0400	DEBUG	wnfs	private/private.go:95	LoadRoot	{"hamtCID": "bafy2bzacebn6le2na7eloclahpr4naz2pd4rwlowlzkrjf26sugwi5u4ebkc6", "key": "Q3goYsxfWBAzOUSdLAgrWjZ5M5jlrF76WQtL5I910G4=", "name": "5e138f6c23438ca08814d4d12eb00414d95f0ba242217801f7004d7481f4128a"}
    2021-10-27T15:26:41.985-0400	DEBUG	wnfs	private/private.go:273	LoadTree	{"name": "", "cid": "bafkreifiz3alz25gq7txk5jjvwrdha56chq75rfskorehsnkhg2u7wlvji"}
    2021-10-27T15:26:41.985-0400	DEBUG	wnfs	private/private.go:381	Tree.Add	{"path": "hello.txt"}
    2021-10-27T15:26:41.985-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreieaujplcex3zm76jgrudvrn5v53jqg2k3sp7gyjcj4zm5a4sl5rvy", "key": "Ke671RvLDBKGSKW82XoIz0RyW1kxG7klaBzatlA90As="}
    2021-10-27T15:26:41.988-0400	DEBUG	wnfs	private/private.go:1069	File.Put	{"name": "hello.txt", "cid": "bafkreif5ruvydv5mvpwwpynaxkz7xvapkxjvfmtcye6dtdlnwgqcmbdzqa", "size": 30}
    2021-10-27T15:26:41.988-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 1}
    2021-10-27T15:26:41.988-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreiabq6fvhmcltepuf4rj4ouijckr4nra3sfffecbmvuhhhkhzmswbm", "size": 30}
    2021-10-27T15:26:41.988-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzacea44chuodgjjqnli4cvddzrobylfoj7vp37jn7wxmx5uik43wlmp6", "key": "SXqwk3k0o0l_qBZDR13p6uruO8B4DctHr0VuynSWJss="}
    2021-10-27T15:26:41.988-0400	DEBUG	wnfs	private/private.go:381	Tree.Add	{"path": "goodbye.txt"}
    2021-10-27T15:26:41.989-0400	DEBUG	wnfs	ratchet/store.go:44	writing ratchet	{"name": "iSOLafDzQHm5Y4dHVbQW3hEtkPnCu7Vc5Bm6TpEQVQI="}
    2021-10-27T15:26:41.989-0400	DEBUG	wnfs	private/private.go:1069	File.Put	{"name": "", "cid": "bafkreicb3mzuot7r2vds7jhhmaoq4mtzwwhijcpmk4xzxzds62anlvsm3u", "size": 8}
    2021-10-27T15:26:41.998-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 2}
    2021-10-27T15:26:41.999-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreicrqcrwep23tcxffwlrlf55eoodrb5akbisypnrwqozdxaamt2pcm", "size": 14}
    2021-10-27T15:26:41.999-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzacecpavqgi26vwmumvoturgdm7rsyczius2mobt7kh2yllgypqcga6o", "key": "SXqwk3k0o0l_qBZDR13p6uruO8B4DctHr0VuynSWJss="}
    2021-10-27T15:26:41.999-0400	DEBUG	wnfs	private/private.go:1199	Merge	{"acid": "bafkreicrqcrwep23tcxffwlrlf55eoodrb5akbisypnrwqozdxaamt2pcm", "bcid": "bafkreiabq6fvhmcltepuf4rj4ouijckr4nra3sfffecbmvuhhhkhzmswbm"}
    2021-10-27T15:26:42.000-0400	DEBUG	wnfs	private/private.go:730	merge root trees	{"local": "bafkreicrqcrwep23tcxffwlrlf55eoodrb5akbisypnrwqozdxaamt2pcm", "remote": "bafkreiabq6fvhmcltepuf4rj4ouijckr4nra3sfffecbmvuhhhkhzmswbm"}
    2021-10-27T15:26:42.000-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreieaujplcex3zm76jgrudvrn5v53jqg2k3sp7gyjcj4zm5a4sl5rvy", "key": "Ke671RvLDBKGSKW82XoIz0RyW1kxG7klaBzatlA90As="}
    2021-10-27T15:26:42.000-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreif5ruvydv5mvpwwpynaxkz7xvapkxjvfmtcye6dtdlnwgqcmbdzqa", "key": "Z5PgGhSwu5ZwzJ885r5e4oEn4kv8W4N7y7BSZq_ppsg="}
    2021-10-27T15:26:42.000-0400	DEBUG	wnfs	private/private.go:1199	Merge	{"acid": "bafkreieaujplcex3zm76jgrudvrn5v53jqg2k3sp7gyjcj4zm5a4sl5rvy", "bcid": "bafkreif5ruvydv5mvpwwpynaxkz7xvapkxjvfmtcye6dtdlnwgqcmbdzqa"}
    2021-10-27T15:26:42.000-0400	DEBUG	wnfs	private/private.go:808	adding link	{"link": {"Name":"hello.txt","Size":30,"Cid":{"/":"bafkreif5ruvydv5mvpwwpynaxkz7xvapkxjvfmtcye6dtdlnwgqcmbdzqa"},"IsFile":true,"Mtime":0,"Key":"Z5PgGhSwu5ZwzJ885r5e4oEn4kv8W4N7y7BSZq_ppsg=","Pointer":"9e96281ab35eed606bd627b22eda20a938262672baf1c7e0c0b7930f457320a3"}}
    2021-10-27T15:26:42.000-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 2}
    2021-10-27T15:26:42.000-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreiddqun7mahuygdhinj6f5pbzfuikvvqzaes2xfao77nptcy4euaia", "size": 38}
    2021-10-27T15:26:42.000-0400	DEBUG	wnfs	private/private.go:182	Root.Put	{"name": "", "hamtCID": "bafy2bzacecpavqgi26vwmumvoturgdm7rsyczius2mobt7kh2yllgypqcga6o", "key": "JbgwFn4kdEFsqK0OCSoMzhv02rzuSnk0Oan7b7IhoCk="}
    2021-10-27T15:26:42.000-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 2}
    2021-10-27T15:26:42.001-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreid2hy6nugovcpqjsnq6ueqbdumo6l6ear4smwyvherotimad6xnea", "size": 38}
    2021-10-27T15:26:42.001-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzaceacwlo22zrsw7xspn7ivdlcppnizjlwagpgg37o62m2o7apnc7at2", "key": "EcethZJTVN2VytYFiK2ypz4z6sChmMxDqNQXERCtY_0="}
    2021-10-27T15:26:42.002-0400	DEBUG	wnfs	private/private.go:95	LoadRoot	{"hamtCID": "bafy2bzaceacwlo22zrsw7xspn7ivdlcppnizjlwagpgg37o62m2o7apnc7at2", "key": "EcethZJTVN2VytYFiK2ypz4z6sChmMxDqNQXERCtY_0=", "name": "0ebcd1958e25ee8c696333c037a772c3b7c7d5b8b15a16b59231b71ab7eb0ccb"}
    2021-10-27T15:26:42.002-0400	DEBUG	wnfs	private/private.go:273	LoadTree	{"name": "", "cid": "bafkreid2hy6nugovcpqjsnq6ueqbdumo6l6ear4smwyvherotimad6xnea"}
    2021-10-27T15:26:42.002-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreif5ruvydv5mvpwwpynaxkz7xvapkxjvfmtcye6dtdlnwgqcmbdzqa", "key": "Z5PgGhSwu5ZwzJ885r5e4oEn4kv8W4N7y7BSZq_ppsg="}
    --- PASS: TestTreeMergeCommit (0.02s)
        --- PASS: TestTreeMergeCommit/remote_overwrites_local_file (0.02s)
    === RUN   TestTreeMergeCommit
    === RUN   TestTreeMergeCommit/remote_overwrites_local_file
    2021-10-27T15:26:42.003-0400	DEBUG	wnfs	private/private.go:381	Tree.Add	{"path": "hello.txt"}
    2021-10-27T15:26:42.003-0400	DEBUG	wnfs	ratchet/store.go:44	writing ratchet	{"name": "cVXwdal2kgfV7AQxtKnLJi-d5TR31UDxfJvVyWMC9WU="}
    2021-10-27T15:26:42.003-0400	DEBUG	wnfs	private/private.go:1069	File.Put	{"name": "", "cid": "bafkreiazx6irmwnnl5tpwivud5tlqbzm2r7fhqkr6hulsajvf6x2k7u5z4", "size": 6}
    2021-10-27T15:26:42.003-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 1}
    2021-10-27T15:26:42.003-0400	DEBUG	wnfs	ratchet/store.go:44	writing ratchet	{"name": "9KiHRlwoJFgwGUFdWcO7iODwqZWu7SvLcUN6eQdMuSo="}
    2021-10-27T15:26:42.003-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreidtl2km63f67ny5ybp3hinacm35isyu6u2yn3zqmpuemqe5bohwvu", "size": 6}
    2021-10-27T15:26:42.003-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzacedwui7xcbytsyox76mapcirbu37t6lx2c4g3xtagnxxjqenjy4kvc", "key": "_m6FpgZlX6gT14a3pVSIe1v0jdqPssppuEjB_j_v1AA="}
    2021-10-27T15:26:42.004-0400	DEBUG	wnfs	private/private.go:95	LoadRoot	{"hamtCID": "bafy2bzacedwui7xcbytsyox76mapcirbu37t6lx2c4g3xtagnxxjqenjy4kvc", "key": "_m6FpgZlX6gT14a3pVSIe1v0jdqPssppuEjB_j_v1AA=", "name": "c15272871c2fa8918afefbc0efe2b98c7069426ecb1496b1e86ed51ea6eb5e38"}
    2021-10-27T15:26:42.004-0400	DEBUG	wnfs	private/private.go:273	LoadTree	{"name": "", "cid": "bafkreidtl2km63f67ny5ybp3hinacm35isyu6u2yn3zqmpuemqe5bohwvu"}
    2021-10-27T15:26:42.004-0400	DEBUG	wnfs	private/private.go:381	Tree.Add	{"path": "hello.txt"}
    2021-10-27T15:26:42.004-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreiazx6irmwnnl5tpwivud5tlqbzm2r7fhqkr6hulsajvf6x2k7u5z4", "key": "XInJvKNoCQRDuzscpOPJNUHJ3nkzdxT0FbXJyAXYgTE="}
    2021-10-27T15:26:42.004-0400	DEBUG	wnfs	private/private.go:1069	File.Put	{"name": "hello.txt", "cid": "bafkreifljoqhgp6ey4escayc4vak6nvuhxr5sfwqqnv5xqgn5c6mrxdvge", "size": 30}
    2021-10-27T15:26:42.004-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 1}
    2021-10-27T15:26:42.005-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreicu3a53ngfydbghjcqdyozsyzxakrqikwtnrdl4rpa6baugcjhvm4", "size": 30}
    2021-10-27T15:26:42.005-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzacedmfiqxemv6ro7xoqb6krzaioibwfnrhmbggwgpeoesbx6oybgzno", "key": "WdhoI4krhMzuUjABVxtBM-7N3zbL7g1qBb3HvW6mzMc="}
    2021-10-27T15:26:42.005-0400	DEBUG	wnfs	private/private.go:381	Tree.Add	{"path": "goodbye.txt"}
    2021-10-27T15:26:42.006-0400	DEBUG	wnfs	ratchet/store.go:44	writing ratchet	{"name": "-I7IamQguuTqmbaA6Y4Cy9i1cKulEHMJCA7k_nGpSWM="}
    2021-10-27T15:26:42.006-0400	DEBUG	wnfs	private/private.go:1069	File.Put	{"name": "", "cid": "bafkreicgnewdjf4zdkytolb5jep7xiz4ue5wogbxlbe57eilwksdci47di", "size": 8}
    2021-10-27T15:26:42.006-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 2}
    2021-10-27T15:26:42.007-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreibgp6ivc2iv4lutj6uut2tfvxw6jls5p22634m7jgwweeipghgutm", "size": 14}
    2021-10-27T15:26:42.007-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzacebhi7tjy3dgrxjlzlb2zatgo63swfvjxeflwqn4wvetscedgho7xo", "key": "WdhoI4krhMzuUjABVxtBM-7N3zbL7g1qBb3HvW6mzMc="}
    2021-10-27T15:26:42.007-0400	DEBUG	wnfs	private/private.go:1199	Merge	{"acid": "bafkreibgp6ivc2iv4lutj6uut2tfvxw6jls5p22634m7jgwweeipghgutm", "bcid": "bafkreicu3a53ngfydbghjcqdyozsyzxakrqikwtnrdl4rpa6baugcjhvm4"}
    2021-10-27T15:26:42.007-0400	DEBUG	wnfs	private/private.go:730	merge root trees	{"local": "bafkreicu3a53ngfydbghjcqdyozsyzxakrqikwtnrdl4rpa6baugcjhvm4", "remote": "bafkreibgp6ivc2iv4lutj6uut2tfvxw6jls5p22634m7jgwweeipghgutm"}
    2021-10-27T15:26:42.007-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreifljoqhgp6ey4escayc4vak6nvuhxr5sfwqqnv5xqgn5c6mrxdvge", "key": "OAxjtUSYQ1rANUePrGNOY9NKCaQazm6HNbFjOi8slO4="}
    2021-10-27T15:26:42.007-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreiazx6irmwnnl5tpwivud5tlqbzm2r7fhqkr6hulsajvf6x2k7u5z4", "key": "XInJvKNoCQRDuzscpOPJNUHJ3nkzdxT0FbXJyAXYgTE="}
    2021-10-27T15:26:42.007-0400	DEBUG	wnfs	private/private.go:1199	Merge	{"acid": "bafkreifljoqhgp6ey4escayc4vak6nvuhxr5sfwqqnv5xqgn5c6mrxdvge", "bcid": "bafkreiazx6irmwnnl5tpwivud5tlqbzm2r7fhqkr6hulsajvf6x2k7u5z4"}
    2021-10-27T15:26:42.007-0400	DEBUG	wnfs	private/private.go:808	adding link	{"link": {"Name":"hello.txt","Size":30,"Cid":{"/":"bafkreifljoqhgp6ey4escayc4vak6nvuhxr5sfwqqnv5xqgn5c6mrxdvge"},"IsFile":true,"Mtime":0,"Key":"XInJvKNoCQRDuzscpOPJNUHJ3nkzdxT0FbXJyAXYgTE=","Pointer":"c148d01bb1d49a007fe7d52e64cc140eccbe16b8813f371244502757bbb0e0bd"}}
    2021-10-27T15:26:42.007-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 2}
    2021-10-27T15:26:42.008-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreiarbg7cvgywbmwwjf5bm2fktga3maaom2u3rbuadrxqr3wx5p7voq", "size": 38}
    2021-10-27T15:26:42.009-0400	DEBUG	wnfs	private/private.go:182	Root.Put	{"name": "", "hamtCID": "bafy2bzacedmfiqxemv6ro7xoqb6krzaioibwfnrhmbggwgpeoesbx6oybgzno", "key": "U7DWVuYfg-oQzjboyU_IzVIFOLN7HjtiygyuikubVmE="}
    2021-10-27T15:26:42.009-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 2}
    2021-10-27T15:26:42.010-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreifvmlrt4td2hxf2henvrck5j6gnfcuvlusrbfzocs7fjoceif7dgi", "size": 38}
    2021-10-27T15:26:42.010-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzaceblwqjisc6eheuggf22cx4fkamu3jokfcnyg4lirx6a3ja2qcr53y", "key": "MG4W_6cFWQjwP8mBDkaSNc3MDoWjkcSDflzOH_yPgKo="}
    2021-10-27T15:26:42.010-0400	DEBUG	wnfs	private/private.go:95	LoadRoot	{"hamtCID": "bafy2bzaceblwqjisc6eheuggf22cx4fkamu3jokfcnyg4lirx6a3ja2qcr53y", "key": "MG4W_6cFWQjwP8mBDkaSNc3MDoWjkcSDflzOH_yPgKo=", "name": "418055cba80b74a84de02d5e4b3f7f5057f913e919dc91721bdea9752a17057b"}
    2021-10-27T15:26:42.010-0400	DEBUG	wnfs	private/private.go:273	LoadTree	{"name": "", "cid": "bafkreifvmlrt4td2hxf2henvrck5j6gnfcuvlusrbfzocs7fjoceif7dgi"}
    2021-10-27T15:26:42.010-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreifljoqhgp6ey4escayc4vak6nvuhxr5sfwqqnv5xqgn5c6mrxdvge", "key": "XInJvKNoCQRDuzscpOPJNUHJ3nkzdxT0FbXJyAXYgTE="}
    2021-10-27T15:26:42.010-0400	DEBUG	wnfs	private/private.go:896	LoadFileFromCID	{"err": "cipher: message authentication failed"}
        private_test.go:204:
            	Error Trace:	private_test.go:414
            	            				private_test.go:204
            	Error:      	Expected nil, but got: &fmt.wrapError{msg:"decoding s-node \"hello.txt\" header: cipher: message authentication failed", err:(*errors.errorString)(0xc0000583d0)}
            	Test:       	TestTreeMergeCommit/remote_overwrites_local_file
    --- FAIL: TestTreeMergeCommit (0.01s)
        --- FAIL: TestTreeMergeCommit/remote_overwrites_local_file (0.01s)
    === RUN   TestTreeMergeCommit
    === RUN   TestTreeMergeCommit/remote_overwrites_local_file
    2021-10-27T15:26:42.011-0400	DEBUG	wnfs	private/private.go:381	Tree.Add	{"path": "hello.txt"}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	ratchet/store.go:44	writing ratchet	{"name": "1VBjFoIl05PeBoNvhDHPKgJoSmVmbk_9Xwi4YuOcca8="}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	private/private.go:1069	File.Put	{"name": "", "cid": "bafkreihxwzf7a4aml5ve4wmhex62hjg5zd3dwvrjndo6r52id7i2ok7you", "size": 6}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 1}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	ratchet/store.go:44	writing ratchet	{"name": "cH6Qz63haopYMCLJUJSyzFc2Z4fK5DBmlulRlfefP5M="}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreig3zgr4tl56nohhs2sx24wqitbo2i57a3z5cebo24wguwizamhhg4", "size": 6}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzacecrnkmokdfxf7dafen3rnnrq6xjqln7sx4kvfppk7rfqqrpzmues2", "key": "jdYSbBIrWucqN0q90bLV9kKWaZL7tFY5BvAH59JCYuY="}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	private/private.go:95	LoadRoot	{"hamtCID": "bafy2bzacecrnkmokdfxf7dafen3rnnrq6xjqln7sx4kvfppk7rfqqrpzmues2", "key": "jdYSbBIrWucqN0q90bLV9kKWaZL7tFY5BvAH59JCYuY=", "name": "7a6debf5ba013ccbb1f8d81caef03ce348d974d5e947163aa56f827668e3bfcb"}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	private/private.go:273	LoadTree	{"name": "", "cid": "bafkreig3zgr4tl56nohhs2sx24wqitbo2i57a3z5cebo24wguwizamhhg4"}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	private/private.go:381	Tree.Add	{"path": "hello.txt"}
    2021-10-27T15:26:42.013-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreihxwzf7a4aml5ve4wmhex62hjg5zd3dwvrjndo6r52id7i2ok7you", "key": "Ke7TBGxKIp-biG5gPhSUfABep_PRKSlHumjx4yQoLZ4="}
    2021-10-27T15:26:42.014-0400	DEBUG	wnfs	private/private.go:1069	File.Put	{"name": "hello.txt", "cid": "bafkreigtqu6nuzz6jduzcxk6qfjo4xsebo2kjsqpkp5uscvtgsl6gjxjuq", "size": 30}
    2021-10-27T15:26:42.014-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 1}
    2021-10-27T15:26:42.015-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreiazzdybx5jggfhgd54okj7rn5hpc2wboojrbkemzschle7aai2nxi", "size": 30}
    2021-10-27T15:26:42.015-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzacebvfwg7rabqo6yzjt32vrfi4wc5clxoq3x6me37va7qn3vt5pmp4g", "key": "Ggp7405rpVTTR78N-WWh9_WmYxq-KlgHL58Ho4HhyG8="}
    2021-10-27T15:26:42.015-0400	DEBUG	wnfs	private/private.go:381	Tree.Add	{"path": "goodbye.txt"}
    2021-10-27T15:26:42.017-0400	DEBUG	wnfs	ratchet/store.go:44	writing ratchet	{"name": "ZcPa5BGMWhBuirRu52BftEW2CZKofthBBrsAsxmDGac="}
    2021-10-27T15:26:42.017-0400	DEBUG	wnfs	private/private.go:1069	File.Put	{"name": "", "cid": "bafkreihkzp6tmqdjdd3hsqxlc7vusv7vtye6gduld5gah53z4fzv7boepu", "size": 8}
    2021-10-27T15:26:42.017-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 2}
    2021-10-27T15:26:42.018-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreigw354nactdfyjftlybzmjyjg7d3hw7hxhb3cymz3o4fyalfojfyy", "size": 14}
    2021-10-27T15:26:42.018-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzaceaxwjtgq6khm246aieupn7y7y4o6ptogmsxynnkqsnbvnlyinote4", "key": "Ggp7405rpVTTR78N-WWh9_WmYxq-KlgHL58Ho4HhyG8="}
    2021-10-27T15:26:42.018-0400	DEBUG	wnfs	private/private.go:1199	Merge	{"acid": "bafkreigw354nactdfyjftlybzmjyjg7d3hw7hxhb3cymz3o4fyalfojfyy", "bcid": "bafkreiazzdybx5jggfhgd54okj7rn5hpc2wboojrbkemzschle7aai2nxi"}
    2021-10-27T15:26:42.018-0400	DEBUG	wnfs	private/private.go:730	merge root trees	{"local": "bafkreiazzdybx5jggfhgd54okj7rn5hpc2wboojrbkemzschle7aai2nxi", "remote": "bafkreigw354nactdfyjftlybzmjyjg7d3hw7hxhb3cymz3o4fyalfojfyy"}
    2021-10-27T15:26:42.018-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreigtqu6nuzz6jduzcxk6qfjo4xsebo2kjsqpkp5uscvtgsl6gjxjuq", "key": "ryQuSigsS7aim3D78QlupjM-o1dGXDnQOoERsCFBGcI="}
    2021-10-27T15:26:42.018-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreihxwzf7a4aml5ve4wmhex62hjg5zd3dwvrjndo6r52id7i2ok7you", "key": "Ke7TBGxKIp-biG5gPhSUfABep_PRKSlHumjx4yQoLZ4="}
    2021-10-27T15:26:42.018-0400	DEBUG	wnfs	private/private.go:1199	Merge	{"acid": "bafkreigtqu6nuzz6jduzcxk6qfjo4xsebo2kjsqpkp5uscvtgsl6gjxjuq", "bcid": "bafkreihxwzf7a4aml5ve4wmhex62hjg5zd3dwvrjndo6r52id7i2ok7you"}
    2021-10-27T15:26:42.018-0400	DEBUG	wnfs	private/private.go:808	adding link	{"link": {"Name":"hello.txt","Size":30,"Cid":{"/":"bafkreigtqu6nuzz6jduzcxk6qfjo4xsebo2kjsqpkp5uscvtgsl6gjxjuq"},"IsFile":true,"Mtime":0,"Key":"Ke7TBGxKIp-biG5gPhSUfABep_PRKSlHumjx4yQoLZ4=","Pointer":"7dc91dc5086aeb883488edbd7d7edb4427c4e279d26231e01b5b734e07985d02"}}
    2021-10-27T15:26:42.018-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 2}
    2021-10-27T15:26:42.019-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreidechurnsamsyzjwblzvwd56q5nyuygu2qrmfer3os56rxqfc3ewi", "size": 38}
    2021-10-27T15:26:42.019-0400	DEBUG	wnfs	private/private.go:182	Root.Put	{"name": "", "hamtCID": "bafy2bzacebvfwg7rabqo6yzjt32vrfi4wc5clxoq3x6me37va7qn3vt5pmp4g", "key": "5CbrTs4zKCmMPfVMACMN_Y9mXgatRfaXYeaHY3fmrvk="}
    2021-10-27T15:26:42.019-0400	DEBUG	wnfs	private/private.go:674	Tree.Put	{"name": "", "len(links)": 2}
    2021-10-27T15:26:42.019-0400	DEBUG	wnfs	private/private.go:706	Tree.Put	{"name": "", "cid": "bafkreibgn5uwlp5nvf3cbrvpupilpugz4kxanfjeurvxalfy52jqsmotte", "size": 38}
    2021-10-27T15:26:42.019-0400	DEBUG	wnfs	private/private.go:198	putRoot	{"name": "", "hamtCID": "bafy2bzacedlkpsctqxtsw6qntgxehqgdl4umk3iwq6fmpxb5zue5xvsxxiay4", "key": "DyglLunw9l55WGj8S4PQQxfixMc1gLxeUuTvk3uX9-k="}
    2021-10-27T15:26:42.019-0400	DEBUG	wnfs	private/private.go:95	LoadRoot	{"hamtCID": "bafy2bzacedlkpsctqxtsw6qntgxehqgdl4umk3iwq6fmpxb5zue5xvsxxiay4", "key": "DyglLunw9l55WGj8S4PQQxfixMc1gLxeUuTvk3uX9-k=", "name": "71db2288f9ba4baf2d3d8a29462a297f4c087ddd39c8787bdb73137dcbc4bd47"}
    2021-10-27T15:26:42.021-0400	DEBUG	wnfs	private/private.go:273	LoadTree	{"name": "", "cid": "bafkreibgn5uwlp5nvf3cbrvpupilpugz4kxanfjeurvxalfy52jqsmotte"}
    2021-10-27T15:26:42.022-0400	DEBUG	wnfs	private/private.go:886	LoadFileFromCID	{"name": "hello.txt", "cid": "bafkreigtqu6nuzz6jduzcxk6qfjo4xsebo2kjsqpkp5uscvtgsl6gjxjuq", "key": "Ke7TBGxKIp-biG5gPhSUfABep_PRKSlHumjx4yQoLZ4="}
    2021-10-27T15:26:42.022-0400	DEBUG	wnfs	private/private.go:896	LoadFileFromCID	{"err": "cipher: message authentication failed"}
        private_test.go:204:
            	Error Trace:	private_test.go:414
            	            				private_test.go:204
            	Error:      	Expected nil, but got: &fmt.wrapError{msg:"decoding s-node \"hello.txt\" header: cipher: message authentication failed", err:(*errors.errorString)(0xc0000583d0)}
            	Test:       	TestTreeMergeCommit/remote_overwrites_local_file
    --- FAIL: TestTreeMergeCommit (0.01s)
        --- FAIL: TestTreeMergeCommit/remote_overwrites_local_file (0.01s)
    FAIL
    exit status 1
    FAIL	github.com/qri-io/wnfs-go/private	0.484s
    
  • Near-Term Roadmapping

    Near-Term Roadmapping

    Features

    • [x] Remove go-ipfs dependency
    • [x] Switch ratchet to SHA3
    • [x] Fix math & tests for Spiral Ratchet
    • [x] Implement/Adapt xxHash Bloom filter
    • [x] Public Tree Merge
    • [x] Private Tree History
    • [x] Private Tree Merge
    • [x] Root Tree Merge
    • [x] Public Tree Diff
    • [ ] Cipherchunk Monotonic Nonces
    • [ ] Pretty Directory

    Refactoring

    • [ ] in-library file size calculations
    • [ ] switch completely to dag-cbor
    • [x] dag-cbor codec for header blocks
    • [ ] base.Node interface: harmonize history, filesystem accessor methods

    Tests

    • [ ] public file size calculations
    • [ ] whole-filesystem fuzz test

    Interop Tests (TS & Go)

    Both implementations should be fed the same values to these low-level building blocks & return the same result:

    • [ ] Spiral ratchet
    • [ ] Bloom Filter

    Specs

    • [ ] Discussion of SHA3 performance in-browser
    • [ ] writeup of rationale for SHA3 choice that includes SHA2+extensions consideration
    • [ ] Cipherchunk Monotonic Nonces, implications for IPFS filestore

    Documentation

    • [ ] library passes golint
    • [ ] establish internal pacakge

    Chores

    • [ ] gometalinter github action
    • [ ] codecov github action
Related tags
Golang client for NATS, the cloud native messaging system.

NATS - Go Client A Go client for the NATS messaging system. Installation # Go client go get github.com/nats-io/nats.go/ # Server go get github.com/na

Jan 4, 2023
Package rsync contains a native Go rsync implementation.

gokrazy rsync Package rsync contains a native Go rsync implementation. ⚠ Beware: very fresh. Might eat your data. You have been warned! ⚠ The only com

Jan 2, 2023
Tapestry is an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate objects. This distributed system provides an interface for storing and retrieving key-value pairs.

Tapestry This project implements Tapestry, an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate

Mar 16, 2022
View, edit, and save text files via http to the file system.

go-wiki View, edit, and save text files via http to the file system. (DONE) https://golang.org/doc/articles/wiki/ Instructions go run main.go In a web

Nov 25, 2021
Fake server, Consumer Driven Contracts and help with testing performance from one configuration file with zero system dependencies and no coding whatsoever
Fake server, Consumer Driven Contracts and help with testing performance from one configuration file with zero system dependencies and no coding whatsoever

mockingjay server Mockingjay lets you define the contract between a consumer and producer and with just a configuration file you get: A fast to launch

Jan 6, 2023
Generates file.key file for IPFS Private Network.

ipfs-keygen Generates file.key file for IPFS Private Network. Installation go get -u github.com/reixmor/ipfs-keygen/ipfs-keygen Usage ipfs-keygen > ~/

Jan 18, 2022
gTunnel fork with Tasks implementation: Cmd+File Transfers+TCPScanner

A TCP tunneling suite built with golang and gRPC. gTunnel can manage multiple forward and reverse tunnels that are all carried over a single TCP/HTTP2 connection. I wanted to learn a new language, so I picked go and gRPC. Client executables have been tested on windows and linux.

Dec 30, 2022
🌐 (Web 3.0) Pastebin built on IPFS, securely served by Distributed Web and Edge Network.
🌐 (Web 3.0) Pastebin built on IPFS, securely served by Distributed Web and Edge Network.

pastebin-ipfs įŽ€äŊ“中文 (IPFS Archivists) Still in development, Pull Requests are welcomed. Pastebin built on IPFS, securely served by Distributed Web and

Jan 1, 2023
GOWS is GoLang web-socket module Provides you with ease of handling web socket connections with a few lines

GOWS GOWS is GoLang web-socket module Provides you with ease of handling web socket connections with a few lines, it supports multi-connection on one

Apr 4, 2022
Go-web-dev - Web Development With Google's Golang Programming Language Code Snippet and Exercises.

Web Development With Google's Golang Programming Language Code Snippet and Exercises This repository contains my code snippets, hands on exercises and

Jan 16, 2022
gRPC Web implementation for Golang and TypeScript

gRPC-Web: Typed Frontend Development gRPC is a modern, HTTP2-based protocol, that provides RPC semantics using the strongly-typed binary data format o

Dec 29, 2022
A cloud native distributed streaming network telemetry.
A cloud native distributed streaming network telemetry.

Panoptes Streaming Panoptes Streaming is a cloud native distributed streaming network telemetry. It can be installed as a single binary or clustered n

Sep 27, 2022
A simple TUN/TAP library written in native Go.

water water is a native Go library for TUN/TAP interfaces. water is designed to be simple and efficient. It wraps almost only syscalls and uses only G

Jan 7, 2023
The Cloud Native Application Proxy
The Cloud Native Application Proxy

Traefik (pronounced traffic) is a modern HTTP reverse proxy and load balancer that makes deploying microservices easy. Traefik integrates with your ex

Dec 30, 2022
🤘 The native golang ssh client to execute your commands over ssh connection. 🚀🚀
🤘 The native golang ssh client to execute your commands over ssh connection. 🚀🚀

Golang SSH Client. Fast and easy golang ssh client module. Goph is a lightweight Go SSH client focusing on simplicity! Installation ❘ Features ❘ Usage

Dec 24, 2022
grobotstxt is a native Go port of Google's robots.txt parser and matcher library.

grobotstxt grobotstxt is a native Go port of Google's robots.txt parser and matcher C++ library. Direct function-for-function conversion/port Preserve

Dec 27, 2022
MOSN is a cloud native proxy for edge or service mesh. https://mosn.io
MOSN is a cloud native proxy for edge or service mesh. https://mosn.io

中文 MOSN is a network proxy written in Golang. It can be used as a cloud-native network data plane, providing services with the following proxy functio

Dec 30, 2022
Cloud Native Tunnel
Cloud Native Tunnel

inlets is a Cloud Native Tunnel written in Go Expose your local endpoints to the Internet or within a remote network, without touching firewalls. Foll

Jan 4, 2022
A native Thrift package for Go

Thrift Package for Go API Documentation: http://godoc.org/github.com/samuel/go-thrift License 3-clause BSD. See LICENSE file. Overview Thrift is an ID

Nov 22, 2022