Tapestry is an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate objects. This distributed system provides an interface for storing and retrieving key-value pairs.

Tapestry

This project implements Tapestry, an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate objects. This distributed system provides an interface for storing and retrieving key-value pairs. From an application’s perspective, the application chooses where to store data, rather than allowing the system to choose a node to store the object at.

Tapestry is a decentralized distributed system, where each node serves as both an object store and a router that applications can contact to obtain objects. In a Tapestry network, objects are “published” at nodes, and once an object has been successfully published, it is possible for any other node in the network to find the location at which that object is published.

High-Level Design Choice

Node Join: When a node trying to join the tapestry, the node first finds the shared prefix of ID and multicast to the existing node sharing the prefix. These nodes will add the new node to their routing table. Then new node will get closest neighbors to fill its own routing table.

Node Leave: When a node leave the network, the node will notify its leaving and try to transfer the replacement node when traversing its own routing table. Objects stored at leaving node will be redistributed.

Interesting Improvement

We made some part of our code in goroutine, which will improve the performance of our project. For the test, we developed some util functions that can make testing much easier.

Test For Tapestry

We have put a lot of effort on the tests for this project. Thus, we have written lots of unit tests. Below are the test we have written. The unit tests concentrate on the four main file: node_init, node_core, node_exit and routing_table . Beside the unit tests, we also implement some testing_utils functions that helping the testing of the project.

node_init_test.go

  • TestNodeIDAndAddress

    This test tests about the Node ID and Addr function.

  • TestMulticast1

    This test tests about Multicast, especially about data transfer in multicast.

  • TestMulticast2

    This test tests about Multicast, especially about bad node removed from routing table.

  • TestMulticast3

    This test tests about Multicast, especially about bad node not in neighbor.

  • TestAddRoute1

    This test tests about AddRoute, especially about adding route to routing table.

  • TestAddRoute2

    This test tests about AddRoute, especially about replacing a node in routing table.

  • TestAddRoute3

    This test tests about AddRoute, especially about not replacing a node in routing table, because new node is further.

node_core_test.go

  • TestNodeGetStoreRemove

    This test tests about Node Get Store Remove, especially when these operation between remote nodes.

  • TestFindRootOnRemoteNode1

    This test tests about FindRoot for deleted node, which will return the following node in the routing table.

  • TestFindRootOnRemoteNode1_BadNode

    This test tests about FindRootOnRemoteNode, especially when find root with one better bad node, the result should returns self.

  • TestFindRootOnRemoteNode2_BadNode

    This test tests about FindRootOnRemoteNode, especially when find root on a bad node.

  • TestPublish

    This test tests about Publish which registers object on the correct root node

node_exit_test.go

  • TestLeave1_SafeLeave

    This test tests about node leave the tapestry, especially when leaving in a graceful way

  • TestLeave2_UnsafeLeave

    This test tests about node leave the tapestry, especially when leaving in an unsafe way

  • TestNotifyLeave1_WithoutReplacement

    This test tests about NotifyLeave, especially when leaving without a replacement node

  • TestNotifyLeave2_Replacement

    This test tests about NotifyLeave, especially when leaving with a replacement node

  • TestNotifyLeave2_WithReplacement

    This test tests about Leave, especially when gracefully leaving with a replacement node

routing_table_test.go

  • TestRoutingTable_Add

    This test tests about Add and Remove node in routing table, especially when function called on remote node

Test Coverage

node_init.go: 85.5%

node_core.go: 85.1%

node_exit.go: 95.7%

routing_table.go: 96.5%

Reference

https://sites.cs.ucsb.edu/~ravenben/classes/papers/tapestry-jsac04.pdf

Similar Resources

An IPFS bytes exchange for caching and retrieving data from Filecoin

🐸 go-hop-exchange An IPFS bytes exchange to allow any IPFS node to become a Filecoin retrieval provider and retrieve content from Filecoin Highlights

Aug 25, 2022

Jazigo is a tool written in Go for retrieving configuration for multiple devices, similar to rancid, fetchconfig, oxidized, Sweet.

Table of Contents About Jazigo Supported Platforms Features Requirements Quick Start - Short version Quick Start - Detailed version Global Settings Im

Jan 5, 2023

A simple tool for retrieving a request's IP address on the server.

reqip A simple tool for retrieving a request's IP address on the server. Inspired from request-ip Installation Via go get go get github.com/mo7zayed/r

Oct 26, 2022

The canonical location of the Clusternet API definition.

apis Schema of the external API types that are served by Clusternet. Purpose This library is the canonical location of the Clusternet API definition.

May 3, 2022

🌕 Server application for storing doujinshi, manga, art collections and other galleries with API and user control. Written in Go.

🌕 Server application for storing doujinshi, manga, art collections and other galleries with API and user control. Written in Go.

Dec 31, 2022

Traefik proxy plugin to extract HTTP header value and create a new header with extracted value

Copy header value Traefik plugin Traefik plugin that copies HTTP header value with format key1=value1; key2=value2 into a new header. Motivation for t

May 26, 2022

Test ssh login key acceptance without having the private key

ssh-key-confirmer This confirms if a SSH public key is listed as a authorized_key on a system Usage $ ssh-key-confirmer -i ./id_rsa.pub [email protected]

Dec 30, 2022
A simple distributed key-value store by using hashicorp/raft

raftkv This repository holds a simple distributed key-value store by using hashicorp/raft. raftkv provides gRPC and HTTP APIs. Please take a look API

Nov 30, 2022
Capture packet request/response pairs for a port and/or IP to aid in Network protocol based Nuclei Templates creation.

network-fingerprint Capture packet request/response pairs for a port and/or IP to aid in Network protocol based Nuclei Templates creation. Resources I

Nov 15, 2022
Go-ipfs-pinner - The pinner system is responsible for keeping track of which objects a user wants to keep stored locally

go-ipfs-pinner Background The pinner system is responsible for keeping track of

Jan 18, 2022
In memory Key/Value store in go using gRPC.
In memory Key/Value store in go using gRPC.

In memory cache, using gRPC Contents About Running Server Local Docker Kubernetes Example Helm Terraform API Add Get GetByPrefix GetAllItems DeleteKey

Dec 26, 2022
red-tldr is a lightweight text search tool, which is used to help red team staff quickly find the commands and key points they want to execute, so it is more suitable for use by red team personnel with certain experience.
red-tldr is a lightweight text search tool, which is used to help red team staff quickly find the commands and key points they want to execute, so it is more suitable for use by red team personnel with certain experience.

Red Team TL;DR English | 中文简体 What is Red Team TL;DR ? red-tldr is a lightweight text search tool, which is used to help red team staff quickly find t

Jan 5, 2023
OpenDILab RL Object Store

Introduction Decision AI Store Installation Prerequisites Linux Python >= 3.6 pip install . Quick Start Start Etcd Server di_store etcd_server ./conf/

Jan 6, 2023
httpstream provides HTTP handlers for simultaneous streaming uploads and downloads of objects, as well as persistence and a standalone server.

httpfstream httpfstream provides HTTP handlers for simultaneous streaming uploads and downloads of files, as well as persistence and a standalone serv

May 1, 2021
A small tool used to correspond to the IP address according to the name, id, and network alias of the docker container, which can be run as a DNS server

A small tool used to correspond to the IP address according to the name, id, and network alias of the docker container, which can be run as a DNS server

Apr 4, 2022
A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection.
A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection.

aScan A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection. Fu

Aug 15, 2022
A simple server which can be used as an RPC endpoint in popular Ethereum wallets.

RPC Endpoint This repository contains code for a simple server which can be used as an RPC endpoint in popular Ethereum wallets. The endpoint is https

Jan 2, 2023