ICPP 2022 (pap351) Under Review

SciCoFK: A Decentralized Scientific Computing Framework Embraced with Consensus and Incentive Mechanism

This repo is the primary code of SciCoFK. In this research work, we propose a novel decentralized computing framework SciCoFK to address collaborative scientific computing tasks, especially for large-scale matrix multiplication, by adopting consensus, incentive and verification mechanisms.

Requirements

Platform

  • Linux server
  • Intel CPUs
  • Nvidia GPUs

Required Packages

  • Latest Go programming Language
  • Latest Intel OneAPI HPC Toolkit
  • Latest Nvidia CUDA Toolkit

Function Demonstration of Each Directory

SciCoFK
├── Proof_of_Real_Contribution
│   
├── matrix_multiplication_OpenMP
│
├── matrix_multiplication_cuda
│
├── matrix_multiplication_multi_GPUs
│
└── matrix_multiplication_verification
  1. The codes in Proof_of_Real_Contribution implement the consensus mechanism — Proof-of-Real-Contribution.
  2. The codes in matrix_multiplication_OpenMP implement the OpenMP-optimized matrix multiplication.
  3. The codes in matrix_multiplication_cuda implement the CUDA-optimized matrix multiplication running on one GPU.
  4. The codes in matrix_multiplication_multi_GPUs implement the CUDA-optimized matrix multiplication running on multiple GPU.
  5. The codes in matrix_multiplication_verification implement the verification mechanism — GVFA — for validating matrix multiplication.

Running of Each Function

  1. The way of running PoRC consensus mechanism:
Proof_of_Real_Contribution
├── WinnerInEachRound.txt
├── count.sh
├── go.mod
├── main.go
├── principal
│   └── principal.go
├── system
│   └── system.go
├── task
│   └── task.go
└── woker
    ├── multiplier.go
    └── worker.go
  1. Modify the values of variable numberOfComputingWorkers and numberOfTasks in file main.go to specify the number of workers and tasks for each consensus round.
  2. Modify the values of variable num in file principal/principal.go to specify the dimension of square matrices.
  3. Run the program:
    $ ./count.sh numberofConsensusRounds
    
    The outputs show the number of bookking rights that each worker obtains.
  4. The winner ID of each consensus round is recorded in file WinnerInEachRound.txt.
  1. The way of running OpenMP-optimized matrix multiplication program:
matrix_multiplication_OpenMP
├── bin
│   └── run
├── clean.sh
├── data
├── include
│   └── matrix.h
├── make
├── run.sh
└── src
    ├── change_C_output.c
    ├── main.c
    └── matrix.c
$ ./make
$ ./run.sh firstMatrixDimension secondMatrixDimension

The generated matrices is stored in data directory. The output shows the running time.

  1. The way of running CUDA-optimized matrix multiplication program running on one GPU:
matrix_multiplication_cuda
├── bin
│   └── run
├── data
├── include
│   └── matrix.h
├── makefile
├── run.sh
└── src
    ├── main.cu
    └── matrix.cu
$ make
$ ./run.sh numberOfRowsofFirstMatrix numberOfColumnsofFirstMatrix numberOfColumnsofSecondMatrix

The generated matrices is stored in data directory. The output shows the running time.

  1. The way of running CUDA-optimized matrix multiplication program running on multiple GPU:
matrix_multiplication_multi_GPUs
├── bin
│   └── run
├── data
├── include
│   └── matrix.h
├── makefile
├── run.sh
└── src
    └── main.cu
$ make
$ ./run.sh numberOfRowsofFirstMatrix numberOfColumnsofFirstMatrix numberOfColumnsofSecondMatrix

The generated matrices is stored in data directory. The output shows the running time.

  1. The way of running the program of GVFA verification mechanism:
matrix_multiplication_verification
├── bin
│   ├── change
│   └── run
├── clean.sh
├── data
├── include
│   └── matrix.h
├── make
├── run.sh
├── src
│   ├── change_C_output.c
│   ├── freivald.c
│   ├── isProduct.c
│   └── main.c
└── transfer_data.sh
  1. Copying matrices data from above three matrix multiplication programs to data directory by running script transfer_data.sh
$ ./transfer_data.sh 1/2/3
  1. Executing verification for matrix multiplication
$ ./make
$ ./run.sh numberOfRowsofFirstMatrix numberOfColumnsofFirstMatrix numberOfColumnsofSecondMatrix

Note: the value of three parameters in above command must be equal to the dimension of matrices in data directory. The outputs show if the result of matrix multiplication is correct and running time.

  1. The funciton of src/change_C_output.c is to produce wrong result matrices.
Owner
This gitHub account provides anonymous source codes to support submitted papers.
null
Similar Resources

Reverse Shell in Golang and PowerShell Fud (27/04/2022)

ShellPwnsh Spanish: Generador de Backdoor en Golang, usando Reverse Shell en PowerShell, con formato de variables en el codigo para Bypass de AMSI y c

Nov 9, 2022

Higher Order Functions using Golang Generics (Hack Days 2022)

hoff: Higher Order Functions (and Friends) Golang 1.18+ implementations of common methods/data structures using Go Generics Requirements Go 1.18 or ne

Jan 4, 2023

Simple webhook to block exploitation of CVE-2022-0811

webhook-cve-2022-0811 This is a really simple webhook that just blocks pod creation if malicious sysctl values are configured. Build go test CGO_ENABL

Nov 9, 2022

🐶 Automated code review tool integrated with any code analysis tools regardless of programming language

🐶 Automated code review tool integrated with any code analysis tools regardless of programming language

reviewdog - A code review dog who keeps your codebase healthy. reviewdog provides a way to post review comments to code hosting service, such as GitHu

Jan 2, 2023

A Golang tool that does static analysis, unit testing, code review and generate code quality report.

A Golang tool that does static analysis, unit testing, code review and generate code quality report.

goreporter A Golang tool that does static analysis, unit testing, code review and generate code quality report. This is a tool that concurrently runs

Jan 8, 2023

GitHub Review Stats

gh-review-stats - GitHub Review Stats A command line tool for examining review statistics for GitHub repositories. Installing Download a pre-built bin

Jul 21, 2022

A command-line to create a pull request to review the entire content of a Github repository.

Pull Request Me Pull Request Me (PRMe) creates a pull request for the entire content of a Github repository. This is useful to solicit review comments

Nov 2, 2021

🔎 Help find Trojan Source vulnerability in code 👀 . Useful for code review in project with multiple collaborators

TrojanSourceFinder TrojanSourceFinder helps developers detect "Trojan Source" vulnerability in source code. Trojan Source vulnerability allows an atta

Nov 9, 2022

Send Slack reports of pull requests pending review

Send Slack reports of pull requests pending review

preport Born out of a desire to get pull requests reviewed faster without having to send manual reminders, preport generates reports of GitLab pull re

Nov 23, 2021

🐶 Automated code review tool integrated with any code analysis tools regardless of programming language

🐶 Automated code review tool integrated with any code analysis tools regardless of programming language

reviewdog - A code review dog who keeps your codebase healthy. reviewdog provides a way to post review comments to code hosting service, such as GitHu

Jan 7, 2023

Snitch - a Slack bot that fetches pending review PRs built in Go

Snitch - a Slack bot that fetches pending review PRs built in Go

Snitch Snitch is a Slack bot that fetches and publish PRs with pendings reviews

Nov 29, 2022

Sion code for review.

Sion Sion is an elastic, high-performance, cost-effective cloud storage that is built atop ephemeral cloud funtions. Prepare EC2 Proxy Amazon EC2 AMI:

Dec 5, 2022

Easy creation of review tasks for the Jira with a Discord notifications.

easy-jira-task-review Easy creation of review tasks for the Jira with a Discord notifications. Any ideas and help are welcome Installation Install go

Dec 1, 2022

Github cli extension to approve or reject pending deployments that are waiting for review.

gh-deploy A gh cli extension to approve or reject pending deployments that are waiting for review. Installation Installation requires a minimum versio

Nov 2, 2022

100% type-safe ORM for Go (Golang) with code generation and MySQL, PostgreSQL, Sqlite3, SQL Server support. GORM under the hood.

go-queryset 100% type-safe ORM for Go (Golang) with code generation and MySQL, PostgreSQL, Sqlite3, SQL Server support. GORM under the hood. Contents

Dec 30, 2022

dockin ops is a project used to handle the exec request for kubernetes under supervision

dockin ops is a project used to handle the exec request for kubernetes under supervision

Dockin Ops - Dockin Operation service English | 中文 Dockin operation and maintenance management system is a safe operation and maintenance management s

Aug 12, 2022

A Kubernetes Native Batch System (Project under CNCF)

A Kubernetes Native Batch System (Project under CNCF)

Volcano is a batch system built on Kubernetes. It provides a suite of mechanisms that are commonly required by many classes of batch & elastic workloa

Jan 9, 2023

Kubernetes Native Edge Computing Framework (project under CNCF)

Kubernetes Native Edge Computing Framework (project under CNCF)

KubeEdge KubeEdge is built upon Kubernetes and extends native containerized application orchestration and device management to hosts at the Edge. It c

Jan 1, 2023
Year-end-review - enables those want to look back on PRs at the end of year to review PRs and the comments as single Markdown file.

year-end-review year-end-review enables those want to look back on PRs at the end of year to review PRs and the comments as single Markdown file. HOW

Dec 31, 2021
Zero - If Google Drive says that 1 is under copyright, 0 must be under copyleft

zero Zero under copyleft license Google Drive's copyright detector says that fil

May 16, 2022
Abfahrt! Der InformatiCup 2022
Abfahrt! Der InformatiCup 2022

informatiCup 2022 Die Gesellschaft für Informatik veranstaltet den informatiCup - im Jahr 2022 bereits im 17. Jahr in Folge. Für den informatiCup 2022

Dec 4, 2022
Lottery program for Go Conference 2022 Spring Online

2022-Spring-Sponsors How to use Dump company names Dump company names into below files. cmd/gentest/_companies/platinum_gold.txt cmd/gentest/_companie

Jan 11, 2022
DevOps Roadmap 2022

Want to learn DevOps the right way in 2022 ? You have come to the right place I have created the complete DevOps roadmap that anyone can follow and be

Dec 28, 2022
Spotify Backend Developer Intern Challenge 2022 (Go, Gin, SQLite3)

Shopify Backend Developer Intern Challenge - Summer 2022 This is an API for managing inventory items. The API is written in Go and uses Gin and sqlite

Jan 19, 2022
Shopify Production Engineer Intern Challenge - Summer 2022

shopify-pe ---------- A tiny inventory management web-application. DESCRIPTION The API backend for this application is written in `go'. It handle

Jan 17, 2022
A simple CRUD app built with Go for Shopify Backend Developer Intern Challenge - Summer 2022

Shopify Backend Developer Intern Challenge - Summer 2022 A simple CRUD app built with Go and Gin. This let you update track your inventory, add new it

Jan 18, 2022
Course system - The project of Group 28 for ByteCamp 2022 Winter

Course System This is the project of Group 28 for ByteCamp 2022 Winter. Quick St

Jul 20, 2022
Vulnerability scanner for Spring4Shell (CVE-2022-22965)

go-scan-spring Vulnerability scanner to find Spring4Shell (CVE-2022-22965) vulnerabilities For more information: https://www.fracturelabs.com/posts/ef

Nov 9, 2022