CDN for the Tech With Tim website using Go

Project logo

Tech With Tim - CDN

Status
codecov Lint & Test GitHub Issues GitHub Pull Requests Licence Discord

CDN for the Tech With Tim website using Go

๐Ÿ“ Table of Contents

๐Ÿ Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See Running with Docker if you want to setup the CDN faster with Docker. ( Docker is optional )

Environment variables

Set the environment variables. Start by writing this in a file named app.env and test.env: (test.env is required for running tests)

app.env and test.env should look like this:

DB_URI=postgres://user:password@localhost:5432/dbname?sslmode=disable
SECRET_KEY=secret
MAX_FILE_SIZE=30
  • SECRET_KEY is the key used for the JWT token encoding.
  • MAX_FILE_SIZE is the maxiumum file size allowed in asset upload (in mb)

Running

  • To create postgres container make postgres
  • To create db make createdb
  • To drop db make dropdb

run go mod tidy to install packages

cli commands

go run main.go migrate_up
go run main.go dropdb
go run main.go migrate_steps --steps int
go run main.go runserver --host localhost --port port (localhost, 5000 are default)

to run migrations on the test database

go run main.go migrate_up -t
go run main.go dropdb -t
go run main.go migrate_steps -t --steps int

Use the make file its your best friend ๐Ÿ› 

Make commands

If you are on windows please use git bash or wsl also you would have to install make for windows
to install make for windows run winget install GnuWin32.Make
make postgres #creates docker container for postgres12
# reads env variables from app.env
make createdb #creates the db in the postgres container
make dropdb #drops the db
make migrate_up #migrates to the latest schema
make sqlc_generate #generates sqlc code if you write queries
make test # tests your code and shows coverage
#its a big output make sure to read it all

๐Ÿณ Running with Docker

Start the cdn docker-compose up

๐Ÿšจ Tests

To test the cdn we can use two methods

make test

If you don't have make installed

go run main.go migrate_up -t
go test ./... -v 

When you contribute, you need to add tests on the features you add.

โ›๏ธ Built Using

  • Go - Language
  • go-chi - Router
  • sqlc - Database Query Helper

โœ๏ธ Authors

See the list of contributors who participated in this project.

Comments
  • Attempt to achieve #1

    Attempt to achieve #1

    Summary

    1. Some small code cleaning because my IDE doesn't like the old code :( Feel free to tell me to undo it if you don't like the change.
    2. Attempt to create endpoint /manage/url and /manage/id.

    Example: GET http://localhost:5000/manage/url/aa.png => {"id":6807691003781186055,"name":"bruh"}

    Checklist

    • [x] If endpoints were changed then they have been documented and tested.
      • [ ] I have updated the docmentation to reflect the changes.
      • [x] I have updated the tests to support the changes.
    • [x] This PR fixes an issue.
    • [x] This PR adds something new (e.g. new endpoint or parameter).
    • [ ] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • Added documentation to the cdn

    Added documentation to the cdn

    Summary

    I added docs to the cdn. The docs are hosted on / and made with Svelte. To generate docs - make generate_docs

    Checklist

    • [x] If endpoints were changed then they have been documented and tested.
      • [x] I have updated the docmentation to reflect the changes.
      • [ ] I have updated the tests to support the changes.
    • [ ] This PR fixes an issue.
    • [x] This PR adds something new (e.g. new endpoint or parameter).
    • [ ] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • Migrate to DB_URL, add caching in dockerbuilds

    Migrate to DB_URL, add caching in dockerbuilds

    Summary

    Checklist

    • [ ] If endpoints were changed then they have been documented and tested.
      • [ ] I have updated the docmentation to reflect the changes.
      • [ ] I have updated the tests to support the changes.
    • [ ] This PR fixes an issue.
    • [x] This PR adds something new (e.g. new endpoint or parameter).
    • [x] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • Bug Fix: docs

    Bug Fix: docs

    Summary

    Checklist

    • [ ] If endpoints were changed then they have been documented and tested.
      • [ ] I have updated the docmentation to reflect the changes.
      • [ ] I have updated the tests to support the changes.
    • [x] This PR fixes an issue.
    • [ ] This PR adds something new (e.g. new endpoint or parameter).
    • [ ] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • Fix bug in docs

    Fix bug in docs

    Summary

    Fixes bug in the docs page, where the Request Body parameter is shown even if there are none

    Checklist

    • [x] If endpoints were changed then they have been documented and tested.
      • [ ] I have updated the docmentation to reflect the changes.
      • [ ] I have updated the tests to support the changes.
    • [x] This PR fixes an issue.
    • [ ] This PR adds something new (e.g. new endpoint or parameter).
    • [ ] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • docs

    docs

    Summary

    Checklist

    • [ ] If endpoints were changed then they have been documented and tested.
      • [ ] I have updated the docmentation to reflect the changes.
      • [ ] I have updated the tests to support the changes.
    • [ ] This PR fixes an issue.
    • [ ] This PR adds something new (e.g. new endpoint or parameter).
    • [ ] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • Changed from dgrijalva/jwt-go to github.com/golang-jwt/jwt

    Changed from dgrijalva/jwt-go to github.com/golang-jwt/jwt

    Summary

    To fix security issue in dgrijalva/jwt-go

    Checklist

    • [ ] If endpoints were changed then they have been documented and tested.
      • [ ] I have updated the docmentation to reflect the changes.
      • [ ] I have updated the tests to support the changes.
    • [x] This PR fixes an issue.
    • [ ] This PR adds something new (e.g. new endpoint or parameter).
    • [ ] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • Migration To use SubModules

    Migration To use SubModules

    Summary

    Checklist

    • [ ] If endpoints were changed then they have been documented and tested.
      • [ ] I have updated the docmentation to reflect the changes.
      • [ ] I have updated the tests to support the changes.
    • [ ] This PR fixes an issue.
    • [x] This PR adds something new (e.g. new endpoint or parameter).
    • [x] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • Create Get: /manage/{id/url}

    Create Get: /manage/{id/url}

    Summary

    Create New Endpoint GetAssetDetails

    doesn't require authentication
    use already created functions:
    • store.GetAssetDetailsByUrl
    • store.GetAssetDetailsById
  • Docs dev branch

    Docs dev branch

    Summary

    Pull Upstream changes

    Checklist

    • [ ] If endpoints were changed then they have been documented and tested.
      • [ ] I have updated the docmentation to reflect the changes.
      • [ ] I have updated the tests to support the changes.
    • [ ] This PR fixes an issue.
    • [x] This PR adds something new (e.g. new endpoint or parameter).
    • [ ] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • logs

    logs

    Summary

    Checklist

    • [ ] If endpoints were changed then they have been documented and tested.
      • [ ] I have updated the docmentation to reflect the changes.
      • [ ] I have updated the tests to support the changes.
    • [ ] This PR fixes an issue.
    • [ ] This PR adds something new (e.g. new endpoint or parameter).
    • [ ] This PR is a breaking change (e.g. endpoint or parameters removed/renamed)
    • [ ] This PR is not a code change (e.g. documentation, README, ...)
  • Migrate to s3

    Migrate to s3

    What we want to do is strip down the whole CDN, and make it for uploading files only, to an s3 bucket. After Uploading we should store the s3 asset URL in the assets table, we have to remove the files table as it would be useless now. Tasks:

    • [ ] Remove Get Asset Endpoints
    • [ ] Use s3 SDK to upload files instead of database
    • [ ] Clean up files remove models, tables, write a migration to delete table
    • [ ] Migration to add an asset URL field
    • [ ] Complete List assets and other endpoints.
  • Use file hash as asset keys

    Use file hash as asset keys

    Current when storing assets in the db, the asset uses a random string or a string passed by the POST request. This should be changed to use the file hash as the asset key instead.

  • Create Get: ListAssetByCreator

    Create Get: ListAssetByCreator

    Summary

    Create New Endpoint Get: ListAssetByCreator Checkpoints:

    • [x] Add Pagination
    • [ ] Use Already Created Function store.ListAssetByCreator (maybe you would have to change the query ping me in the server if we do)
    • [ ] Requires User_id
Cdn - CDN microservice to upload files to zachlatta.com that only accepts traffic from Tailscale IPs

cdn CDN microservice to upload files to zachlatta.com that only accepts traffic from Tailscale IPs. source code available at https://github.com/zachla

Jun 26, 2022
Access more HTTP ports over CDN with this application.
Access more HTTP ports over CDN with this application.

More-Ports More Ports is a proxy service to establish all web-based applications on different ports on the server-side over a well known TCP port. It

May 8, 2022
CDN Node

CDN Node Init ็”Ÿ็”ข็’ฐๅขƒไผบๆœๅ™จ่ซ‹ไฝฟ็”จ Ctrl Server Output ็š„ install.sh ๅฎ‰่ฃ Dev go get github.com/gin-gonic/gin go get github.com/prometheus/common/log go get gopkg.i

Dec 20, 2021
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.

Thank you for your interest in ZASentinel ZASentinel helps organizations improve information security by providing a better and simpler way to protect

Nov 1, 2022
Titan CDN+ network

Titan CDN+ Titan is a Filecoin extension, backed by Filecoin and focused on last-mile CDN+ networks. Currently, Titan's goals are: Focus on the last m

May 4, 2023
An autoclaimer for the website We Heart It

weheartit-claimer An autoclaimer for the website We Heart It Table of contents Setting up the claimer Understanding the configuration files Setup Firs

May 26, 2021
Access my website from the terminal with SSH!
Access my website from the terminal with SSH!

daniel.is-a.dev (ssh version) What is this? I built a SSH server written in Golang that lets you basically view my website all from the terminal. You

Nov 11, 2022
Simple website fingerprinting tool

onefinger ไธ€ไธช็ฎ€ๅ•็š„ๆŒ‡็บน่ฏ†ๅˆซๅทฅๅ…ท๏ผŒ่ง„ๅˆ™ๆๅ–่‡ชgobyๅนถ็จไฝœไฟฎๆ”น๏ผŒๆŸไธช็Ÿฅ่ฏ†ๆ˜Ÿ็ƒ็š„ไฝœไธš Usage Usage: onefinger [-v] | (-t=<target>... | --tf=<targetFile>) [--threads=<threads>] [--timeout=<ti

Mar 16, 2022
A simple project which enables you to check wheather the website is up
A simple project which enables you to check wheather the website is up

Website Status Checker A simple project which enables you to check wheather the website is up and working fine or is the website has been crashed. Abo

Nov 8, 2021
Podcast RSS feed sharing website

Yarr A website for storing, sharing and viewing podcasts in RSS format. Powering yarr.ps Building go build --tags "fts5" TODO Add ability to add pods

Nov 6, 2021
Forms814 - A website builder, useful for writing data collection webapps quickly.
Forms814 - A website builder, useful for writing data collection webapps quickly.

forms814 A website builder, useful for writing data collection webapps quickly. Project Design The method in use here is to mix it with complicated fo

Oct 25, 2022
WebsiteChecker - A GoLang program that verifies if a website is available or not

Website Checker ?? This is my first program in Go! The purpose of this applicati

Apr 30, 2022
Back end of e-books and papers collection website. Only for internal use.

Back end of e-books and papers collection website. Only for internal use.

Nov 1, 2022
Turning a folder into a gallery website to share your videos, songs and images in the local netwrok.

What Carp Web Gallery Turning a folder into a gallery website to share your videos, songs and images in the local netwrok. Browser videos and audios i

Mar 14, 2022
serve a static website as a .onion hidden service

hidden service server A CLI that will host a static website as a .onion hidden service. Comes with an additional binary that can be used to generate v

Sep 4, 2022
Code for the website / Cรณdigo para o site

Site crdpa.net Code for the new website This is the frontend and backend code for my new website. It is for my use only. If you want to use it, you ar

May 11, 2022
This service is intented to collect data using grpc using Go lang backend and cassandra DB as storage

Go Data Collection This service is intented to collect data using grpc using Go lang backend and cassandra DB as storage. Dev Setup make test_env make

Apr 13, 2022
A library to simplify writing applications using TCP sockets to stream protobuff messages

BuffStreams Streaming Protocol Buffers messages over TCP in Golang What is BuffStreams? BuffStreams is a set of abstraction over TCPConns for streamin

Dec 13, 2022
Pure-Go library for cross-platform local peer discovery using UDP multicast :woman: :repeat: :woman:
Pure-Go library for cross-platform local peer discovery using UDP multicast :woman: :repeat: :woman:

peerdiscovery Pure-go library for cross-platform thread-safe local peer discovery using UDP multicast. I needed to use peer discovery for croc and eve

Jan 8, 2023