I added zerolog so that we can use leveled logging.

For gin, if it is an error we would still just use panic(err) so that the middleware can handle it, but for logging errors that doesn't cause a panic, use information and debug information, we can use leveled logging so that we know what type of log it is.

Another useful thing about this is that debug logs will show when we are in development but not when in production.

Fixes #9

User registration only

Example query

mutation {
  registerAccount(input: {email: "[email protected]", password: "helloworld"}) {
    __typename
    ... on LoginResult {
      accessToken
    }
  }
}

Contributes to #17

Added graphql schema generator and API server endpoint. ~~I have left the folder structure as default.~~

General workflow for developing with the gqlgen tool is as follow.

1. Make changes in pkg/graph/schema.graphqls.
2. Run make generate in repo root.
3. It should update the pkg/graph/schema.resolvers.go file.
4. If you deleted anything in step 1, it will probably throw an error, so go into pkg/graph/schema.resolvers.go and find the code to be deleted at the bottom, check that they are no longer required and deleted them. Then run make generate again.
5. Edit pkg/graph/schema.resolvers.go to add the database code or business logic.

Currently with go-colly, there is a potential issue where if 1 page fails to load then the data is considered corrupted because we need the whole set of data in order to determine which job listings are active or inactive.

There is no retry functionality in go-colly and error handling is not very useful.

I think it might be better for us to fetch the html as string (where we can have our own retry logic) then use an HTML parser to process the data instead.

This will be more similar to the logic of the scraper in the .NET version.

Get html node in go with css selector: https://github.com/PuerkitoBio/goquery

Retry: https://github.com/avast/retry-go

If we use the following

type Job struct {
    gorm.Model
    Title     string         
    Company   string     
    Salary    string         
    Location  string 
}

It will return the following JSON in API

{
  "ID" : 1,
  "CreatedAt" : "foo",
  "UpdatedAt" : "foo",
  "DeletedAt": "foo",
  "Title" : "foo",
  "Company" : "foo",
  "Salary" : "foo",
  "Location" : "foo",
}

Annotating the struct with json"field" won't lowercase the fields in gorm.Model

type Job struct {
    gorm.Model
    Title     string `json:"title"` 
    Company   string `json:"company"`
    Salary    string `json:"salary"`
    Location  string `json:"location"`
}

It will return the following JSON in API

{
  "ID" : 1,
  "CreatedAt" : "foo",
  "UpdatedAt" : "foo",
  "DeletedAt": "foo",
  "title" : "foo",
  "company" : "foo",
  "salary" : "foo",
  "location" : "foo",
}

To lowercase all the fields we can't use gorm.Model

type Job struct {
	ID        uint           `gorm:"primarykey" json:"id"`
	CreatedAt time.Time      `json:"created_at"`
	UpdatedAt time.Time      `json:"updated_at"`
	DeletedAt gorm.DeletedAt `gorm:"index" json:"deleted_at"`
	Title     string         `json:"title"`
	Company   string         `json:"company"`
	Salary    string         `json:"salary"`
	Location  string         `json:"location"`
}

It will return the following JSON in API

{
  "id" : 1,
  "created_at" : "foo",
  "updated_at" : "foo",
  "deleted_at": "foo",
  "title" : "foo",
  "company" : "foo",
  "salary" : "foo",
  "location" : "foo",
}

I prefer the above JSON fields since it's the common convention in API response. What do you think?

When we run the scraper cmd, it automatically create new jobs based on the jobs returned by the scrapers. Sometimes, the jobs is already exist in the DB, how do we prevent it to be inserted?

Some ideas, store the job links in db as well, then we query based on links before we insert the job to DB

• User should be able to specify filter parameters

  • keywords
  • location
  • salary

• Notify daily?

• Notification medium

  • web push
  • email
  • app push (our own standalone app or something like pushover?)

• [x] Create database schema

• [ ] Create system architecture

• [x] Create sequence diagram

• [ ] Implement

It's not an enjoyable development experience to keep on scraping the websites especially when you have unreliable internet connection. Suggestion is to have another cmd programme that seeds data to the DB. We can use export an SQL file from existing data and create a new cmd programme that imports the SQL file.

Salary reporting similar to glassdoor.

This would be a fairly big feature so the details should be discussed before development.

• Allow users to report salary anonymously.
• Include information such as sectors, industry, years of experience, fulltime/parttime.
• Show infographic with filters.

References:

• https://www.tokyodev.com/insights/2020-developer-survey/
• https://www.payscale.com/research/SG/Job=Software_Developer/Salary
• https://www.levels.fyi/