Journalist. An RSS aggregator.

journalist

Docker Cloud Build Status

Journalist. An RSS aggregator.

Download the latest version for macOS, Linux, FreeBSD, NetBSD, OpenBSD & Plan9 here.

WARNING: journalist is highly experimental software and not ready for use. Don't rely on it and expect changes in data structures, with no possibility to migrate existing data at the moment.

What is journalist?

Get more information here.

Repository

This repository contains the source code of journalist. The code is being actively developed in the develop branch and only merged into master and tagged with a version as soon as it's stable enough for a release.

If you intend to create PRs, please do so against develop.

Build

make

Info: This will build using the version 0.0.0. You can prefix the make command with VERSION=x.y.z and set x, y and z accordingly if you want the version in journalist --help to be a different one.

Usage

\

Please make sure to export JOURNALIST_DB="postgres://postgres:[email protected]:5432/journalist" or however your PostgreSQL connection string might look.

You can change the log level using the JOURNALIST_LOG_LEVEL env variable, e.g. JOURNALIST_LOG_LEVEL=10 for debug. By default, the level is set to warn.

Database

Journalist requires you to have your own PostgreSQL database running somewhere. Running it can be as easy as this, in case you're using Docker:

docker run -it --name postgres \
  -e POSTGRES_PASSWORD=postgres \
  -e POSTGRES_DB=journalist \
  -p 127.0.0.1:5432:5432 \
  -d postgres:alpine

There are also plenty of cloud platforms where you can easily launch a fully- managed PostgreSQL instance. For example on DigitalOcean you can get a PostgreSQL single node cluster for as little as $15 per month.

CLI

The journalist binary is a daemon as well as a CLI client for managing subscriptions and service configuration. There is no way (yet) to manage subscriptions through the Fever API / a RSS client connecting to it.

Listing subscriptions

You can list all feeds a user is subscribed to using the subscriptions command:

JOURNALIST_LOG_LEVEL=10 \
JOURNALIST_DB=postgres://postgres:[email protected]:5432/journalist \
journalist subscriptions

By default this command would list subscriptions for the user nobody (password: nobody). It's possible to specify -u (username) and -p (password) flags in order to list subscriptions for an individual account.

Subscribing to a feed

You can subscribe to a feed by using the subscribe command:

JOURNALIST_LOG_LEVEL=10 \
JOURNALIST_DB=postgres://postgres:[email protected]:5432/journalist \
journalist subscribe https://xn--gckvb8fzb.com/index.xml -g "Cool People"

-g adds the feed to a custom group, in this case Cool People. Groups are automatically created when specified via -g.

By default this command would subscribe as the user nobody (password: nobody). It's possible to specify -u (username) and -p (password) flags in order to subscribe to a feed under an individual account.

Unsubscribing from a feed

You can unsubscribe from a feed by using the unsubscribe command:

JOURNALIST_LOG_LEVEL=10 \
JOURNALIST_DB=postgres://postgres:[email protected]:5432/journalist \
journalist unsubscribe https://xn--gckvb8fzb.com/index.xml

If the feed was the last one in its group, the group is also being removed.

By default this command would unsubscribe as the user nobody (password: nobody). It's possible to specify -u (username) and -p (password) flags in order to unsubscribe from a feed under an individual account.

Running the RSS server

In order to be able to connect using any Fever API capable client you'll need to run journalist in server mode:

JOURNALIST_LOG_LEVEL=10 \
JOURNALIST_DB=postgres://postgres:[email protected]:5432/journalist \
journalist server

You can then connect to it using your favourite Fever API client (e.g. Reeder for macOS/iOS). Simply specify http://localhost:8000/fever/ (or the machine you're running journalist server on) and either use nobody and nobody as credentials or – if you've subscribed to feeds using custom credentials – use your own.

Environment Variables

General (CLI & server)

  • JOURNALIST_LOG_LEVEL: The log level, 0 being the lowest, 10 the highest
  • JOURNALIST_DB: The PostgreSQL connection string

Server only

  • JOURNALIST_SERVER_BINDIP: The IP to bind the server to, default: 0.0.0.0
  • JOURNALIST_SERVER_PORT: The port the server should run on, default: 8000
  • JOURNALIST_SERVER_REFRESH: The refresh interval (in seconds) at which the server should update subscriptions, default: 0 (disabled)
  • JOURNALIST_SERVER_API_FEVER: The Fever API, boolean value, default: true (enabled)
  • JOURNALIST_SERVER_API_GREADER: The Google Reader API, boolean value, default: false (disabled) NOT YET AVAILABLE

Docker

Official images are available on Docker Hub at mrusme/journalist and can be pulled using the following command:

docker pull mrusme/journalist

GitHub release versions are available as Docker image tags (e.g. 0.0.1). The latest image tag contains the latest code of the main branch, while the develop tag contains the latest code of the develop branch.

It's possible to build journalist locally as a Docker container like this:

docker build -t journalist:latest . 

It can then be run using the following command:

docker run -it --rm --name journalist \
  -e JOURNALIST_LOG_LEVEL=10 \
  -e JOURNALIST_DB="postgres://postgres:[email protected]:5432/journalist" \
  -p 0.0.0.0:8000:8000
  journalist:latest
Similar Resources

yet another rss reader

yet another rss reader

yarr yarr (yet another rss reader) is a web-based feed aggregator which can be used both as a desktop application and a personal self-hosted server. I

Dec 29, 2022

golang rss/atom generator library

gorilla/feeds feeds is a web feed generator library for generating RSS, Atom and JSON feeds from Go applications. Goals Provide a simple interface to

Dec 26, 2022

A telegram bot that fetches multiple RSS cryptocurrency news feeds for sentiment analysis

Crypto News Telegram Bot A simple telegram bot that will help you stay updated on your latest crypto news This bot will help you keep track of the lat

Aug 22, 2021

Podcast RSS feed sharing website

Yarr A website for storing, sharing and viewing podcasts in RSS format. Powering yarr.ps Building go build --tags "fts5" TODO Add ability to add pods

Nov 6, 2021

terminal rss reader

TODO Add author field [x] Add Source as own type to support both http sources and file types sources [x] Move Load http call to own method that Load m

Nov 7, 2021

This command line converts thuderbird's exported RSS .eml file to .html file

thunderbird-rss-html This command line tool converts .html to .epub with images fetching. Install go get github.com/gonejack/thunderbird-rss-html Us

Dec 15, 2021

Colored RSS feeds in your console

RSS Console Feed Read colored rss feeds in your console Usage ./rss-console-feed

Dec 22, 2021

Read data from rss, convert in pdf and send to kindle. Amazon automatically convert them in azw3.

Kindle-RSS-PDF-AZW3 The Kindle RSS PDF AZW3 is a personal project. The Kindle RSS PDF AZW3 is a personal project. I received a Kindle for Christmas, a

Jan 10, 2022

Watches container registries for new and changed tags and creates an RSS feed for detected changes.

Tagwatch Watches container registries for new and changed tags and creates an RSS feed for detected changes. Configuration Tagwatch is configured thro

Jan 7, 2022

Startpage - Lambda for reading rss feeds and generating a minimal start page for a static site

startpage generate a startpage of links for a static site hosted via AWS What It

Sep 6, 2022
Comments
  • ent fails: Immutable undefined (type *edge.Descriptor has no field or method Immutable)

    ent fails: Immutable undefined (type *edge.Descriptor has no field or method Immutable)

    $ make
    fatal: No names found, cannot describe anything.
    ent generate --feature sql/upsert ./ent/schema
    entc/load: # command-line-arguments
    .entc/entc_github.com_mrusme_journalist_ent_schema_1667657943.go:115:19: ed.Immutable undefined (type *edge.Descriptor has no field or method Immutable)
    
    make: *** [Makefile:7: ent] Error 1
    

    Ubuntu 22.10 amd64 go 1.19.2

  • panic: interface conversion: interface {} is nil, not string

    panic: interface conversion: interface {} is nil, not string

    Retrieving a feed by id - eg. /api/v1/feeds/8166cabe-1061-4809-8026-ad401579d47e - crashes the server:

    {"level":"info","ts":1667660273.543035,"caller":"fiberzap/fiberzap.go:85","msg":"api.request","context":{"pid":"1343134","time":"12.856µs","response":{"type":"text/plain; charset=utf-8","code":204},"request":{"fullPath":"/api/v1/feeds/8166cabe-1061-4809-8026-ad401579d47e","ip":"127.0.0.1","method":"OPTIONS","route":"/api","headers":{"user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0"}}}}
    panic: interface conversion: interface {} is nil, not string
    
    goroutine 33 [running]:
    github.com/mrusme/journalist/api/v1/feeds.(*handler).Show(0xc0003e4d60, 0xc00022c580)
            /home/yonas/git/journalist/api/v1/feeds/show.go:51 +0xbf8
    github.com/gofiber/fiber/v2.(*App).next(0xc000550f00, 0xc00022c580)
            /home/yonas/go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:132 +0x1c2
    github.com/gofiber/fiber/v2.(*Ctx).Next(0xc000878e50?)
            /home/yonas/go/pkg/mod/github.com/gofiber/fiber/[email protected]/ctx.go:803 +0x53
    github.com/mrusme/journalist/api.authorizer.func1(0x1?)
            /home/yonas/git/journalist/api/api.go:89 +0x4b2
    github.com/gofiber/fiber/v2.(*Ctx).Next(0xc0001c8f18?)
            /home/yonas/go/pkg/mod/github.com/gofiber/fiber/[email protected]/ctx.go:800 +0x43
    github.com/gofiber/fiber/v2/middleware/cors.New.func1(0xc00022c580)
            /home/yonas/go/pkg/mod/github.com/gofiber/fiber/[email protected]/middleware/cors/cors.go:141 +0x286
    github.com/gofiber/fiber/v2.(*App).next(0xc000550f00, 0xc00022c580)
            /home/yonas/go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:132 +0x1c2
    github.com/gofiber/fiber/v2.(*Ctx).Next(0xc000498f50?)
            /home/yonas/go/pkg/mod/github.com/gofiber/fiber/[email protected]/ctx.go:803 +0x53
    github.com/mrusme/journalist/middlewares/fiberzap.New.func1(0xc00022c580)
            /home/yonas/git/journalist/middlewares/fiberzap/fiberzap.go:54 +0x156
    github.com/gofiber/fiber/v2.(*App).next(0xc000550f00, 0xc00022c580)
            /home/yonas/go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:132 +0x1c2
    github.com/gofiber/fiber/v2.(*App).handler(0xc000550f00, 0x49ea17?)
            /home/yonas/go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:159 +0x45
    github.com/valyala/fasthttp.(*Server).serveConn(0xc000153200, {0x13e3280?, 0xc000014008})
            /home/yonas/go/pkg/mod/github.com/valyala/[email protected]/server.go:2305 +0x1248
    github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc000549040, 0xc00004e020)
            /home/yonas/go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:224 +0xa9
    github.com/valyala/fasthttp.(*workerPool).getCh.func1()
            /home/yonas/go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:196 +0x38
    created by github.com/valyala/fasthttp.(*workerPool).getCh
            /home/yonas/go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:195 +0x1b0
    
  • Journalist v1

    Journalist v1

    Journalist v1

    This is a complete rewrite of Journalist, using https://github.com/gofiber/fiber and https://github.com/ent/ent and focusing on a pure RSS experience while still allowing for multi-client synchronization.

    More information will soon become available here, stay tuned!

yarr (yet another rss reader) is a web-based feed aggregator
yarr (yet another rss reader) is a web-based feed aggregator

yarr (yet another rss reader) is a web-based feed aggregator which can be used both as a desktop application and a personal self-hosted server.

Jan 2, 2023
RSS master is a RSS subscription function aggregation tool, You can use it easily!

???? 中文 rsm ✨ RSS master[rsm] is a RSS subscription function aggregation tool, You can use it easily! How to start? ?? Start rsm with rsm run -c,--cfg

Oct 18, 2022
Svenska-yle-rss-content-fixer - Attach content to Svenska Yle RSS feeds

svenska-yle-rss-content-fixer This little tool attaches article content to the S

Oct 4, 2022
egg - the simple error aggregator

egg - the simple error aggregator egg ingests errors and aggregates them egg has 1st class support for sentry SDKs so you dont have to change any code

May 1, 2022
Run The World. Command aggregator output. Define many services watch them in one place.

Run The World. Command aggregator output. Define many services watch them in one place.

Feb 2, 2022
A lightweight, cloud-native data transfer agent and aggregator
A lightweight, cloud-native data transfer agent and aggregator

English | 中文 Loggie is a lightweight, high-performance, cloud-native agent and aggregator based on Golang. It supports multiple pipeline and pluggable

Jan 6, 2023
A Go library for fetching, parsing, and updating RSS feeds.

rss RSS is a small library for simplifying the parsing of RSS and Atom feeds. The package could do with more testing, but it conforms to the RSS 1.0,

Dec 14, 2022
Parse RSS, Atom and JSON feeds in Go
Parse RSS, Atom and JSON feeds in Go

gofeed The gofeed library is a robust feed parser that supports parsing both RSS, Atom and JSON feeds. The library provides a universal gofeed.Parser

Jan 8, 2023
iTunes and RSS 2.0 Podcast Generator in Golang

podcast Package podcast generates a fully compliant iTunes and RSS 2.0 podcast feed for GoLang using a simple API. Full documentation with detailed ex

Dec 23, 2022
Monitor newznab/torznab rss and add new media to sonarr/radarr

Nabarr Nabarr monitors Newznab/Torznab RSS feeds to find new media to add to Sonarr and or Radarr. Table of contents Installing nabarr Introduction Me

Dec 28, 2022