Reading through the paper about WPR it is hard to tell how you would decide to weight a given graph.Link(n1, n2, ???). When should a node be given more weight?

I saw you changed the project to use uint32 which increases the number of nodes from 2 billion to 4 billion. Can we also reduce the memory usage by changing float64 to float32 for the graph value? float32 can hold a number big enough.

Update: I'm going to go ahead and just test it: https://github.com/Xeoncross/pagerank/commit/66df7eaf707098c9a7be0cb1e44ab303819e8273

In example below order of ranks is depends on order of internal map(map of edges). But for same input i expect deterministic output. How do i reach this?

package main

import (
	"fmt"

	"github.com/alixaxel/pagerank"
)

const (
	damping   = 0.85
	tolerance = 0.0001
)

type Rank struct {
	idx   int
	score float64
}

func main() {
	g := pagerank.NewGraph()

	g.Link(0, 1, 0.33333)
	g.Link(1, 2, 0.33333)
	g.Link(2, 0, 0.33333)

	var ranks []int
	g.Rank(damping, tolerance, func(sentenceIndex uint32, rank float64) {
		ranks = append(ranks, int(sentenceIndex))
	})

	var ranks2 []int
	g.Rank(damping, tolerance, func(sentenceIndex uint32, rank float64) {
		ranks2 = append(ranks2, int(sentenceIndex))
	})

	fmt.Printf("%v\n", eq(ranks, ranks2)) // ! sometimes false !
}

func eq(l, r []int) bool {
	if len(l) != len(r) {
		return false
	}

	for i := range l {
		if l[i] != r[i] {
			return false
		}
	}

	return true
}

The uint can handle values past 4 billion so that means this system can handle graphs with up to 4294967295 nodes in them. Currently, a graph with only 1,000,000 nodes ( with 100,000,000 links) takes 2.6GB of memory to calculate. Approaching 1 billion nodes (with 10 or 100 links each) would take a massive server to calculate.

Is there any way we could split the processing up by sharding the graph into smaller parts rather than processing the whole thing in memory? Perhaps we could even combine the graph.edges and graph.nodes weight values since they contain duplicate weights (or swap them for pointers to a shared float64 value).