These are some pages that could be an interesting read (mostly for the page ranking later on in the process)
https://snap.stanford.edu/class/cs224w-readings/Brin98Anatomy.pdf
—> Google’s original research paper, including pagerank explanation
https://en.wikipedia.org/wiki/PageRank
—> wikipedias page about pagerank
https://storage.googleapis.com/gweb-research2023-media/pubtools/4449.pdf
https://en.wikipedia.org/wiki/MapReduce
—> distributed computing method for large datasets developed by google for pagerank calculation
https://en.wikipedia.org/wiki/Tf%E2%80%93idf
—> basic information retrieval ranking algorithm
https://en.wikipedia.org/wiki/Okapi_BM25
—> improved information retrieval ranking algorithm based on tf-idf
https://anvil.works/blog/how-to-build-a-search-engine
https://anvil.works/blog/search-engine-pagerank
https://anvil.works/blog/search-engine-tokenization-and-indexing
—> three part guided search engine build (1: basic scraping + content matching. 2: pagerank ranking. 3: indexing + tokenization for speed)
https://github.com/elastic/elasticsearch
—> very fast, scalable and distributable search framework
https://en.wikipedia.org/wiki/Google_Hummingbird
—> more google algorithm
https://en.wikipedia.org/wiki/HITS_algorithm
—> link analysis algorithm for page ranking
https://en.wikipedia.org/wiki/Ranking_(information_retrieval)
—> even more page ranking stuff