• QuazarOmega@lemy.lol
    link
    fedilink
    English
    arrow-up
    2
    ·
    17 hours ago

    I can’t recall which right now, but there are ones that manage to scrape the entire content by spoofing the Google crawler.
    Since websites want to maximise their SEO, they must provide the raw content to be indexed better