Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Very small amount of sourcecode: Under 20k by my count. Are they outsourcing the actual crawl to a different tool?



SiteOne Crawler offers 2 open-source tools: 1) command-line version (which is the core of the crawler itself): https://github.com/janreges/siteone-crawler 2) a graphical application for Win/macOS/Linux, which is just a wrapper over the command-line version and displays the output from it through Xterm: https://github.com/janreges/siteone-crawler-gui




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: