OnionSearch – A Script That Scrapes Urls On Different .Onion Search Engines

From kitploit.com

Multi-processing behaviour

By default, the script will run with the parameter mp_units = cpu_count() - 1. It means if you have a machine with 4 cores, it will run 3 scraping functions in parallel. You can force mp_units to any value but it is recommended to leave to default. You may want to set it to 1 to run all requests sequentially (disabling multi-processing feature).

Please note that continuous writing to csv file has not been heavily tested with multiprocessing feature and therefore may not work as expected.

Please also note that the progress bars may not be properly displayed when mp_units is greater than 1. It does not affect the results, so don’t worry.

Read more…