menu

Connection



Port is HTTP port to access the sites. 80 is standard value and should not be changed


Support robot exclusion protocol (robots.txt and Robots META tag) - if this box is checked the Atomic Lead Extractor will not process the pages disabled in a file "robots.txt" located on the processed web server. See more about file robots.txt.


Custom loading URL limits (to avoid overloading domain):


Common loading thread count ... threads - is a number of simultaneousely procesed pages. 5 is a default value. If your Internet connection is fast and your computer is modern then you may increase it, otherwise decrease it to optimize the search speed.


1 request per ... seconds - is time period when Atomic Lead Extractor waits the requetsed page. Increase the value if your Internet connection is too slow.


Identify as (user agent) is a value that Atomic Lead Extractor will use when it loads pages from the sites it will search. The recommended value is your current Internet browser, so the sites will think what you are browsing them, but not analizyng the data.




See also:


Working with Atomic Lead Extractor


Registration details


Customer support