General

Web Crawler tutorial video

SCM web crawler tool.

If you ever wanted to find all the links on a page (ie spider a site) you can do it inside SCM now.

SCM web crawler visits a web page using a real web browser so it can load dynamic content, finds links and follows them to find more links.

You can build a target list of article urls very easily by using this tool.