June 2024 Changelog
June went into scraping. Dynamic Scraper stopped mangling its own CSV files, Google Maps stopped walking past contact details it had already loaded, and exporting to Google Sheets stopped asking for access to your whole Drive.
The other half of the month went into the API, so you can find, start and stop jobs from your own scripts without taking the app down when something is missing.
Read this if you
- Scrape with Dynamic Scraper and open the CSV afterwards.
- Pull business contacts out of Google Maps.
- Export results to Google Sheets.
- Drive SCM from your own scripts through the API.
- Run Article Creator jobs that download images.
Dynamic Scraper stopped ruining its own CSV
Adding results to an existing CSV used to corrupt the file, and any scraped text containing a quote mark broke the columns. The file you open in a spreadsheet now matches what you scraped.
- Quotes inside scraped text stop splitting a cell across the wrong columns.
- Appending to a CSV you already have keeps it readable instead of corrupting it.
- Blank rows are dropped, and the click and next page steps stay out of the file.
- In the preview, no column takes more than a third of the table and cells cut off past 257 characters, so one long paragraph stops pushing the rest off screen.
- Row numbers have room to show.
One run can walk through every page of results
- Point the scraper at the next page link and it keeps going, so you collect the whole result set in one run instead of ten.
- Write %page% into a url and SCM numbers the pages for you.
- A new page url action hands you the address of the page you are on, useful when the results themselves carry no links.
- Move the picker out of the way when it sits on top of the thing you are trying to select.
- Sample values refresh when you load or navigate a page, so what you see is what the rule will actually grab.
- Hammering the reload button no longer fires off Google search after Google search in the background.
Google Maps brings back the contacts
- Results carry the business name again.
- The scraper tries the business site directly for social profiles first, then falls back to the homepage, so it finds profiles it used to miss.
- It stops giving up on the contact hunt early, so more listings come back with an email or a phone number.
- Choose which pages get searched for contacts, whether the sites you target use contact, about, or something else.
- The cookie consent screen gets clicked for you, so a run does not stall on it.
- Exports to Google Sheets no longer arrive with quote marks wrapped around every cell.
- You can send Google Maps results straight to a Google Sheet.
Google Sheets export asks for less
- SCM only asks for access to your spreadsheets now. Linking your whole Google Drive is no longer part of setup.
- Pick a sheet through Google's own file picker in your browser.
- Refresh the list of sheets without restarting, and export to any tab, not just the first one.
- If access was never granted properly, you get told, instead of an export that quietly does nothing.
You can drive SCM from your own scripts
- Three new API calls: find a project, read its blog settings, and abort a running project.
- Deleting a task that is already gone gives you an answer instead of crashing the app.
- A call that arrives without your API key gets refused rather than taking the app down with it.
- If you moved your content cache, the API tells you where it actually is.
Article Creator and the rest
- Article Creator stopped crashing on the image download step, and downloaded images get written to disk.
- The AI outline uses your %title% macro when the article has one, so the outline matches the title you asked for.
- Word and line counts are back in the article preview.
- The Google News insert is gone. It had stopped returning anything, so leaving it in only produced empty sections.
- Video scraper: search inside a channel, paste a YouTube url directly, and get the number of results you asked for.
- Quora questions return results again.
- Search and news scraping run on several threads, so a long keyword list finishes sooner.
- The post uploader takes a hotlinked image as the featured image, with no download step first.
- An update that fails to install says so instead of quietly not restarting.
- Fixed: OpenAI SSL and 403 errors, Groq rate limit handling, Claude max token limits, and Google search results that sat waiting forever.
Related SCM guides and product history
Full developer changelog
The summary above is written for users. The original 50-item developer changelog is preserved below for exact release detail.
July 1, 2024
- fix: Google maps not returning name of places
June 27, 2024
- fix: SCM API delete task doesn't crash if task doesn't exist already
- fix: SCM api returning wrong content cache path if you moved it
- fix: Error scm api would crash if api key set but calls did not include key
- feat: Added new api calls, project/find /blog-settings /project/abort
June 25, 2024
- fix: Claude ai max tokens
- fix: Dynamic scraper, csv output preview col width no larger than 33% of table width, trim long output to 257 chars
- style: Dynamic scraper csv preview, row number cell given more room
June 24, 2024
- fix: Dynamic scraper, escape double quotes with double quotes
- feat: Dynamic scraper can click on next page links to find extra results
- feat: dynamic scraper, dont write click and next page actions to csv
June 23, 2024
- fix: Dynamic scraper, csv output append to file corrupting output
- fix: Image downloader not saving files to disk
- fix: Google maps scraper, exiting too early when looking for contacts
- feat: Google maps, customize which pages to search for contacts
June 22, 2024
- fix: Show error message when updater has problems installing and restarting the App
- feat: Google maps will go direct to url of any site first to find socials. It will then go to the domain page if not found
- fix: Google maps sheet export had double quotes in all cells
- feat: Dynamic scraper picker position can be changed
June 21, 2024
- fix: CSV output, added escaping double quote chars
- fix: Dynamic scraper not updating rule samples on page load and navigation
- feat: Dynamic scraper, new action 'page url' returns current url of page
- fix: Google sheets, check that access to sheets was properly granted
June 20, 2024
- feat: Google sheet export allow refresh dropdown of sheets
- feat: Dynamic scraper, quickly generate url with paging numbers with %page% macro
- fix: Dynamic scraper, spamming reload page button wont cause infinite google search calls
- fix: Export google sheets error if not selecting first sheet in spreadsheet
- other: Remove requirement to link google drive to SCM, it only requires access to spreadsheets only.
- other: Google sheet selection via Google file picker in native browser
- fix: Video scraper, not capping results correctly
- feat: Find videos using direct url from youtube
June 18, 2024
- fix: Unable to use google search results. stuck waiting
June 16, 2024
- feat: Post uploader, allow hotlinked images as source for featured image uploading
June 15, 2024
- fix: Auto click consent screen in Google maps scraper
- fix: groq rate limit error code fix
June 14, 2024
- feat: Article creator, ai outline will use the %title% macro if available
- fix: open ai ssl errors
- fix: Potential fix for openai 403 error and SCM API generic error
June 13, 2024
- fix: Article creator crashing on image download step
June 12, 2024
- other: SCM api err logging
- feat: Channel searching inside video scraper
June 9, 2024
- feat: Add threads for search and news scraper
June 8, 2024
- fix: article creator images, window undefined
June 7, 2024
- other: Article creator remove google news insert as it doesn't work anymore
- feat: Article preview, add back word/line count for preview
June 6, 2024
- fix: Dynamic scraper, fix 'detect article' action causing error
June 5, 2024
- feat: Google maps scraper export to google sheet
- feat: Email notify test send button added
- feat: Dynamic scraper, csv output will remove blank rows
- fix: Quora not returning results for keyword question
See all monthly updates in the SCM changelog archive. New users can also try SCM from the trial signup page.