GitHub for Non-Coders, Part 1

Get started contributing to City Scrapers with only a GitHub account! No installation necessary.

Follow these 4 steps: (1) find an issue, (2) fork the code, (3) edit code, (4) open a pull request.

GitHub for Non-Coders, Part 2

You opened a pull request, but the checks are failing!  This video walks through how to spot and resolve common error.

Hint: If you update a spider, you will have to update the corresponding test.

How to Run a Spider (Scraper)

You installed everything and set up my environment—now what? This video walks through running a City Scraper spider on your own computer.

Note: Skip to 2:12 for a general explanation of how a spider works.

Test Code with Scrapy Shell

You've decided to build a new spider—what tools do you need to know? This video introduces `scrapy shell,` an interactive shell where you can test your code without running the entire spider.

We also walk through parsing a date from a website.

Build and Use Scrapy Pipelines

We use pipelines to clean up, geocode and store events. This video explains how pipelines work and how they are organized in the City Scrapers project. Pipelines are a good place to start if you’re interested in building infrastructure for the project (as opposed to spiders).