Posts in Category: development

scrapy_and_aws

Installing scrapy and scrapyd on AWS EC2

See the updated version for installing scrapy 1.0 and above here.

This post will cover the basics of getting started with Amazon AWS, creating an account, creating an EC2 instance, installing scrapy and scrapyd and finally making sure you do it all for free!

Getting Started

Keeping It Free!

Before you go any

Read More

unnamed

ImportError: No module named settings

So after scratching my head for a while as to why my shiny new spider wasn’t doing what it was supposed to do I finally found this issue which hasn’t been merged in as yet.

Simple fix, do not call your scrapy project test!

Read More